Hello, I'm Kévin (@zecakeh), and I'm here to tell you about some of my recent contributions to EteSync Notes.

Here I was, back in December, still trying to find a good notes app. Those were my criteria:

• Open source
• Available on Android and Linux (or be compatible with other apps)
• Sync easily between my devices
• Nice look
• Have simple styling of notes
• Support simple to-do lists

And the extras:

• Self-hostable
• Encrypted
• A WYSIWYG editor
• Adapt to the screen size

I stumbled upon the brand new EteSync Notes app. After playing with it for a few minutes, I realized it didn't check all the boxes, but had a lot of potential: it was already fully functional and needed just a bit of polish. Since I was also looking into contributing to a community project at the time, I took a look at the repository and found out it was using React Native, which I was already familiar with, and I agreed with most of the feature requests in the issues. It was time to start my journey with the EteSync Notes project!

My first contribution

Since projects are not always very active, I decided to start with a small problem to see if help was welcome. It was something that was annoying me and was already mentioned in one of the issues: the FAB on the notes list would go over the dialog to create a new note when the keyboard was up. Working on an UI problem would allow me to do it quickly because I was already familiar with the framework and to familiarize myself with the code without having to dig too deeply. And here's the result:

I created a Pull Request and was amazed when it was reviewed and accepted one hour later! This motivated me even more!

Keeping it coming

Since then, I have kept going through the issues in the repository, or the ones I have encountered. That allowed me to have a better understanding of how the app worked little by little and to work on more complicated tasks. And when I had questions, I just needed to go to the IRC/Matrix Chat and ask.
Here are some notable improvements I have worked on:

What's coming next

A few things I might be working on next:

• Adaptive view on large screen with a split view
• Make the app feel more native on web with a pointer on links and visual change on hover
• Import/export notes
• Search notes
• Desktop versions?
• Improve integration with GNOME: GOA, GNOME Notes…
• Whatever you want…

I really want to thank Tom for being so approachable, available, helpful and patient with his reviews.

As you probably know EteSync is now supported natively by both KDE and GNOME. This means that you can now add EteSync accounts directly to GNOME, KDE and apps in their ecosystem such as Evolution, GNOME Calendar, Kontact and the likes.

This has been possible for a very long time using the DAV bridge, though with these integrations, the bridge is no longer needed.

As we said in the original announcement, the integrations will be released and packaged in distributions following the next GNOME and KDE releases. However, a lot of you have been asking about starting to use it now, and we can't blame you, as we wouldn't want to wait either! :)

Thanks to some hard work by daftaupe, sjolly, and others, there are now easy to use packages available for multiple distributions!

For the most up to date packaging information, please refer to the GNOME user guide and the KDE user guide. We are working on packages for more distributions every day. However at the time of this writing there are already packages for Arch (AUR), Ubuntu, Debian, Fedora, CentOS, OpenSuse and Mageia depending on whether you use Evolution or KDE. In addition, if your distribution isn't listed above, you can always just build the plugins from source.

Do you know how to create packages for your favourite distro and are willing to help? Need any help with setting up the modules? Please join us at the chat (IRC/Matrix/Web) and let us know!

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

We are very excited to announce the official release of EteSync 2.0!

This would not have been possible without the hundreds of testers that helped us make sure everything is solid! So thank you all!

If you already know what EteSync 2.0 is and just want to upgrade your account, please refer the EteSync 2.0 migration guide.

As we said in the post announcing EteSync 2.0, time flies. We still can't believe it has already been three and a half years since EteSync was first released. It started out as a simple end-to-end encrypted sync solution for Android. We have since added support for sharing data with other users, and we now have clients for the desktop (DAV bridge), the web, and iOS, built-in support in and Tasks.org, and there are add-ons being built for GNOME, KDE and Thunderbird.

What is EteSync 2.0?

EteSync uses the EteSync protocol (now Etebase protocol) behind the scenes to power all of the apps. While the existing protocol has served us well, it limited our progress and prevented us from achieving some of the things we would like to achieve for EteSync, so after three and a half years, it's finally time for a major protocol upgrade.

EteSync 2.0 is a great improvement over the 1.0 version, while still maintainig everyone's favourite features and capabilities, including, end-to-end encryption, sharing and a full version history. However, since so much has changed under the hood, EteSync 2.0 and 1.0 are incompatible, and you need to migrate your account to 2.0 in order to start using it.

What does it mean for you?

It means we are finally be able to offer some long awaited and often requested features such as notes synchronization and secure location sharing. It means that your sync is faster. It means the EteSync applications are simpler to develop, which means they are easier to improve. We are already seeing these benefits in all of the apps, which already work much better, and helped us reduce large amounts of code.

In addition, there are a lot of user-facing improvements and changes, including:

1. EteSync 2.0 uses usernames for identification (rather than emails previously), so you will need to choose a new username. Usernames should be at least 6 characters long. Shorter usernames will probably be allowed in the future. Changing your email and username will both be possible later on.
2. EteSync 2.0 has just one password that's used for both the encryption, and the login. It uses a zero-knowledge proof to authenticate to the server making sure your password never leaves your device. This makes it more secure than before but much easier to use. Please make sure to keep it safe and don't lose it, as without it you won't be able to access your data!
3. The change log (previously change journal) used to show all of the changes in one long list. Changes are now grouped per item with only the latest changed shown. To show previous changes you need to click on an item and check the past revisions.
4. Invitations UI is a bit different, and there's now a special page for accepting or rejecting invitations, and it's now also easier to leave collections you've been invited to.
5. "Journal" has now been renamed to "Collection", which is a more accurate term and people find it less confusing.
6. You can now easily change your account's email address directly from the dashboard.

What's next

The above is just the tip of the iceberg. The biggest benefit of changing to EteSync 2.0 is not how it improves things, but rather what it enables us to do next!

Thanks to EteSync 2.0 we were already able to release EteSync Notes. Additionally, as we said in the previous post, we aren't ready to get into details just yet, but we plan on bringing in some long-awaited features such as secure location sharing. Expect a few more announcements in the coming month or two which will make EteSync a more complete solution for your data syncing needs!

We also plan on formally verifying the protocol using Verifpal and releasing a formal spec to make it easier for other developers to examine, use and improve it, and potentially making it into a standard.

Upgrade to EteSync 2.0

Upgrading to EteSync 2.0 is very easy. First thing to do is to make sure you've update all of your apps to their latest versions.

As we said above, EteSync 2.0 accounts are incompatible with EteSync 1.0 accounts, so a new account needs to be created. However, the billing for both is linked. The migration tool automatically takes care of linking the billing, so you don't need to worry about that.

For step-by-step instructions please refer to the EteSync 2.0 migration guide.

Developers: easily build end-to-end encrypted applications

Over the years many developers have reached out to us about adding end-to-end encryption to their applications. Building end-to-end encrypted applications is both hard to get right, and very time consuming, so using an existing solution makes a lot of sense, especially since EteSync is versatile and isn't limited to contacts, calendars, tasks and notes.

We are now also ready for developers. For more information, please take a look at the developer homepage (Etebase).

Thank you NLnet and NGI0

The work on EteSync 2.0 is made possible with financial support from NLnet Foundation, courtesy of NGI0 Discovery and the European Commission DG CNECT's Next Generation Internet programme.

The NLnet foundation in general and the NGI0-PET in particular is funding projects to protect everyone's digital privacy, especially in the context of the "next generation" of the internet. It's an extremely important cause which we at EteSync are very much aligned with. Please help by spreading the word about them.

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

We are happy to announce that EteSync 2.0 is now ready for testers! Are you interested in helping us with testing? Please read on! Not sure what is EteSync 2.0? Please read our previous post on the topic.

Introduction

We have been working on EteSync 2.0 for the last six months, though in a sense for a lot longer. It incorporates a lot of the feedback we got from you over the years, and the things we have learned in the three and a half years of running this service. As mentioned in our previous post, a lot of the changes are behind the scenes and will enable us to build on top of, though there are a few changes that are worth mentioning here again as they are important for testers.

1. EteSync 2.0 uses usernames for identification (rather than emails previously), so you will need to choose a new username. Usernames should be at least 6 characters long. Shorter usernames will probably be allowed in the future. Changing your email and username will both be possible later on.
2. EteSync 2.0 has just one password that's used for both the encryption, and the login. It uses a zero-knowledge proof to authenticate to the server making sure your password never leaves your device. Please make sure to keep it safe and don't lose it, as without it you won't be able to access your data!
3. The change log (previously change journal) used to show all of the changes in one long list. Changes are now grouped per item with only the latest changed shown. To show previous changes you need to click on an item and check the past revisions.
4. Invitations UI is a bit different, and there's now a special page for accepting or rejecting invitations, and it's now also easier to leave collections you've been invited for.
5. "Journal" has now been renamed to "Collection", which is a more accurate term and people find it less confusing.

In addition to the above, we also improved the clients while working on them. Especially the web UI which should now be faster and easier to use than ever. There may also be some other changes we forgot to mention.

How to start testing

As mentioned above, EteSync 2.0 uses usernames instead of emails and has a different authentication mechanism. This means that EteSync 2.0 accounts are internally new accounts that need to be created.

If you are new to EteSync and would like to give it a go, you can signup directly from the new web and Android clients (links below).

If you already use EteSync 1.0, there is a migration tool that automatically handles both the account creation and migrating your data over (including the full change history). It's currently only available for the web client, but will soon also be available for the Android and iOS clients.

There are migration tools available for migrating your account. The easiest is probably the web based migration tool which is available here: EteSync 2.0 migration tool. The Android app comes with its own migration tool. You can access it by clicking on the account you would like to migrate, then the top-right menu, and then "EteSync 2.0 Migration".

The migration tool only copies your data over, so if you change your mind, you can always go back to version 1.0. Just be aware that new changes made to 2.0 won't sync back to 1.0 and vice-versa, so you will have to copy them over manually if you change back to 1.0.

To make testing easier, billing is currently disabled for EteSync 2.0 accounts, they are all on infinite trials. We will however turn billing on for all EteSync 2.0 accounts in the next few weeks, and will automatically link your existing EteSync accounts to the EteSync 2.0 accounts using your email address so billing information will be transferred over automatically. Edit: billing is now turned on for EteSync 2.0.

Here are links for the EteSync 2.0 compatible apps:

• Android: available on the Play Store and F-Droid.
• Web
• EteSync-DAV (DAV Bridge) - Version 0.30.0 or newer
• iOS: available on the App Store
• GNOME (Evolution): ready
• KDE (Kontact and etc): ready
• External apps:
  • OpenTasks: works.
  • Tasks.org: works with the new OpenTasks-like integration.

What is there to test? Everything. A few of us already use EteSync 2.0 instead of EteSync 1.0 and it works very well, however we realise that there may be some bugs lying around. So please, report every issue you see. You can do it either in the respective bug trackers, by mail, or our chat (see links at the bottom of the post).

We are aware that not all of the strings have been updated (e.g. some dialogs still refer to 1.0 journals), and on Android we still haven't implemented restoring an item to a specific state or sending email invitations, and the signup flow needs some more work, but other than that, everything should work. So please, report every issue you encounter.

If you are self-hosting, the server is available here, though it doesn't yet have all of the easy-config stuff that EteSync 1.0 had that made self-hosting easier. We will port them over in the coming weeks.

What's coming soon

As said in the original post, EteSync 2.0 brings with it a lot of improvements, but in addition to the improvements to the existing EteSync apps, where it really shines is what it enables us to do next.

We aren't ready to get into details just yet, but we plan on bringing in some long-awaited features such as notes synchronization and secure location sharing. These, however, are just the tip of the iceberg. We have a lot more coming, so expect a few more announcements in the coming month or two which will make EteSync a more complete solution for your data syncing needs!

As for the EteSync 2.0 release itself, we don't have an exact date yet, but it's imminent. There are no major bugs that we are aware of, and everything seems to be working well. We would just like to have more people test it, so we can be extra sure that everything works as smoothly as it should.

Developers: using the EteSync protocol in your app

Over the years many developers have reached out to us about adding end-to-end encryption to their applications. Building end-to-end encrypted applications is both hard to get right, and very time consuming, so using an existing solution makes a lot of sense, especially since EteSync is versatile and isn't limited to contacts, calendars and tasks.

We are now also ready for developers. For more information, please take a look at the developer homepage (Etebase).

Thank you NLnet and NGI0

The work on EteSync 2.0 is made possible with financial support from NLnet Foundation, courtesy of NGI0 Discovery and the European Commission DG CNECT's Next Generation Internet programme.

The NLnet foundation in general and the NGI0-PET in particular is funding projects to protect everyone's digital privacy, especially in the context of the "next generation" of the internet. It's an extremely important cause which we at EteSync are very much aligned with. Please help by spreading the word about them.

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

Since we've had a lot of new visitors from the GNOME and KDE community getting to this blog post we've added a short introduction section.

What is EteSync?

EteSync is a secure, end-to-end encrypted, and privacy respecting sync for your contacts, calendars and tasks. Easy to use and open source - it seamlessly integrates with your existing apps so you won't even notice you are using it! Think CalDAV but encrypted in a way that no one but you can access it. Not hackers getting access to your servers or you server admins!

For an introduction to end-to-end encryption please take a look at our related blog post. For more information about EteSync, please head to our website.

The rest of the post

As readers of this blog you are probably aware that there are two EteSync related projects in this year's Google Summer of Code. The first project is an EteSync backend for Akonadi, the KDE personal information management server. The second project is an EteSync module for Evolution data server, the GNOME personal information management server.

We have previously written here about the great work both sjolly and nourmat have been doing, though this post is different. It is finally time for beta testing! Both of the modules have been thoroughly tested locally, and have been reviewed by the maintainers of GNOME, KDE and EteSync and are ready for some real-life testing by real users.

Are you a GNOME or KDE user and willing to give the modules a try? Please come join us at the community chat (IRC/Matrix/Web), or email us at gsoc-testing@etesync.com for instructions on how to get started with the testing.

Your help will be much appreciated and will help us catch bugs and fix them sooner rather than later!

Thank you for your help!

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

Even though there's a Thunderbird add-on in the works, and that the GNOME and KDE EteSync plugins are making great progress, it does not mean we have forgotten about EteSync-DAV, the desktop DAV bridge. We are still working hard on it, and we don't plan on stopping in the forseeable future.

As you may know, EteSync-DAV is an EteSync compatibility layer that lets you use EteSync with any CalDAV and CardDAV supporting client such as Thunderbird, Evolution, Outlook and Calendar.app. You just run it locally, point your DAV client to it, and all of the encryption and syncing is handled automatically for your in the background.

There have been been two major changes in the most recent DAV bridge release that greatly improve the user experience! In summary, syncs are faster now, and they happen more often if your DAV client requests it. Additionally, if you use want to use SOCKS proxy (e.g. Tor), you can now do it!

Changing the sync to being semi-synchronous

As you may remember, we have previously improved the DAV bridge to work in an asynchronous manner. If you don't remember why we did it, please take a look at our previous post on the topic for more information. Here is how it looked before the most recent page (picture taken from the previous post):

This improvement was great and made everything faster by doing the syncs in the background. The main problem with this change though is that as you can see, the DAV bridge is replying to Thunderbird before it finished fetching data from the server. This means that EteSync DAV may return stale information!

This is not an issue in most cases, as the sync happens automatically in the background, but it was very annoying when testing things, because you are not getting immediate feedback, and was very confusing for users.

We have no changed it so that the sync happens in the background, like before, though unlike before it now also tries to wait for a short period of time to see if it's able to return fresh information before giving up and returning stale one.

Socks proxy support

A lot of people use proxies for various reasons, for example if their corporate environment requires using one, or if they use Tor for example.

The DAV bridge has always supported proxies, though due to an oversight, the docker image and the pre-built binaries didn't have this support included. This has now fixed, and you can finally use SOCKS proxies with the most recent DAV bridge version by setting the HTTPS_PROXY (https_proxy on some platforms) enivornemnt variable and launching the DAV bridge.

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

When you first login into your account, the different EteSync apps automatically create a few default collections for you to use. One for calendar, one for contacts and one for tasks.

Until very recently, the default name for each of these collections was "Default". Many users found it confusing over the years, though we never got around to fixing it. This has finally changed thanks to a user that reminded us of how confusing it is. We rely on your feedback to make the service better, so please, keep it coming!

The name names for the collections are now as follows: "My Calendar", "My Contacts", and "My Tasks". This is how it looked before vs now:

Do you have anything else you think we should improve? Or do you think the naming is not great? Please let us know!

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or  are experiencing any issues? Please send patches, report issues or just contact us.

Come chat with us on IRC/Matrix, or follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

If you follow this blog, you already know that there's a native Thunderbird add-on in the works (hopefully with a version out soon), and probably GNOME (Evolution) and KDE integrations will be worked on in this summer's Google Summer of Code and Outreachy. After these projects are finished, users of the above desktop clients won't need the DAV bridge anymore, and can just use EteSync directly!

Even with these exciting projects in the works, we still haven't forgotten about the desktop DAV bridge! The DAV bridge is still needed for applications where EteSync isn't, or can't be, supported natively such as Contacts.app (macOS) and Outlook (Windows).

There have been a few major improvements to the desktop DAV bridge, recently, here are the highlights.

Initialise new accounts

Since EteSync is end-to-end encrypted, the server can't access user data. This is a great thing, and why we created EteSync in the first place, however it also means that most of the logic needs to be implemented in every client, and this includes the DAV bridge.

Specifically, until recently the DAV bridge would not initialise new accounts, but instead just assume that the account has already been initialised by another client, such as the Android or the web one. Most people wouldn't even notice it because they use multiple clients anyway, but those that were using EteSync only on the desktop needed to first login from another client in order to properly use EteSync.

Not having to open another client makes the on-boarding for new desktop-only users much smoother.

Management UI improvements

The desktop DAV bridge can be configured in one of two ways. The first is using the command line interface, and the second (recommended) way is through the web management UI. We made a few improvements to the management UI recently, but the most notable one is the unification of the DAV server with the management UI server.

Before this change, the main DAV bridge url was http://localhost:37358 and if you opened it in the browser, it would redirect you to http://localhost:37359 which was the management UI. Notice the slightly different port numbers. A lot of people found this very confusing, and more than once we saw people trying to use the wrong URL in their DAV clients.

It took a bit of tinkering, but we finally managed to get both services to run from the same port (and thus URL), with the management UI being redirected to automatically when opened from the browser.

A few more bug fixes and improvements

We fixed an issue that was causing a high CPU usage when there was no internet connection. The issue was in the error-recovery logic. The DAV bridge is designed to try again on error and wait a certain amount of time between retries. However, in some cases it would skip the wait and get into a loop of trying and failing, driving up the CPU usage.

We also fixed an issue with the local cache getting locked which could cause syncs to fail with multiple concurrent requests. This was because we were using the underlying database (SQLite) instead of the more appropriate WAL mode.

In addition, we also changed the location of the DAV bridge's data directory. Due to legacy reasons, the DAV bridge was saving its data in the user's configuration directory (~/.config/etesync-dav on Linux) instead of the user's data directory (~/.local/share/etesync-dav on Linux). Most users won't even notice the change, but we are happy to have finally fixed this legacy relic.

As usual, we would like to remind you that we rely on your feedback and contributions to make EteSync better. Do you have any suggestions or are experiencing any issues? Please send patches, report issues or just contact us.

Follow us on Mastodon, Twitter, Facebook, reddit or RSS for the latest updates and privacy-related content!

Even though there is a native Thunderbird client in the works, it does not mean we have forgotten about EteSync-DAV, the desktop DAV bridge. It's quite the contrary.

As you may know, EteSync-DAV is an EteSync compatibility layer that lets you use EteSync with any CalDAV and CardDAV supporting client such as Thunderbird, Evolution, Outlook and Calendar.app. You just run it locally, point your DAV client to it, and all of the encryption and syncing is handled automatically for your in the background.

Many people rely on it for syncing their EteSync contacts and calendars with their favourite desktop clients, and while it got the job done, there were two major issues that were affecting many users. The first was slow sync times for users that were geographically far from our servers. The second was that contact photos were not being synced. Both of these have now been fixed! You can get the latest changes by updating to version 0.13.0.

If you are interested in some more technical details, please read on.

Slow sync times

EteSync-DAV serves as a proxy between your DAV desktop clients and the EteSync server. This means, that when your desktop client (e.g. Thunderbird) initiates a sync with EteSync-DAV there are a few syncs going on behind the scenes.

Let's use Thunderbird as an example desktop client. When syncing, Thunderbird would first initiate the sync with EteSync-DAV, then EteSync-DAV would initiate a sync with the EteSync server, wait for it to finish, and only then would it send back the fresh information to Thunderbird. The sync looked something like this:

Since EteSync-DAV is run locally, the connection between Thunderbird and EteSync-DAV is quite fast, and the connection between EteSync-DAV and the EteSync server is comparably slow. Though because EteSync-DAV was waiting for the sync with the EteSync server to finish before replying to Thunderbird, the local sync was slow too.

With the new version, we have decoupled the two syncs. The slower syncs between EteSync-DAV and the EteSync server now happen periodically in the background (are also triggered more often if requested by the Thunderbird), so the sync between Thunderbird and EteSync-DAV can be instantaneous. After the changes the sync looks more like:

As you can see, the local sync is now completely independent from the slower remote sync, which makes editing events in Thunderbird and other desktop clients instantaneous.

Missing Photo field when syncing contacts

Before this change, contact photos were not synced when using EteSync-DAV. This was because EteSync uses the newer vCard version 4.0 interally, but Radicale, the Cal/CardDAV server we use to power EteSync-DAV, as well as other clients such as Apple's Contacts.app only support vCard version 3.0. In order to solve this inconsistency we transform between the two versions in EteSync-DAV. Until now, this conversion didn't include the Photo and Logo properties, but with the latest EteSync-DAV these too are now supported.

Feedback, suggestions and contributions

As usual, we would like to remind you that we rely on your feedback to make EteSync better. Do you have any suggestions or are experiencing any issues? Please send patches, report issues or just contact us.

In addition, your contributions are always welcome! Be it testing, design, code or helping us spread the word! Every bit helps in making EteSync better for everyone. If you are interested in contributing and don't  know where to start, don’t hesitate to contact us!

Follow us on Mastodon, Twitter, Facebook or RSS for the latest updates and privacy-related content!

Until next time!

The images above use modified icons by Freepik from flaticon.com that are licensed under CC 3.0 BY.