Shared calendaring improvements

On Wednesday, 8th November, we updated the service which provides our shared calendar and contact support at FastMail. This post explains the changes that were made.

Event reminders are now per-user

Each user accessing a shared calendar now has their own alarm reminders and transparency on all events, which won’t be visible to any other users.

Previously alarms reminders and transparency were shared. If anyone added a "display" alarm it would trigger for everyone who could see the calendar. If anyone added an "email" alarm it would always be sent (only) to the calendar owner, regardless who of who added it. Now each user can have their own display and email reminders.

Note: Event reminders that were set prior to November 8th will now only be seen by the calendar owner, regardless of who originally added them.

As a result of this change, if a user relies on receiving a reminder for an event in a shared calendar, they need to add their own reminder. Reminders can be set whether you are invited to the event or not.

URL update for accessing shared calendars and contacts

Shared calendars and contacts are available at a different URL. Most desktop and mobile clients will update themselves automatically, however some may require reconfiguration.

We have validated many commonly used clients to confirm that they automatically update themselves after the change.

We will directly email everybody who used a client that we haven’t validated, to access shared calendars or addressbooks within the past month.

The calendar sharing specification distingishes between two methods of sharing calendars: "team mode" and "secretary mode". In secretary mode, all invitations from events created in the calendar are sent from the owner of the calendar, regardless of who creates the event. In team mode, invitations are sent from the creator of the event.

Shared calendars at FastMail are currently all "secretary mode". We plan to support a choice of mode in a future update.

Now let’s look at what changed and why…

Background

When we first added CalDAV 3 years ago, there was no standard way to share calendars or contacts across users.

For multi-user accounts we had a shared addressbook, and we needed a way to represent this view of our Contacts service for CardDAV clients. For calendars, we wanted to support a shared calendar concept as part of our service, including for CalDAV clients.

Due to the lack of a standard, we checked what Google were doing and copied it – returning absolute URLs outside the requested homeset to a PROPFIND request.

This worked with almost all clients, and allowed us to release in 2014 with sharing available.

A proposed standard appears

Meanwhile, others were looking at the same problem. Apple wanted to support calendar sharing in their clients, but they didn’t want to have to configure sharing "out of band" (our sharing could only be edited through the settings screens in the FastMail UI) so they built something on top of DAV notifications and generic DAV resource sharing.

This work led to a draft standard for CalDAV sharing and an implementation of that draft standard sharing model was added to the upstream version of our Cyrus mail server.

In order to keep FastMail’s sharing model working, we added a special fastmailsharing: yes configuration option to Cyrus and kept going with our various hacks.

While fastmailsharing only uses ACLs to select which calendars are visible, the standard sharing model consists of an offer to share, and an acceptance of the sharing offer. This was implemented in the same way that IMAP sharing can work, by storing a list of calendars that you actually want to see in the SUBSCRIPTION database (LSUB in IMAP terms). Other than the subscription change, the only other difference is that shared calendars and addressbooks appear with different URLs for each user under the standard sharing model.

Conversion

Clearly both methods work fine, and we eventually wanted to convert our users to the standard URLs and avoid maintaining two different sets of code.

This involved a few steps:

1. Make sure subscriptions get created and removed when sharing is updated via our web interface
2. Make sure every existing shared collection is subscribed
3. Test that existing *DAV clients can handle the URLs changing

Step 3 was a big one! We gathered a bunch of devices and every client which had been used by more than 100 FastMail customers in the past month, then built a giant matrix and set up calendar and contacts to a test account on each device.

Once we’d tested every client to see that they could be used with the old configuration, we flipped the setting to fastmailsharing: no and watched each client re-sync.

Almost every client was fine, which is great – because the alternative of tracking the setting associated with every single user’s app-passwords and keeping both versions running until everyone changed passwords would be a total pain. If it’s only a few users at worst, then we can handle the support load of just contacting them and telling them they may need to reconfigure.

And there we are! One more special-case removed, and we’re a little more interoperable.