How we built the Evernote Calendar Connector

APIs can be difficult services to explain, especially to non-technical users, so it generally helps to have a tangible use case for people to explore as a starting point. You can then have a conversation about the similarities and differences rather than abstract concepts.

Our Evernote Calendar Connector creates a calendar event whenever a reminder is set on a note. What’s more you can then edit the reminder and note information in either Evernote or your calendar and the two stay in sync.

So if your app is an online booking service, the setting of a reminder is analogous to a booking being made and the event is created in the calendar of the professional servicing the appointment.

If the professional then changes that event, either by moving it or adding additional information your booking service is notified and can reschedule or undertake whatever workflow is appropriate.

This post is deliberately technical but, generally code agnostic, in an effort to explain some of the concepts behind the approach taken. That said the source code, in Ruby, is available on Github.

My hope is that understanding how we built this app will give you a better understanding of some of the considerations and challenges you may face when you’re deciding whether and how to integrate your application or service into your users’ calendars.

Solving synchronization problems not just doing CRUD

Linking Evernote to a calendar represents some difficult synchronisation challenges that are present when you start to connect many classes of applications.

The Cronofy API is deliberately not just a protocol conversion layer on top of the existing calendar APIs. Traditional calendar APIs are based around client-server interaction, that client being the single source of truth and the event representation being a collapsed form of a far richer object graph. You can smell some of the symptoms of this in constraints around organisers and attendees present in all calendar apps.

We’ve designed our API to support the operations required of server-to-server interaction. But also to optimise sync operations which are a key aspect of many of our API use cases.

Our role to is to allow application developers to map a representation of an event or allocation of time from their system into people’s calendars. As well as vice versa, allow them to map user generated events onto their application domain.

This means we can’t just map CRUD operations and expect our customers to deal with the consequences.

Avoiding polling and knowing what to sync

Both Evernote and Cronofy provide push notifications for changes. They’re crucial for sanity and not having the expense and maintenance overhead of polling services when, more often than not, nothing has changed.

The pattern we’ve used for receiving web hooks is to queue a background task to perform the synchronization at a later point. This both prevents the app from blocking the upstream provider and allows us to scale up workers in a controlled fashion.

With this approach the app then has various reference point strategies for querying changes that may be relevant:

1. a known point in the timeline for the upstream service
2. the push notification contains a remote time reference
3. local system time.

The last is the least desirable because of inevitable timing differences between disconnected systems. This makes for some complex state tracking challenges as you have to over-compensate to ensure you don’t miss anything. However, in doing so you can’t avoid potential duplication.

Evernote provides an Update Sequence Number which serves the purpose for 1.

Each of these objects also has an update sequence number (USN) that is changed whenever the object is modified on the service. Each updated object receives a new USN whenever a change is committed. The USN values are unique and monotonically increasing within an account, so that a client can inspect any two objects in order to determine which was more recently modified.

Cronofy has adopted the approach of 2. Whereby the push notification provides a changes_sinceparam which is reliably queryable time for filtering changes.

Handling deletions

Deletions are valid operations in both Evernote and calendars and have meaning to the service we’re providing.

If the reminder is removed from the note in Evernote or the note itself is deleted then we need to delete the corresponding calendar event. Likewise if the calendar event is deleted, we need to remove the reminder against the note.

Fortunately both APIs allow us to read deletions as a distinct state transition rather than trying to infer that it has happened based on the item no longer appearing in queries.

For Cronofy, the Read Events endpoint accepts a include_deleted parameter.

In Evernote’s case they have a deleted attribute of the Note metadata. We also consider the operation to be a deletion if the reminder time has been unset.

However, this could represent a state management challenge. Evernote does not indicate that the reminder time has been unset; it merely gives the current value. This would normally require our app to start maintaining that state so it can conditionally delete the event if it sees a transition.

Luckily we’ve accounted for this scenario and it’s partly enabled due to us using your ids for events rather than us arbitrarily assigning a unique id.

The magic of using YOUR ids

ID mapping is crap work. I shudder to think how many mapping table records exist in applications around the Internet. Their only purpose is to map one arbitrary ID to another.

We’ve deliberately avoided that and allow you to use your ID to identify your events.

This isn’t just because we’re nice people it actually has some key benefits when it comes to supporting the synchronization operations we’ve discussed above.

You’ll note that the Create Events endpoint is actually an upsert. So you can happily just send us the event multiple times and we’ll handle whether it’s a creation or update on the downstream calendar.

Deletions work in the same way. We track whether we:

a) know about the event
b) need to delete the event on the downstream calendar.

So, in the case of the Evernote reminder we can issue a delete whenever we come across a note that doesn’t have a reminder. We don’t have to track the state of the reminder time as the Cronofy API handles that for the app.

Applying this to your own project

I hope that this article has gone some way to describing some of the considerations when syncing two applications; not just calendars but any systems. Calendars are the special case we’ve decided to tackle with our API and we’ve worked extremely hard to give you the best possible experience.

The sorts of things to consider when you’re planning out your own calendar integration are:

How is the event modelled in your application domain? What information do you have that could be of use (build a better experience for the user) if that were in their calendar?

Should different users see different information about an event? App links, location information, name, etc. could all be tailored to meet the needs of the user who’s receiving the event.

How should your application respond when the user changes or deletes the event? Internal apps could cancel appointment records and restaurant booking could cancel the reservation but a grocery delivery should probably initiate a customer service email or call.

So whether you’re proposing this as a feature to the product owner or you’ve been tasked with making calendar integration happen, you’re hopefully better equipped.

If you have any questions about your own project then don’t hesitate to get in touch: hello@cronofy.com.

‍