Synchronisation

This section describes the basic aspects of synchronisation using Kinto.

Note

If you are looking for a ready-to-use synchronisation solution, jump to Implementations.

The basic idea is to keep a local database up to date with the Kinto server. Remote changes are downloaded and applied on the local data. Local changes are uploaded using HTTP headers to control concurrency and overwrite.

In short:

  1. Poll for remote changes using ?_since=<timestamp>
  2. Apply changes locally
  3. Apply creations remotely
  4. Use concurrency control to send updates and deletes

Polling for changes

When fetching the collection records with ?_since=<timestamp>, Kinto returns every records that was created/updated/deleted after this timestamp.

  1. Cold sync → no ?_since.
  2. GET /buckets/default/collections/articles?_since=<timestamp> (with header If-Unmodified-Since = <timestamp>)
  3. If response is 304 Not modified, done → up to date, nothing to do.
  4. If response is 200 OK store ETag response header value for next synchronisation and handle the obtained list of changes.

Note

  • Deleted records have an attribute delete: true.
  • Created/updated records are both returned in their new version.
  • Since Kinto does not keep any history, there is no diff for updates.

Paginated changes

By default, Kinto does not paginate the records list. If an explicit limit is set in the server settings or using the ?_limit=<nb> parameter, then polling for changes will be paginated.

It basically consists in fetching the list until the Next-Page is present.

Since changes can occur between the first and the last page, the synchronisation process is a bit more complex.

  1. Cold sync → no ?_since.
  2. GET /buckets/default/collections/articles?_since=<timestamp> (with header If-Unmodified-Since = <timestamp>)
  3. If response is 304 Not modified, done → up to date, nothing to do.
  4. If no Next-Page response header, done → store ETag response header valuè for next synchronisation.
  5. If Next-Page response header is present → store the ETag response header value into a variable timestamp and go on to it (it’s an url) using a If-Match: <timestamp> request header.
  6. If response is 200 OK → repeat previous step.
  7. If response is 412 Precondition Failed, some changes occured since the last page → Store the ETag``response header into ``before
  8. Fetch and handle the changes using GET collections?_since=<timestamp>&_before=<before>, and store the ETag``response header in to ``timestamp
  9. Go back to step 5 (follow the Next-Page)

Apply changes

Applying remote changes to the local database consists in adding new records, updating changed records and remove deleted records.

From the client perspective, Kinto does not distinguish creations from updates. In the polling for changes response, created records are simply the records unknown by the client (using id field).

If the records to be updated or deleted had also been modified locally then the developper must choose a relevant strategy. For example, merge fields or ignore deletion.

Concurrency control

As described in Server timestamps, Kinto uses ETag for concurrency control.

ETags are provided in response headers, for the collection as well as individual records.

Even though it is recommended to consider them as opaque and abstract, it can still be useful to notice that ETags are a string with the quoted record last modified value ("<record.last_modified>")

Protected creation with PUT

Add a If-None-Match: * request header to the PUT to make sure no record exists on the server with this ID.

This can be useful to avoid overwrites when creating records with PUT instead of POST.

Protected update and delete

Add a If-Match: "<record.last_modified>" request header to the PUT, PATCH or DELETE request.

Kinto will reject the request with a 412 Precondition Failed response if the record was modified in the interim.

If the remote record was already deleted, a 404 Not found response will be returned. The client can choose to ignore it.

Offline-first

Since the server won’t be available to assign record identifiers while offline, it is recommended to generate them on the client.

Record identifiers are UUID, a very common format for unique strings with almost zero [1] collision probability.

When going back online, the set of changes can be sent to the server using a POST /batch request.

Implementations

The current implementation of reference for offline-first records synchronisation is :rtd:`Kinto.js <kintojs>`_.

Before that, some other clients were implemented in the context of the ReadingList project, such as RL Web client (React.js), Android RL sync (Java) or Firefox RL client (asm.js).

[1]After generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%. Source