Synchronisation¶
This section describes the basic aspects of synchronisation using Kinto.
Note
If you are looking for a ready-to-use synchronisation solution, jump to Implementations.
The basic idea is to keep a local database up to date with the Kinto server. Remote changes are downloaded and applied on the local data. Local changes are uploaded using HTTP headers to control concurrency and overwrite.
In short:
- Poll for remote changes using
?_since=<timestamp>
- Apply changes locally
- Apply creations remotely
- Use concurrency control to send updates and deletes
Polling for changes¶
When fetching the collection records with ?_since=<timestamp>
, Kinto returns
every records that was created/updated/deleted after this timestamp.
- Cold sync → no
?_since
. GET /buckets/default/collections/articles?_since=<timestamp>
(with headerIf-Match: <timestamp>
)- If response is
304 Not modified
, done → up to date, nothing to do. - If response is
200 OK
storeETag
response header value for next synchronisation and handle the obtained list of changes.
Note
- Deleted records have an attribute
delete: true
. - Created/updated records are both returned in their new version.
- Since Kinto does not keep any history, there is no diff for updates.
Paginated changes¶
By default, Kinto does not paginate the records list. If an explicit limit is
set in the server settings or using the ?_limit=<nb>
parameter, then polling for
changes will be paginated.
It basically consists in fetching the list until the Next-Page
is present.
Since changes can occur between the first and the last page, the synchronisation process is a bit more complex.
- Cold sync → no
?_since
. GET /buckets/default/collections/articles?_since=<timestamp>
(with headerIf-Match: <timestamp>
)- If response is
304 Not modified
, done → up to date, nothing to do. - If no
Next-Page
response header, done → storeETag
response header value for next synchronisation. - If
Next-Page
response header is present → store theETag
response header value into a variabletimestamp
and go on to it (it’s an url) using aIf-Match: <timestamp>
request header. - If response is
200 OK
→ repeat previous step. - If response is
412 Precondition Failed
, some changes occured since the last page → Store theETag``response header into ``before
- Fetch and handle the changes using
GET collections?_since=<timestamp>&_before=<before>
, and store theETag``response header in to ``timestamp
- Go back to step 5 (follow the
Next-Page
)
Apply changes¶
Applying remote changes to the local database consists in adding new records, updating changed records and remove deleted records.
From the client perspective, Kinto does not distinguish creations from updates.
In the polling for changes response, created records are simply the records
unknown by the client (using id
field).
If the records to be updated or deleted had also been modified locally then the developper must choose a relevant strategy. For example, merge fields or ignore deletion.
Concurrency control¶
As described in Server timestamps, Kinto uses ETag for concurrency control.
ETags are provided in response headers, for the collection as well as individual records.
Even though it is recommended to consider them as opaque and abstract, it can still
be useful to notice that ETags are a string with the quoted record last modified value
("<record.last_modified>"
)
Protected creation with PUT¶
Add a If-None-Match: *
request header to the PUT
to make sure no
record exists on the server with this ID.
This can be useful to avoid overwrites when creating records with PUT
instead of POST
.
Protected update and delete¶
Add a If-Match: "<record.last_modified>"
request header to the PUT
, PATCH
or DELETE
request.
Kinto will reject the request with a 412 Precondition Failed
response if
the record was modified in the interim.
If the remote record was already deleted, a 404 Not found
response will be
returned. The client can choose to ignore it.
Offline-first¶
Since the server won’t be available to assign record identifiers while offline, it is recommended to generate them on the client.
Record identifiers are UUID, a very common format for unique strings with almost zero [1] collision probability.
When going back online, the set of changes can be sent to the server using a POST /batch request.
Implementations¶
The current implementation of reference for offline-first records synchronisation is Kinto.js.
Before that, some other clients were implemented in the context of the ReadingList project, such as RL Web client (React.js), Android RL sync (Java) or Firefox RL client (asm.js).
[1] | After generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%. Source |