Application-Server to Sync-Gateway Replication


#1

Our application server needs to change data particular to a single user (say its score).
That user is running a Couchbase Lite on his device, which syncs against a Sync-Gateway that the application-server now needs to access directly.

Doing it with “Bucket Shadowing” sounds expensive in terms of disk space and overall complexity.

Also, and regardless, when the user changes something on his device local database, I need the server to be triggered with an event (just like the device SDKs allow). Is that even possible?

It seems that Couchbase Mobile is a platform to keep devices in sync (with the cloud and among themselves) without allowing a back-end server to join this sync party. Injecting data from a server to a client (or a set of clients) seems like a very typical use-case. It would be extremely helpful if there was a Couchbase Lite SDK for the server side as well, with all the facilities included in the device SDKs (without, of course, maintaining a “local” copy of the database). Without this ability, the app-server can’t take part of the updates that the various clients make.


#2

Our application server needs to change data particular to a single user (say its score).

Use the Sync Gateway’s REST API to view and update documents.

when the user changes something on his device local database, I need the server to be triggered with an event

Again, use the REST API: monitor the _changes feed to detect changes to documents.

It would be extremely helpful if there was a Couchbase Lite SDK for the server side as well, with all the facilities included in the device SDKs (without, of course, maintaining a “local” copy of the database)

Any CouchDB client library will give you exactly that; they’re available for pretty much any language you want.

—Jens


#3

Thanks for the quick answer jen.

Please verify my understanding, though:

  1. Sync GW doesn’t have a rest api of its own. Its apis are these of the underlying database.

  2. Sync GW stores data in the database using ‘wrapped’ documents with additional fields to manage concurrent updates, deletes, conflicts, etc.

  3. sending me to interact directly with the database underlying SyncGW means that I have to follow the internal logic of how documents are wrapped, and not mess anything up when making updates. that’s very different approach compared to using the CB lite SDKs.

I do hope that I’m missing something here and that SyncGW provides APIs that offer similar services to the mobile SDKs. If there are such APIs please refer me to the documentation, as I couldn’t find it.

Thx
Zach


#4

Thanks for the quick answer jen.

Please verify my understanding, though:

  1. Sync GW doesn’t have a rest api of its own. Its apis are these of the underlying database.

  2. Sync GW stores data in the database using ‘wrapped’ documents with additional fields to manage concurrent updates, deletes, conflicts, etc.

  3. sending me to interact directly with the database underlying SyncGW means that I have to follow the internal logic of how documents are wrapped, and not mess anything up when making updates. that’s very different approach compared to using the CB lite SDKs.

I do hope that I’m missing something here and that SyncGW provides APIs that offer similar services to the mobile SDKs. If there are such APIs please refer me to the documentation, as I couldn’t find it.

Thx
Zach


#5

The Sync Gateway does indeed have its own REST API. It’s just not well documented at the moment; we’re transitioning to newly-written documentation and it’s still in a bit of flux. Sorry about that!

The REST API is very similar to Couchbase Lite’s. It offers full CRUD access to documents, and change notifications.


#6

I found some sample code that demonstrates using the SG REST APIs from a server-side app, including change notifications:

(This is a utility to be used with the CouchChat iOS app that will send iOS push notifications to a user when they get chat messages.)


#7

As per my understanding, not all the SDK operations are covered by the Sync GW REST API like bulk delete. Is my understanding correct, that its a subset of the SDK ops? If so, do you know of any documentation that lists these differences.


#8

Well, they’re very different APIs with overlapping features. The Couchbase Server APIs derive from memcached, while the Gateway APIs derive from CouchDB.

You can perform a bulk delete in SG by using the _bulk_docs API, setting each doc to a JSON object with nothing but "_deleted":true.


#9

Thanks. We are doing as you suggested. But moving from a couchbase bucket to sync gw bucket, we feel like missing on these SDK capabilities going via the REST apis. Any plans to make them seamless and uniform?

Back to the original question, my take would be not to use bucket shadowing if starting fresh. For any changes that the user does on the local device, and then syncs, you can have a webhook defined and fired to trigger the server component(as of sync gw 1.1). We are currently building data flows in our app using change-webhook pattern and use that to move the data between the clients and the server components.


#10

Any plans to make them seamless and uniform?

Yes, in the long term. Couchbase Mobile has a somewhat richer data model including revision histories and attachments. There’s design work going on to move some of that down to the bucket level, so SG doesn’t have to synthesize it. That will make it possible to have the CB APIs operate on SG buckets without screwing things up. Exciting stuff! But it’s still far enough out that we can’t yet promise it or quote schedules.