Syncgateway as import interface scenario

Mobile
1

HI everybody,

we are planing to use Couchbase as back-end database.
Furthermore we need to upload xml files from a customer (local windows server which automatically retrieves xml files)
and then process them with a xslt transformation and store the result as document in couchbase for aggregation.

So, one way would be to use a WebApi with authentication and a post request to upload the xml files to the server, nothing special about that.
But it would require to write logic for authentication, uploading and failover (connection lost, server down…).
So my question is, if something like that could be build with couchbase lite .net and sync gateway.

Based on the information I have collected so far, I would suggest the following scenario.
Use Couchbase Lite .Net inside a Windows Service on the local customer Windows server, if a xml file arrives, create a new document with some unique id(doesn’t matter because the relevant Id’s are living inside the xml file) and store it with the xml file as attachment (files have a maximum size of 1 or 2 MB).
On server side, use Sync Gateway to authenticate incoming requests and sync with the local Couchbase Lite instances.
Use a channel for every customer, so that the local instances only receive there documents and updates from Sync Gateway.
So, now the documents are stored in the syc_gateway bucket on the server, next step would be to process the xslt transformation and store the result in another ‘aggregation’ bucket.
I thought this would be possible by another Windows Service which runs on server side and acts as another ‘couchbase lite client’ (via RESTapi!?) which looks on a channel called ‘unprocessed’ which contains all documents from all customers that are not processed yet(could be some property with processed : false which is only set once from sync gateway during import from local client).
After processing the xml and storing the result inside the aggregation bucket, the service should delete the document from Sync Gateway to reduce disc space and of course the file should also be deleted from the local Couchbase Lite instance automatically during next ‘sync’ .

So my question is, if that is a possible workflow that Couchbase Lite .net and Sync Gateway could cover, or if I better use simple Web Api and handle failover on client side by my self.

Thanks for helping me out.

Best regards from Germany

2

That sounds like a reasonable use case.

The server-side transformation can be done by reading the Sync Gateway’s “changes feed” which lists all document updates. See the REST API docs for /db/_changes.

As for whether it’s overkill to use Couchbase Mobile … you probably wouldn’t have too much work implementing the authentication part on your own, but the failover part gets difficult. And if you ever need to send data back to the client from the server, it’s even harder to deal with connection issues. So having replication already available would be a big win for you.

—Jens

3

Hi Jens,

thanks for the information and feedback.
Good to know that I was on the right track.
After posting this question and some hours of playing around with Sync Gateway and Couchbase Lite, I had a working Implementation, of course a very basic one.
Anyway, the hole time I found the solutions to complicate for such a simple operation and I reconsidered my requirements.
Sync a Xml file from one machine to the other with some sort of authentication and failover.
I realized, that’s obviously Dropbox :slight_smile:
Of course we can’t use Dropbox, but there are other solutions out there like OwnCloud or Bittorrent Sync which fulfill all requirements, sync, authentication and failover.

So, again, thanks for your advise and if we need some sort of server -> client communication I come back to Sync Gateway.

Greetings
Marc