Data insertion from SQL Server to Couchbase

Hi Couchbase team,

We have an asp.net mvc appication which use Microsoft SQL Server. We use ms sync framework to sync data from sql server to client app. We are planning to use another mechanism to sync the data between apps, because of some limitations of the sync framework. Couchbase Server is one of our considerations. We need to keep the buisness domain of the server intact and insert or sync the new db entries to the Couchbase server and vice versa real time. Clients will sync the changed data directly from the Couchbase server. Is it possible to implement it. I have read the following article.

http://www.simba.com/blog/setting-up-couchbase-as-a-linked-server/

Grateful to you if you can come up with a solution. Thanks

@manura -

I am not sure their is a single, packaged solution for what you are trying to achieve. One possible solution for pushing to SQL server from Couchbase would be implement a client which subscribes to https://github.com/couchbaselabs/udcp and then writes each udcp message into SQL server using ADO.NET.

For syncing between mobile devices and Couchbase server checkout: http://www.couchbase.com/nosql-databases/couchbase-mobile

-Jeff

we have implemented the solution using Couchbase XDCR and Microsoft Sync Framework. You can do it using XDCR and Implementing a connector like elastic search Connector or spark connector . depending upon which makes you more comfortable with.

cause the data replication in that way can work both ways.

In Order to implement the
Sync Framework you guys need to work on Knowledge Provider and implement a CustomSyncProvider

@khalid.khalil
Hi, would you be able to share the source code just to get an overview how I could start doing this

What language are you trying to implement?

Unfortunately I can’t share the code as it is the intellectual property of the company I worked for but I can share information and guide you where to look or can start a new GIthub page from scratch and rethink the whole code.

@khalid.khalil
I am currently implementing it on C# , I am looking into the https://github.com/couchbaselabs/couchbase-xdcr-nancy . Still wrapping my head how to integrate MSF into this whole thing

Well

All the synchronization platform require version keepings. Similarly we have to store timestamp and version in couchbase or you can use memorymapfiles where your app will reside.

Though everytime there is a change in couchbase it will send the list of changes that has changed and you have to ask which one you require to update.

Couchbase keep its data in rev_Id which contain the mutations and the revision number.

Your temp stoRe will keep the document and convert them into dataset and then you can use that dataset to sync it with sql server.

Similarly you have to receive the dataset from sql server. Convert it into document and update them into couchbase and also keep the revision number. As soon as you update couchbase .couchbase will send you the same document again. You have to build a mechanism to discard that document as you are the one that is updating it.

Thank you I will try to follow your advice

By the way we also working on components for Talend platform, where you can also build data pipelines to move data between various datastores.

Currently we are in developer preview, but first beta scheduled in the middle of April. Here is how it will look like https://github.com/couchbaselabs/talend-components/blob/rel-0.16/doc/tutorial.md