Migrating Oracle DB to Couchbase

Hi, I’m working on project which Migrating data from oracle Db to Couchbase ,
The volume is quite big and the BI logic can be very complex.
First ,we extract the data into json and from there we loading it using couchbase loader.
Im not coniviced this would be the best approach from perfroamnce aspects , what do u think?

Second Q - for the data validation part ,we need to compare the source and target
(after load) , is there any tool we can use? is any scripting language can do that good and fast?
Perl?Phyton?
any other tips are welcome!

Hi @BgOKcomp,

That sounds like a decent approach to me. But if it’s not performing well enough, you might want to check out the Golden Gate connector for Couchbase: https://github.com/mahurtado/CouchbaseGoldenGateAdapter

As for comparing the data after the load, Couchbase offers a number of SDKs: Python, but also Java, C, Go, .NET, .NET Core, PHP, Node, and several others that are community supported. There is probably a Perl SDK, but not an officially supported one.

@BgOKcomp - Did you find the way to move the huge volume of data in efficient way ?
@matthew.groves - this talks about data Sync with change data capture but no migration .
It is sad that moving to couchbase there is no great guidelines a publication how client can handle huge volume of data migration (relational to couchbase)
thanks

@eldorado,

There are many approaches and tools to migrating data from relational to Couchbase, and recommendations vary greatly depending on your requirements and use case. In addition to the GoldenGate connector, here are some resources to start with:

Thanks for providing the links to read in . The only link which I can resembles to migration is below https://blog.couchbase.com/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase/
However CURL lacks with limitations …Anyway do you know with same approach in CURL with N1QL we can have parameters to access on huge volume of Oracle data pragmatically and keep the connection live for billions of records ?
Talend is extra tool to buy in just for migration is not sounds reasonable for us … sorry

@eldorado,

I’d recommend starting a new thread in the N1QL forum: https://forums.couchbase.com/c/n1ql

I’m tagging @binh.le to see if he has any additional insight for you.

thanks Matt .I am trying to get @binh.le attention on other threads already . Just waiting for his feedback on 3 weeks old post