I am trying to import 18 million records into couchbase with Talend but it seems to only import about 60 ops. Has anyone else had experience with talend that may have some insight on tuning this connector.
Hi @fnsnoop
I have used the Talend Open Studio for Big Data to import data into Couchbase - is that what you’re using? http://blog.couchbase.com/mysql-couchbase-using-talend-etl
I don’t remember it being that slow though. What system are you importing the data from and do you have a way to check that Couchbase is the only system that’s slow to write data into?
-Will
Yes I am using the big data version of talend. I am importing from oracle and I can write to a csv at about 10k per second. I can write my own script to in GoLang to import that csv quickly at 500+ records per second. I am having trouble if I connect oracle directly to the couchbase connector. Is it possible to do a bulk load in batches with the connector?