I am aware of cli tools like cbexport to export the data in CSV fomat .I want to know if I can export/download the data from couchbase to my local backup folder in CSV format using Java.
There are many third party libraries for converting JSON to CSV. For example, the Jackson JSON library has a CSV converter.
You could use either the Java DCP client, or something that layers on top of it such as the Couchbase Kafka Connector, in order to get a stream of all Couchbase documents. And then convert those into CSV. But you’d need to write that code, and make sure that your processing nodes could keep up with the stream. Is there a reason not to use the existing CLI tools, or the Couchbase backup service, which will handle the complexity for you?
Hey @graham.pople thanks for the reply.
Actually I have to automate, backing up data from couchbase to a bucket in s3 daily(scheduled manner).So I am writing a java code , where I am using the RawQueryExecutor to get the data in JSON and converting this data into CSV format. Then I am storing this CSV to s3 bucket. I am not sure if I can use the CLI commands to automate this, is that possible?
I’m not an expert on the CLI backup tools, but perhaps @pvarley can help here.
In Couchbase Server Enterprise Edition the backup tool - cbbackupmgr can backup and restore directly to S3, for more information please see the docs Please note that it will not be in CSV format but if it’s just for backup purpose the format should not matter.
- Is there a need for the format on S3 to be CSV?
- Is each document its own object in S3?
cbexport only does JSON format, the reason for this is that JSON is multiple dimensions where generally CSV is two dimension. (There’re CSV formats that can do multiple but they’re rare)