Problem importing CSV data with cbtransfer tool

[angeloassis]

Hi All,

I’m using cbtransfer to backup data from a bucket to a CSV file. The problem is that when I try to restore data, the document appears with metadata inside it.

For example:
In the original bucket the document has the key:
0001:2014-01-01T00:00:00.000+0000
And the value:
{
“value”: 2.479090921778098
}

When I backup it to a CSV file and restore in another bucket appears the following value:
{
“cas”: 1457544300929482800,
“dtype”: 1,
“rev”: 1,
“value”: “{"value":2.479090921778098}”,
“flags”: 33554432,
“expiration”: 0,
“vbid”: 149,
“id”: “0001:2014-01-01T00:00:00.000+0000”
}

I found a related topic, but it doesn’t help:

Cbtransfer import and export

Hello, I am slightly confused as to the logic of exporting in CSV, yet importing Documents. Basically it appears to be exporting data in a format that is not importable. Why does it work this way? What is the cbtransfer syntax that will export one bucket to a file that can be directly imported (without alteration) into another bucket (on localhost). When I do get JSON to be exported, I see all the Couchbase metadata in there (eg. values for ‘cas’, ‘vvid’, ‘rev’). When this is imported, Couchbas…

Please help me!
Thanks

[matt.carabine]

Hey @angeloassis,

This issue has been reported by a few people, I shall take a look tomorrow but I suspect that there may be a way to avoid adding this metadata by using the extra argument try_xwm=0 (I will have to test the exact behaviour though).

That said, I am not sure that cbtransfer was designed to perform a backup to csv files and then to restore from these same csv files. As the documentation states cbtransfer is designed to be a ‘lightweight extract-transform-load (ETL) tool that transfers data between clusters and to and from files’.

My question to you would be, if you intend to use this as a backup which you wish to later restore, why not instead use the tool cbbackup which has been specifically designed for this task, is there a reason that you want your backup format to be in the form of .csv files?

[angeloassis]

Hi Matt,

thanks for your help.
I already tried cbbackup tool, but the document format is the same for both commands. The CSV file is like this:

id,flags,expiration,cas,value,rev,vbid,dtype
0001:2014-01-01T00:00:00.000+0000,33554432,0,1457550024029372416,"{"“value”":2.479090921778098}",1,199,1

So the import continues with the problem.

The result is the same using the argument tre_xwm=0

My reason to use CSV is that I’m thinking to use a external tool to generate the CSV file and import it into couchbase, so I need to understand the formats to import and how it works

[tinygipxy]

If cant’ import csv then what is the purpose of export and why it’s documented on couchbase website ?
This topic is already 184 days old, but still in open state!

[sam.wilks92]

I’m also really frustrated this hasn’t been resolved. We have to restore data to a client and the most recent backup we have is a csv export. I need this to work!

[penacho]

I stumbled on the same problem today. cbtransfer can export cvs files and it can import cvs files, but not the same cvs files.

So, despite the tool is called cbtransfer, apparently you can not use it to transfer the content of one cluster to another (using cvs files).

[mahesh.mahadevan]

Same here … I am facing same problem , why this hasnt been resolved yet. Moreover , I had to resort to cbtransfer because cbrestore does not import all the documents as exported using cbbackup … that is another issue…

[RogerP]

Same issue here with couchbase version 5.0.0.
Is there a workaround? I’m just interested in the documents, not the meta data…