Importing json array from MongoDB

Couchbase Server
1

I am trying to migrate from MongoDB to CouchBase. I have exported each collection in MongoDB to a Json array since I thought that would be the most compatible format, now I am attempting to use cbdocloader to import the Json Array into CouchBase by putting all the collections json files into a folder and importing the folder, but I get the error:

Traceback (most recent call last):
File “/opt/couchbase/lib/python/cbdocloader”, line 237, in
main()
File “/opt/couchbase/lib/python/cbdocloader”, line 229, in main
docloader.populate_docs()
File “/opt/couchbase/lib/python/cbdocloader”, line 189, in populate_docs
self.enumerate_and_save()
File “/opt/couchbase/lib/python/cbdocloader”, line 151, in enumerate_and_save
self.save_doc(dockey, fp)
File “/opt/couchbase/lib/python/cbdocloader”, line 125, in save_doc
raw_data = datafile.read()
MemoryError

So can anyone point me to a tutorial on how to make this work, or else walk me through the error?

2

Hi,

Can you give us more information about the format of your json files and directory structure when you do the import?

Also you are talking about CouchDB, note that Couchbase and CouchDB are 2 different things. Could you confirm that you are using Couchbase?

Regards
@tgrall

3

Hi,
Sorry about the CouchDB confusion, that was just a typo, I have fixed it to say Couchbase. Here is an example of a collection that has been outputted from MongoDB to Json Array:

[{ “_id” : { “$oid” : “5001b9e5cb5b59a722e89e30” }, “user” : “showdown”, “readOnly” : false, “pwd” : “4e855ab7166cf9ad16a4bf98d08fbe86” },{ “_id” : { “$oid” : “5033d233f8640864f0625f9d” }, “user” : “showdown”, “readOnly” : false, “pwd” : “703cdf8f2b60b3b79f02a306c6c3c4b3” },{ “_id” : { “$oid” : “5033d24ff8640864f0625f9e” }, “user” : “admin”, “readOnly” : false, “pwd” : “d17e8b379b1121f36ffbc4e4f20fc63c” },{ “_id” : { “$oid” : “50cb72b88861ebbf5bf8a937” }, “user” : “showdown”, “readOnly” : false, “pwd” : “4e855ab7166cf9ad16a4bf98d08fbe86” },{ “_id” : { “$oid” : “50cb88a5ecadfd1c05f719d0” }, “user” : “rockmongo”, “readOnly” : false, “pwd” : “c3d8f5fd75a998aec0b6e4fa28d0a39a” }]

4

Hello,

You cannot use the cbdocloader directly on a Mongo export file.

The Mongo file contains either:

  • one json document by line
  • an array of json documents

where cbdocloader tool is using 1 file for each json document.

Also Mongo is using special types for ids and date, so it is sometimes useful to serialize it in a proper JSON format.

I have created a smal tool that allows you to copy your Mongo database/collections directly into Couchbase or you can export these documents into a directory that you can use directly with cbdocloader tool.

You can find the developer version here:

Note: feel free to test and give me feedback (even if this is still early stage).

Regards
Tug
@tgrall