There is a problem with json cbimport

[leech]

Hello. I Think There is a problem with json cbimport.

here is my Command I used.

C:\Users\leech>D:\Couchbase\Server\bin/cbimport json -c couchbase://localhost -u Administrator -p admin123 -b test -d file://C:/upload/testList.json -f list -g %name% -t 2
JSON file://C:/upload/testList.json imported to couchbase://localhost successfully
Documents imported: 2 Documents failed: 0

Content in Json file

[
{“name”: “caaaa”, “age”: “3339”},
{“name”: “baaaa”, “age”: “39”}
]

result here.

It’s normal up to here. Here’s my problem.

Why does the query say this? I have to use a query, but I don’t know what the problem is.

I’d be glad if you could help me.

Thank you.

[jamesl33]

Hi @leech,

This is interesting, the first thing to note is that you’re seeing this value for the document with key “baaa”.

“test”: “<binary (32 b)>”

This is because the document (as it’s currently stored) is not valid JSON i.e. it’s binary data. In theory the “cbimport json -f list” command will only successfully import data if it’s valid JSON so after a successful import I would expect the imported documents to also be valid JSON.

I’ve had a go a reproducing this issue with the data you’ve provided on a couple of different releases of Couchbase Server and I’m currently unable to do so. Please could you provide me with some additional information to aid with the debugging process.

1. What version of Couchbase server are you using?
2. Could you provide an attachment with the exact file you’re using?

Below are the setups I’m currently using to try to reproduce this issue:

1. Create a cluster with the data server, indexing service and the query service
2. Create a Couchbase bucket
3. Use cbimport to load the data
4. Create a primary index on the created bucket (using the query workbench)
5. Run the “select * from bucket” query
6. Inspect the results to determine if there are any which are binary data

Are there any other steps which I should be taking to match the behavior you’re seeing?

Thanks,
James

[leech]

Hello. @jamesl33 .

Thank you for trying to help.

I used Couchbase Server 7.0 Beta version.

Here is the file I used.
testList.zip (199 Bytes)

In the third stage of the stage you mentioned, I did it in cmd, not UI.

if you used cbimport in cmd, It matches my behavior.

So I tried it on the UI and found the following error.

I don’t understand this error message.

Why do I get this error message?

Thank you.

[jamesl33]

Hi @leech,

Thanks for the update, using the file you provided I’ve been able to reproduce the issue and it appears that you’ve hit multiple bugs.

1. MB-44423
2. MB-44424

The 7.0.0-beta cbimport tool switched to using a different JSON library; this library had a bug which would result in whitespace not being removed when parsing raw JSON values. If you look at the example document below, you’ll see that the value is prefixed with a carriage return/newline.

$ cbc cat xe -u Administrator -P password -U couchbase://192.168.2.22/test|xxd
xe                   CAS=0x1664d3b005930000, Flags=0x0, Size=31, Datatype=0x01(JSON)
00000000: 0d0a 2020 7b22 6e61 6d65 223a 2022 7865  ..  {"name": "xe
00000010: 222c 2022 6167 6522 3a20 2234 3422 7d    ", "age": "44"}

Please note the ‘0d0a’ which is ‘\r\n’ prefixed to the document value. This is something that’s has already been fixed in the underlying library and we updated the dependency in MB-44424 meaning this issue should be resolved for the second 7.0.0-beta.

This is interesting because as discussed in the JSON RFC this is completely valid JSON.

Insignificant whitespace is allowed before or after any of the six structural characters.

Although these documents are valid according to cbimport and the data service, they’re not valid according to the query service; this is the reason for the query returning documents which are binary data. I’ve raised MB-44423 to have this issue looked into.

Regarding the error that you’re seeing when importing via the UI, I believe this issue has already been addressed. Please see the screenshot below which shows what the popup now looks like.

Thanks,
James

[leech]

Hello @jamesl33 .

The cause of this problem was whitespace. I had no idea.

The problem was solved. Thank you.

However, this issue continued in the UI.

I don’t care much because I won’t use it in the UI, but I’m uploading it because I’m wondering if it’ll help.

If you think there is no problem, you can ignore it.

Thank you for your help.

testList.zip (194 Bytes)

[jamesl33]

Hi @leech,

Thanks for the update. With regards to the UI import tab error I believe you might be hitting/referring to MB-43946. In which case this issue has already been fixed; please refer to the previous screenshot which I shared which displays what the completion dialog now looks like.

Thanks,
James