Couchbase JsonObject toString() method does not escape special characters


#1

Hi all,

I am using Couchbase Java Client 2.0.1 to interact with Couchbase server 3.0.1.
I have noticed that the toString() method of Couchbase JsonObject does not escape special characters (escape sequences) of string values - resulting in an invalid Json string that cannot be serialized back to a Json object (using Jackson for instance).

Document example (as viewed through Couchbase UI)-

{
  "data": {
    "source1": {
      "description": "test description \n blabla"
    }
  },
  "id": 1234556
}

And here is my piece of code -

...
JsonDocument doc =  bucket.get(key);
JsonObject content = getDocument(key).content();
String strObj = content.toString();
Map<String,Object> jsonObj = jacksonMapper.readValue(strObj, Map.class);
...

As a result strObj value is -

'{"data":{"source1":{"description": "test description \n blabla"}},"id": 1234556}'

Instead of being -

'{"data":{"source1":{"description": "test description \\n blabla"}},"id": 1234556}'

Making jacksonMapper.readValue() fail and throw the following exception -

com.fasterxml.jackson.core.JsonParseException: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

Just for the reference, here is the toString() method of Couchbase JsonObject -

/**
 * Converts the {@link JsonObject} into its JSON string representation.
 *
 * @return the JSON string representing this {@link JsonObject}.
 */
@Override
public String toString() {
    StringBuilder sb = new StringBuilder("{");
    int size = content.size();
    int item = 0;
    for(Map.Entry<String, Object> entry : content.entrySet()) {
        sb.append("\"").append(entry.getKey()).append("\":");
        if (entry.getValue() instanceof String) {
            sb.append("\"").append(entry.getValue()).append("\"");
        } else {
            if (entry.getValue() == null) {
                sb.append("null");
            } else {
                sb.append(entry.getValue().toString());
            }
        }
        if (++item < size) {
            sb.append(",");
        }
    }
    sb.append("}");
    return sb.toString();
}

Note that special characters are not being escaped in the following line of code (taken from the toString method) -

...
sb.append("\"").append(entry.getValue()).append("\"");
...

By the way, this is how I indexed the document into Couchbase in the first place -

public RawJsonDocument updateDocument(String key, Map<String,Object> content) 
{
	RawJsonDocument doc = RawJsonDocument.create(key, jacksonMapper.writeValueAsString(content));
	return bucket.upsert(doc);
}

What is your opinion about it?
Am I doing something wrong?
Is it a bug?

Thanks!!
Ronen.


#2

By the way, it can be solved by Jackson using the following configuration -

   	     jacksonMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
	     jacksonMapper.configure(JsonParser.Feature.ALLOW_BACKSLASH_ESCAPING_ANY_CHARACTER, true);

I was just wondering what do you think about it?

The description of the toString() method is:

Converts the {@link JsonObject} into its JSON string representation

Figured it should be a valid Json string.

Thanks again.
Ronen.


#3

@Taykey yup this seems like a bug. I created a ticket to track it - https://issues.couchbase.com/browse/JCBC-663 … we also love contributions! :wink:


#4

@daschl Thanks for the response!

Another related issue that probably requires its own post but also worth mentioning here.

JsonObject.toMap() method returns the underlying Map representation, while one of the values can be a JsonObject by itself (the mapping is not recursive).

/**
 * Creates a copy of the underlying {@link Map} and returns it.
 *
 * @return the {@link Map} of the content.
 */
public Map<String, Object> toMap() {
    return new HashMap<String, Object>(content);
}

For instance, I retrieved the following Couchbase document with a the nested json content -

{
  "name": "david",
  "description": {
    "place": "london"
  }
}

...
JsonObject myContent = myDoc.content();
Map<String, Object> myMap = myContent.toMap();
...

As a result myMap is of a Map type, but myMap.get(“description”) is of a JsonObject type.
I would have expected it to be a Map as well. Actually the all object (myMap) should be totally independent of couchbase - In a way that serializing the Map object with any 3rd party library (a valid one) will construct a valid Json string.

For instance, using Jackson -

String contentJsonStr = jacksonMapper.writeValueAsString(myContent.toMap());


Back to the previous issue (/case/post) -

It could have helped because I could have easily used -

String strObj = jacksonMapper.writeValueAsString(content.toMap());

Instead of using -

String strObj = content.toString();

And have a workaround for those toString() issues previously mentioned.


What do you think?

Thanks!
Ronen.


#5

Hi @Taykey,
I’ve fixed the parsing issue this morning (see jira issue above), but it is for now only committed in master branch, which correspond to our future 2.1 release. Note that there’ll be a sooner 2.0.3 release with this fix as well, but for the moment the only place you can get a peak at the change is master on github.

I don’t know about toMap, the implementation as is is pretty simple and can rely on the backing Map, whereas what you suggest would imply to iterate over the whole Map, potentially recursively… But having a deep conversion to Map still makes sense. @daschl what do you think?


#6

Hi @simonbasle,
I appreciate the quick response and fix - thanks!

As for the toMap(), it is possible to keep it relatively simple, either with a manual recursive conversion or by using JsonObject.toString() (the fixed one) and deserializing with Jackson to a Map object.

What do you think?


#7

Well, using toString and then back to Map seems very wasteful to me, given that we already have a map.

I think you are right that in the toMap implementation, we should convert all JsonArrays to a list and all JsonObjects to a map (so basically the opposite conversion). I think there is some internal code relying on it which needs to be adapted, but it is definitely the more strict thing to do.

I created an enhancement, please lets follow up there: https://issues.couchbase.com/browse/JCBC-669


#8

Ack, thank you @daschl and @simonbasle


#9

@Taykey the modification of toMap and toList has been merged into master just in time before the end of the year :smile:.
it’ll be part of 2.1.0 release (not 2.0.3 I’m afraid). toMap now returns a copy of the underlying content where each JsonArray is converted recursively to a List and each JsonObject to a Map.


#10

(for clarification, the original issue JCBC-663 on character escaping have been resolved and merged in time for 2.0.3 release)