Backtick handling in subdoc lookup

Couchbase Server
1

Hello,
I’m having trouble trying to store and retrieve documents containing ` (backtick) characters in document’s property names.

Following test is “successful”:

Test 
@Test
public void backtickTest()
{
	final String docId = UUID.randomUUID().toString();
	final JsonObject storedContent = JsonObject.from(ImmutableMap.of("prop`", 5));

	bucket.upsert(JsonDocument.create(docId, storedContent));

	final JsonDocument document = bucket.get(docId);
	assertThat(document.content()).isEqualTo(storedContent);

	final DocumentFragment<Lookup> lookupResult = bucket.lookupIn(docId).get("prop``").execute();
	assertThat(lookupResult.content(0)).isEqualTo(5); // <- FAILS on 5th run
}

Although it’s successful, it can be run only 4 times, on 5th try lookup result is null with status SUBDOC_PATH_NOT_FOUND. Starts working again after couchbase restart. Behavior is repeatable, each time after restart we can run 4 lookups, 5th and so on fails. For properties without backticks everything works like a charm at any point.

Following fails each time:

Failing test 
@Test
public void repeatedLookupBacktickTest()
{
	final String docId = UUID.randomUUID().toString();
	final JsonObject storedContent = JsonObject.from(ImmutableMap.of("prop`", 5));

	bucket.upsert(JsonDocument.create(docId, storedContent));

	DocumentFragment<Lookup> lookupResult = bucket.lookupIn(docId).get("prop``").execute();
	assertThat(lookupResult.content(0)).isEqualTo(5);

	lookupResult = bucket.lookupIn(docId).get("prop``").execute();
	assertThat(lookupResult.content(0)).isEqualTo(5); // <- FAILS
}

Second lookup always return null with status SUBDOC_PATH_NOT_FOUND.

When document is inserted using subdoc api, lookup also always fails:

Failing test (subdoc insert) 
@Test
public void subdocUpsertBacktickTest()
{
	final String docId = UUID.randomUUID().toString();
	final JsonObject storedContent = JsonObject.from(ImmutableMap.of("prop`", 5));

	bucket.mutateIn(docId).upsertDocument(true)
			.upsert("prop``", 5).execute();

	final JsonDocument document = bucket.get(docId);
	assertThat(document.content()).isEqualTo(storedContent);

	final DocumentFragment<Lookup> lookupResult = bucket.lookupIn(docId).get("prop``").execute();
	assertThat(lookupResult.content(0)).isEqualTo(5); // <- FAILS
}

As before, lookup result is null with status SUBDOC_PATH_NOT_FOUND.

Everything above was tested against couchbase 5.0.1 and 5.1.1 (docker containers), using both sync and async apis. Client used was Java 2.5.4, however all issues are reproductible using python client, so the problem seems not to be client-related.

Did anyone encounter similar issues? Is there any workaround?

Regards,
Krzysztof

2

Hi Krzysztof,

I haven’t seen anything similar - thanks for reporting this. Would it be possible for you to provide a Wireshark / tcpdump network capture between the client and server when running this test?

3

Thanks for quick response.

Here is a tcp dump from when I ran 3rd test (as a new user I can’t upload files here):
https://pastebin.com/5AdYjn4b

I hope it helps.

4

Could you capture / upload in pcap format? (Wireshark has support for dissecting the Couchbase binary protocol; but that requires the a pcap file and not the plain text output).

5

pcap for the same test:

6

Thanks for the pcap. So some of what you’re seeing is expected; some of it appears to be less so :slight_smile:

Looking at your trace in Wireshark; the initial Dict_Upset of path “path``” succeeds; creating an element with key “path`” (note the single backtick):

24
Screen Shot 2018-06-27 at 14.02.24.png1463×903 196 KB

42
Screen Shot 2018-06-27 at 14.02.42.png1463×903 194 KB

This perhaps doesn’t do what you initially expected - backtrack is actually the path escape symbol; used when your key contains path metacharacters.

The backtick would be required to actually access this path - as “name.with.periods”. For example, if your object key was “name.with.periods”; then raw that’s interpreted by the Subdoc API as looking up a nested dictionary of the form:

"name": {
    "with": {
        "periods": ...
    }
}

Thus far things are working as expected. However, the request you make to read the specific path via subdoc is failing:

17
Screen Shot 2018-06-27 at 14.03.17.png1463×903 193 KB

I’m still looking into this…

7

It does do exactly what I expected. I know backtick is used to escape dots and so on in property name, but I also want to use it just as one of characters in property name :wink:.

8

This is indeed a bug in how escaped paths are parsed - thanks for reporting it. We will have a fix for it in the next major release (5.5.0).

1 Like
9

Ok, thanks for letting me know.