What does this flag mean?

Other
1

We are reading items written by the Perl driver that should be just binary, and we get this flag: 50331672 for item type while the same items written in Go have 50331648.

End result is that Perl cannot read what Go writes.

What does the 50331672 flag mean? Maybe by understanding that we can write our records in a way that he Perl can understand what we write.

2

Could you point to Perl driver implementation? Because there is no official Couchbase SDK for Perl.

Supported SDKs use this RFC for encoding information in the flags: https://github.com/couchbaselabs/sdk-rfcs/blob/master/rfc/0020-common-flags.md

3

We are using:

http://search.cpan.org/~mnunberg/Couchbase-2.0.3/lib/Couchbase/Bucket.pm

4

mnunberg is sometimes reading here, but afaik he don’t have time to do active development on the perl library. but while it’s more or less a wrapper around the official C library maybe the support has a idea?

do you have sample code? just writing binary in perl and then try to read in perl? I know it works with JSON and simple text, we use that in a mixed go/perl environment every day.

0x03 << 24 is 50331648, so should be the same as Go.

5

the common flags is mostly about JSON. You can see that both flags have (0x01 << 24), which means they are using “private” encoding, which allowed to be non-portable.

6

So we fixed it but it’s super dirty.

When writing a binary to CB, Perl uses this bizarre 50331672 flag. Go can read it, but when writing back to CB, the flag is (correctly) set to 50331648.
Then Perl cannot read it anymore.

So we had to write a transcoder that sets the flag to 50331672. We don’t know why the perl driver is not capable of handling the good flag.

7

thanks piero, very interesting topic. I tried it today and get the same issue. Also sometimes the other way: items written in perl can’t read in go. I’ve never seen it before, because we use only JSON. Maybe best is, if possible, to avoid binary data completly and store them base64 encoded inside a JSON.

8

I also had to study the ‘flags’ field recently, and let me share my findings, so other people can quickly see how to work with it.

flags value (decimal) flags value (binary) bits #[5,6,7,8] decimal type note
16777216 00000001000000000000000000000000 0001 1 Private couchbase docs example
33554432 00000010000000000000000000000000 0010 2 JSON couchbase docs example
33554438 00000010000000000000000000000110 0010 2 JSON default from http rest api
50331648 00000011000000000000000000000000 0011 3 Raw Binary couchbase docs example
67108864 00000100000000000000000000000000 0100 4 Non-JSON String couchbase docs example
67108868 00000100000000000000000000000100 0100 4 Non-JSON String extra ‘JSON.stringify(entry)’ with nodejs official sdk 
// how to convert values in browser console

0x02 << 24
// 33554432

(33554432).toString(2).padStart(32, '0')
// "00000010000000000000000000000000"

"00000010000000000000000000000000".substring(4, 8)
// "0010"

parseInt("0010", 2)
// 2

The same as a gist: