Ah, now I see what you mean. This is because query engine uses JSON parser to sniff and determine if the value payload is JSON, while all SDKs agreed on using flags to encode type information.
Well, we need to have the same behavior no ?
So decode JSON on get and query or never.
Yes, it should be consistent. I will raise it in SDK group, and get back to you here
Thank you 
Iām having a major issue. Weāre using Bucket ->get and it seems that at random some documents are being converted to strings. Itās really hard to explain, I canāt get my head around it.
$couchResult = $couchBucket->get('playerdata:' . $uuid);
$data = json_decode(json_encode($couchResult->value), true);
is the code weāre using but when echoing the data with echo json_encode($data) some data is coming back fine but other data is coming back as a string version of JSON (it seems to have been encoded twice).
Some of the data is, when var_dumpāed coming back as array and other times itās coming back as string with lots of escaped JSON. Itās a mess.
if(is_string($data)){echo $data;}else{if(is_array($data)){echo json_encode($data);}}
That line works but itās really not ideal and I want to know what on earthās causing thisā¦
object(CouchbaseMetaDoc)#94 (5) { [āerrorā]=> NULL [āvalueā]=> string(2053)
whereas Iām actually expecting:
object(CouchbaseMetaDoc)#58 (5) { [āerrorā]=> NULL [āvalueā]=> object(stdClass)#94 (21) {
Some data is coming through as expected whilst some is coming as a string.
Each var_dumpās there are taken at the same place in the code but on different executions (different user ID provided).
$couchResult = $couchBucket->get(āplayerdata:ā . $uuid);
What theā¦?
why do you do this at all? if you donāt like stdClass, just cast it with (array).
when it comes as a string, that means you have set it as a string, not as an object. If you really need data conversion and encoding, you have to write your own transcoder, and do not manipulate on the value. Because transcoder also affects flags.
please, show the "flags" property of the CouchbaseMetaDoc
[āflagsā]=> int(0) [ācasā]=> string(9) āai2nndgr8ā [ātokenā]=> NULL } for the data thatās returning as a string.
[āflagsā]=> int(33554432) [ācasā]=> string(10) ā1yskv0bnysā [ātokenā]=> NULL } for the data thatās correct.(stdClass)
in first case flags tell you āthis is RAW binary BLOB, and should be represented as a bytestringā, so the SDK is right in this case
How did you set that key?
What? How did we put it into the database in the first place? Same was as the other one - through the Java SDK upsert.
In java SDK you are using JsonDocument, right? Could you post sample of the code, which writes the docs (which occasionally have zero flags)?
/cc @daschl
Can try but this hasnāt been a problem beforeā¦?
Iām not the Java developer and heās away right now so I can try decompile. I know it uses Gson.
In this case they have to use their custom code, because by default we use Jackson-based encoder with JsonDocument. If so, Iād like to take a look at their transcoder.
public static Object getData(UUID uuid, String data, Object def, boolean create)
{
String[] dataSplit = data.split("\\.");
Bucket b = connect();
JsonDocument j = b.get("playerdata:" + uuid.toString());
JsonObject d;
if (j == null)
{
if (!create) {
return null;
}
JsonObject d = JsonObject.empty();
d.put("uuid", uuid.toString());
d.put("rank", "DEFAULT");
JsonObject s = JsonObject.empty();
s.put("chat_ping", true);
s.put("chat", true);
s.put("flight", false);
d.put("settings", s);
b.insert(JsonDocument.create("playerdata:" + uuid.toString(), d));
}
else
{
d = (JsonObject)j.content();
}
if (dataSplit.length > 1)
{
JsonObject change = d;
for (int i = 0; i < dataSplit.length - 1; i++)
{
boolean f = i >= dataSplit.length - 2;
JsonObject jo = change.getObject(dataSplit[i]);
if (jo == null) {
jo = JsonObject.empty();
}
if (f)
{
if (jo.containsKey(dataSplit[(i + 1)])) {
return jo.get(dataSplit[(i + 1)]);
}
setData(uuid, data, def);
return def;
}
if (!jo.containsKey(dataSplit[(i + 1)])) {
jo.put(dataSplit[(i + 1)], JsonObject.empty());
}
change.put(dataSplit[i], jo);
change = jo;
}
}
if ((j == null) || (((JsonObject)j.content()).get(data) == null))
{
setData(uuid, data, def);
return def;
}
return ((JsonObject)j.content()).get(data);
}
This may illustrate it better:
public static void setData(UUID uuid, String data, Object value)
{
String[] dataSplit = data.split("\\.");
Bucket b = connect();
JsonDocument j = b.get("playerdata:" + uuid.toString());
if (j == null)
{
JsonObject d = JsonObject.empty();
d.put("uuid", uuid.toString());
d.put("rank", "DEFAULT");
JsonObject s = JsonObject.empty();
s.put("chat_ping", true);
s.put("chat", true);
s.put("flight", false);
d.put("settings", s);
b.insert(JsonDocument.create("playerdata:" + uuid.toString(), d));
}
else
{
if (dataSplit.length > 1)
{
JsonObject change = (JsonObject)j.content();
for (int i = 0; i < dataSplit.length - 1; i++)
{
boolean f = i >= dataSplit.length - 2;
JsonObject jo = change.getObject(dataSplit[i]);
if (jo == null) {
jo = JsonObject.empty();
}
if (f) {
jo.put(dataSplit[(i + 1)], value);
} else if (!jo.containsKey(dataSplit[(i + 1)])) {
jo.put(dataSplit[(i + 1)], JsonObject.empty());
}
change.put(dataSplit[i], jo);
change = jo;
}
}
else
{
((JsonObject)j.content()).put(data, value);
}
b.upsert(j);
}
}
Itās that bad that Iāve had to pull my site down and create a dev subdomain that only I can access until this is resolvedā¦
Do we know if this is an issue with PHP or Java? (It always was working fine earlier before Iāve done a code overhaul on PHPās side).
Itās a database issue. All of the documents that are being returned as strings, as per @avsej suspicions, have flags: 0 in the database. Is there any way to resolve this so that they can be retrieved the same as any other data?
Iāve read up on flags and found the following.
When the SDK serializes the Document, it notes the type of serialization performed (JSON) and sends a corresponding type code along with the serialized document to the server to be stored. This type code is stored in the flags field within the itemās metadata.
Does this mean that itās a fault with where the data is being upserted? e.g. in the Java code?
Yes, it might be. Also some operations, which supposed to create document, does not allow to specify flags in payload (like counters).
Just to clarify, you donāt create documents through UI, right?
For your application you could wrap default transcoders like this to workaround zero flags (if you are sure that the documents are always JSON and none of the writers uses zero intentionally)
<?php
$cluster = new CouchbaseCluster("couchbase://localhost");
$bucket = $cluster->openBucket("default");
printf("Default transcoders\n");
var_dump($bucket->get("foo_json_flags"));
var_dump($bucket->get("foo_zero_flags"));
$bucket->setTranscoder(
// encoder($value) -> array($bytes, $flags, $datatype)
couchbase_default_encoder,
// decoder($bytes, $flags, $datatype) -> $value
function ($bytes, $flags, $datatype) {
if ($flags == 0) {
$flags = COUCHBASE_CFFMT_JSON;
}
return couchbase_default_decoder($bytes, $flags, $datatype);
}
);
printf("Wrapped transcoders\n");
var_dump($bucket->get("foo_json_flags"));
var_dump($bucket->get("foo_zero_flags"));
The output is like this
Default transcoders
object(CouchbaseMetaDoc)#5 (5) {
["error"]=>
NULL
["value"]=>
object(stdClass)#4 (1) {
["foo"]=>
string(3) "bar"
}
["flags"]=>
int(33554438)
["cas"]=>
string(10) "20vtgyjz10"
["token"]=>
NULL
}
object(CouchbaseMetaDoc)#5 (5) {
["error"]=>
NULL
["value"]=>
string(13) "{"foo":"bar"}"
["flags"]=>
int(0)
["cas"]=>
string(10) "2a9yzs4844"
["token"]=>
NULL
}
Wrapped transcoders
object(CouchbaseMetaDoc)#6 (5) {
["error"]=>
NULL
["value"]=>
object(stdClass)#4 (1) {
["foo"]=>
string(3) "bar"
}
["flags"]=>
int(33554438)
["cas"]=>
string(10) "20vtgyjz10"
["token"]=>
NULL
}
object(CouchbaseMetaDoc)#4 (5) {
["error"]=>
NULL
["value"]=>
object(stdClass)#6 (1) {
["foo"]=>
string(3) "bar"
}
["flags"]=>
int(0)
["cas"]=>
string(10) "2a9yzs4844"
["token"]=>
NULL
}