Ok @ingenthr , I now have a consistent case
Situation: I have an email and I want to find the key for the user and then load the entire user. The user info is split into two documents:
-
User
that contains the “non-personal” info of the user
-
Private
that contains personal info. This makes it easier to control where the personal information goes (e.g. in the Sync.Gateway).
So first I search for any Private
documents with the email address. If found then we have a key. We then use that key to lookup the User
directly by using ...get("User:2124DEFEC111BA8FC1257ED20034B387") and if found we get the
Privatedocument directly by using
…get(“User:Private:2124DEFEC111BA8FC1257ED20034B387”) - and we then combine the two documents into one User object in the code.
So with the above in mind this output should be clearer:
CouchbaseUserDAO.loadUser: Find by email: john@dalsgaard-data.dk
BaseCouchbaseDAO.getAllFromDb: Query=SELECT * FROM data WHERE type='Private' AND (lower(email)=lower('john@dalsgaard-data.dk'))
BaseCouchbaseDAO.getAllFromDb: Result from Query=N1qlQueryResult{status='success', finalSuccess=true, parseSuccess=true, allRows=[{"data":{"address":"Solbjergvej 42\r\nSolbjerg","cellphone":"40566308","city":"Høng","country":"DK","email":"john@dalsgaard-d
ta.dk","key":"2124DEFEC111BA8FC1257ED20034B387","name":"John Dalsgaard","parenttype":"User","password":"x5w77zZoW3","phone":"49141248","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","zip":"4270"}}], signature={"*":"*"}, info=N1qlM
trics{resultCount=1, errorCount=0, warningCount=0, mutationCount=0, sortCount=0, resultSize=348, elapsedTime='6.388261ms', executionTime='6.284091ms'}, profileInfo={}, errors=[], requestId='016d8d47-7303-403a-8774-687df685a030', clientContextId='dcda9323-
4f2-4f0b-9176-4a1efe13cc4c'}
CouchbaseUserDAO.getKeyByEmail: Result={"data":{"zip":"4270","country":"DK","address":"Solbjergvej 42\r\nSolbjerg","city":"Høng","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","password":"x5w77zZoW3","phone":"49141248","name":"Joh
Dalsgaard","cellphone":"40566308","parenttype":"User","email":"john@dalsgaard-data.dk","key":"2124DEFEC111BA8FC1257ED20034B387"}}
CouchbaseUserDAO.loadUser: Found key: 2124DEFEC111BA8FC1257ED20034B387
CouchbaseUserDAO.loadUser: Find by direct key: User:2124DEFEC111BA8FC1257ED20034B387
CouchbaseUserDAO.loadUser: Found user doc. Get private doc: User:Private:2124DEFEC111BA8FC1257ED20034B387
Rest.handle: ERROR - java.lang.RuntimeException: Could not decode snappy-compressed value.
Rest.dumpRawExceptionData: ERROR - Exception: CouchbaseException - java.lang.RuntimeException: Could not decode snappy-compressed value.
com.couchbase.client.core.endpoint.AbstractGenericHandler.decode(AbstractGenericHandler.java:369)
com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec$2.decode(MessageToMessageC...
com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMes...
com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessa...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMes...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerC...
com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessa...
com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDe...
com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChann...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
com.couchbase.client.deps.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler....
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(Default...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelP...
com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(Abstract...
com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:...
com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEvent...
com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java...
com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEv...
com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRun...
java.lang.Thread.run(Thread.java:811)
This is the code that fails:
Util.info("Found user doc. Get private doc: " + getPrivateDocId(key));
--> JsonDocument privDoc = getDb().get(getPrivateDocId(key));
And here is the output of the cbc cat
command:
cbc cat User:Private:2124DEFEC111BA8FC1257ED20034B387 -U http://db1/data -u xxxxx -P yyyyy
User:Private:2124DEFEC111BA8FC1257ED20034B387 CAS=0x15bd8c734d220000, Flags=0x2000000, Size=339, Datatype=0x01(JSON)
{"address":"Solbjergvej 42\r\nSolbjerg","cellphone":"40566308","city":"Høng","country":"DK","email":"john@dalsgaard-data.dk","key":"2124DEFEC111BA8FC1257ED20034B387","name":"John Dalsgaard","parenttype":"User","password":"x5w77zZoW3","phone":"49141248","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","zip":"4270"}
I just queried the document from the db admin panel (where the lookup by id works fine). The meta data is:
{
"meta": {
"id": "User:Private:2124DEFEC111BA8FC1257ED20034B387",
"rev": "15-15bd8c734d2200000000000002000000",
"expiration": 0,
"flags": 33554432,
"type": "json"
},
"xattrs": {
"_sync": {
"rev": "5-a5e4c60689b9662026bb4fb9f2ef1e5d",
"sequence": 424079,
"recent_sequences": [
187871,
424071,
424074,
424076,
424079
],
"history": {
"revs": [
"4-2565a10f9deddeece8201479ce81441a",
"5-a5e4c60689b9662026bb4fb9f2ef1e5d",
"1-92b7043cf752430e905306c52b043359",
"2-7432f4999e86db5c881f69a70036dd7e",
"3-1b3a53baf64c38677f9effeaebfe94b7"
],
"parents": [
4,
0,
-1,
2,
3
],
"channels": [
[
"channel.2124DEFEC111BA8FC1257ED20034B387"
],
[
"channel.2124DEFEC111BA8FC1257ED20034B387"
],
[
"channel.2124DEFEC111BA8FC1257ED20034B387"
],
[
"channel.2124DEFEC111BA8FC1257ED20034B387"
],
[
"channel.2124DEFEC111BA8FC1257ED20034B387"
]
]
},
"channels": {
"channel.2124DEFEC111BA8FC1257ED20034B387": null
},
"access": {
"2124DEFEC111BA8FC1257ED20034B387": {
"channel.2124DEFEC111BA8FC1257ED20034B387": 187871
}
},
"cas": "0x0000224d738cbd15",
"value_crc32c": "0x3adda133",
"time_saved": "2019-08-23T14:17:52.235653954+02:00"
}
}
}
And this is consistent!
Please let me know what I can do to further help?
Thoughts:
This system has been migrated from another database - so in some places I guess it could be optimized. In this case I do the SELECT on the private doc to find it by email and after loading the user doc I load the private doc again. But as I understand it, a direct lookup is quick - so I haven’t bothered to do anything about it here…
This is also a development/test environment - so earlier versions of the sync. formula could also have done “funny” things to the data. I guess you may be able to detect if this is the issue by looking at the meta data. If this is the case then I could just start over from scratch with a fresh copy from the production database (but I want to do this right: Best way to clone a bucket to a demo environment?)