Strange issue: Could not decode snappy-compressed value

jda · October 1, 2019, 7:09am

Ok @ingenthr , I now have a consistent case

Situation: I have an email and I want to find the key for the user and then load the entire user. The user info is split into two documents:

User that contains the “non-personal” info of the user
Private that contains personal info. This makes it easier to control where the personal information goes (e.g. in the Sync.Gateway).

So first I search for any Private documents with the email address. If found then we have a key. We then use that key to lookup the User directly by using ...get("User:2124DEFEC111BA8FC1257ED20034B387") and if found we get thePrivatedocument directly by using…get(“User:Private:2124DEFEC111BA8FC1257ED20034B387”) - and we then combine the two documents into one User object in the code.

So with the above in mind this output should be clearer:

CouchbaseUserDAO.loadUser: Find by email: john@dalsgaard-data.dk
BaseCouchbaseDAO.getAllFromDb: Query=SELECT * FROM data WHERE type='Private' AND (lower(email)=lower('john@dalsgaard-data.dk'))
BaseCouchbaseDAO.getAllFromDb: Result from Query=N1qlQueryResult{status='success', finalSuccess=true, parseSuccess=true, allRows=[{"data":{"address":"Solbjergvej 42\r\nSolbjerg","cellphone":"40566308","city":"Høng","country":"DK","email":"john@dalsgaard-d
ta.dk","key":"2124DEFEC111BA8FC1257ED20034B387","name":"John Dalsgaard","parenttype":"User","password":"x5w77zZoW3","phone":"49141248","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","zip":"4270"}}], signature={"*":"*"}, info=N1qlM
trics{resultCount=1, errorCount=0, warningCount=0, mutationCount=0, sortCount=0, resultSize=348, elapsedTime='6.388261ms', executionTime='6.284091ms'}, profileInfo={}, errors=[], requestId='016d8d47-7303-403a-8774-687df685a030', clientContextId='dcda9323-
4f2-4f0b-9176-4a1efe13cc4c'}
CouchbaseUserDAO.getKeyByEmail: Result={"data":{"zip":"4270","country":"DK","address":"Solbjergvej 42\r\nSolbjerg","city":"Høng","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","password":"x5w77zZoW3","phone":"49141248","name":"Joh
 Dalsgaard","cellphone":"40566308","parenttype":"User","email":"john@dalsgaard-data.dk","key":"2124DEFEC111BA8FC1257ED20034B387"}}
CouchbaseUserDAO.loadUser: Found key: 2124DEFEC111BA8FC1257ED20034B387
CouchbaseUserDAO.loadUser: Find by direct key: User:2124DEFEC111BA8FC1257ED20034B387
CouchbaseUserDAO.loadUser: Found user doc. Get private doc: User:Private:2124DEFEC111BA8FC1257ED20034B387
Rest.handle: ERROR - java.lang.RuntimeException: Could not decode snappy-compressed value.
Rest.dumpRawExceptionData: ERROR - Exception: CouchbaseException - java.lang.RuntimeException: Could not decode snappy-compressed value.
   com.couchbase.client.core.endpoint.AbstractGenericHandler.decode(AbstractGenericHandler.java:369)
   com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec$2.decode(MessageToMessageC...
   com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMes...
   com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessa...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
   com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMes...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
   com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerC...
   com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessa...
   com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDe...
   com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChann...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
   com.couchbase.client.deps.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler....
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstract...
   com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(Default...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra...
   com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelP...
   com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(Abstract...
   com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:...
   com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEvent...
   com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java...
   com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
   com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEv...
   com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRun...
   java.lang.Thread.run(Thread.java:811)

This is the code that fails:

	Util.info("Found user doc. Get private doc: " + getPrivateDocId(key));
-->	JsonDocument privDoc = getDb().get(getPrivateDocId(key));

And here is the output of the cbc cat command:

cbc cat User:Private:2124DEFEC111BA8FC1257ED20034B387 -U http://db1/data -u xxxxx -P yyyyy
User:Private:2124DEFEC111BA8FC1257ED20034B387 CAS=0x15bd8c734d220000, Flags=0x2000000, Size=339, Datatype=0x01(JSON)
{"address":"Solbjergvej 42\r\nSolbjerg","cellphone":"40566308","city":"Høng","country":"DK","email":"john@dalsgaard-data.dk","key":"2124DEFEC111BA8FC1257ED20034B387","name":"John Dalsgaard","parenttype":"User","password":"x5w77zZoW3","phone":"49141248","sex":"M","type":"Private","userkey":"2124DEFEC111BA8FC1257ED20034B387","zip":"4270"}

I just queried the document from the db admin panel (where the lookup by id works fine). The meta data is:

{
  "meta": {
    "id": "User:Private:2124DEFEC111BA8FC1257ED20034B387",
    "rev": "15-15bd8c734d2200000000000002000000",
    "expiration": 0,
    "flags": 33554432,
    "type": "json"
  },
  "xattrs": {
    "_sync": {
      "rev": "5-a5e4c60689b9662026bb4fb9f2ef1e5d",
      "sequence": 424079,
      "recent_sequences": [
        187871,
        424071,
        424074,
        424076,
        424079
      ],
      "history": {
        "revs": [
          "4-2565a10f9deddeece8201479ce81441a",
          "5-a5e4c60689b9662026bb4fb9f2ef1e5d",
          "1-92b7043cf752430e905306c52b043359",
          "2-7432f4999e86db5c881f69a70036dd7e",
          "3-1b3a53baf64c38677f9effeaebfe94b7"
        ],
        "parents": [
          4,
          0,
          -1,
          2,
          3
        ],
        "channels": [
          [
            "channel.2124DEFEC111BA8FC1257ED20034B387"
          ],
          [
            "channel.2124DEFEC111BA8FC1257ED20034B387"
          ],
          [
            "channel.2124DEFEC111BA8FC1257ED20034B387"
          ],
          [
            "channel.2124DEFEC111BA8FC1257ED20034B387"
          ],
          [
            "channel.2124DEFEC111BA8FC1257ED20034B387"
          ]
        ]
      },
      "channels": {
        "channel.2124DEFEC111BA8FC1257ED20034B387": null
      },
      "access": {
        "2124DEFEC111BA8FC1257ED20034B387": {
          "channel.2124DEFEC111BA8FC1257ED20034B387": 187871
        }
      },
      "cas": "0x0000224d738cbd15",
      "value_crc32c": "0x3adda133",
      "time_saved": "2019-08-23T14:17:52.235653954+02:00"
    }
  }
}

And this is consistent!

Please let me know what I can do to further help?

Thoughts:
This system has been migrated from another database - so in some places I guess it could be optimized. In this case I do the SELECT on the private doc to find it by email and after loading the user doc I load the private doc again. But as I understand it, a direct lookup is quick - so I haven’t bothered to do anything about it here…

This is also a development/test environment - so earlier versions of the sync. formula could also have done “funny” things to the data. I guess you may be able to detect if this is the issue by looking at the meta data. If this is the case then I could just start over from scratch with a fresh copy from the production database (but I want to do this right: Best way to clone a bucket to a demo environment?)