Can't load JSON document with more than 13863 fields

Couchbase Server
1

Hello there,
I have have the following problem:

**When loading a JSON document with more than 13863 fields, I get the following error. **
**Funny thing is it works fine with 13862 JSON fileds. **
I’d be thankful for help/oppinions. :slight_smile:

I am using Java SDK.
com.couchbase.client
java-client
2.5.5

Output:
filed# 13863

Error while encoding Request, cancelling.
II 26, 2019 7:38:17 PM com.couchbase.client.core.endpoint.AbstractGenericHandler exceptionCaught
WARNING: [/192.168.56.100:11210][KeyValueEndpoint]: Caught unknown exception: java.lang.RuntimeException: Could not snappy-compress value.
com.couchbase.client.deps.io.netty.handler.codec.EncoderException: java.lang.RuntimeException: Could not snappy-compress value.
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:106)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec.write(MessageToMessageCodec.java:116)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.write(AbstractGenericHandler.java:257)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:732)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:724)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:35)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1061)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1050)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Could not snappy-compress value.
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:307)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:280)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:131)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:268)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:81)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec$1.encode(MessageToMessageCodec.java:67)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88)
… 12 more
Caused by: java.lang.IndexOutOfBoundsException: index: -32690, length: 4 (expected: range(0, 36643))
at com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1118)
at com.couchbase.client.deps.io.netty.buffer.SlicedByteBuf.getInt(SlicedByteBuf.java:153)
at com.couchbase.client.core.endpoint.util.Snappy.encode(Snappy.java:131)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:305)
… 18 more

II 26, 2019 7:38:17 PM com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline onUnhandledInboundException
WARNING: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
com.couchbase.client.deps.io.netty.handler.codec.EncoderException: java.lang.RuntimeException: Could not snappy-compress value.
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:106)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec.write(MessageToMessageCodec.java:116)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.write(AbstractGenericHandler.java:257)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:732)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:724)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:35)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1061)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1050)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Could not snappy-compress value.
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:307)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:280)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:131)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:268)
at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:81)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec$1.encode(MessageToMessageCodec.java:67)
at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88)
… 12 more
Caused by: java.lang.IndexOutOfBoundsException: index: -32690, length: 4 (expected: range(0, 36643))
at com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1118)
at com.couchbase.client.deps.io.netty.buffer.SlicedByteBuf.getInt(SlicedByteBuf.java:153)
at com.couchbase.client.core.endpoint.util.Snappy.encode(Snappy.java:131)
at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:305)
… 18 more

2

That’s interesting. Does size fit impact it too?

First, it’d be interesting to know if you see the same behavior with the latest SDK. 2.7.4 as of this writing. We update dependencies and maybe there’s an issue fixed in the snappy lib?

I’m guessing it might be the same though and then we’d need to look into it.

Perhaps as a workaround, you can adjust the threshold for compression so it won’t attempt to compress. See the docs for details on the settings.

3

@mkrustev Did you find a resolution to this? I have a client hitting this error, and this is the only page on Google that matches the exact error message other than the actual source code :confused:

4

@ingenthr @Brad_Wood
We upgraded to the latest SDK 2.7.14 and this has resolved our issue. Thanks for all your help.

1 Like