Couchbase JVM client threading issues

[moscicky]

Hi, we are using couchbase.client:3:0:10. We have noticed that when calling

bucket.reactive.defaultCollection()
.get(id)
.map [...] //more processing downstream

all operators downstream of get() are executing on cb-io-kv thread pool, which is a dedicated thread pool for netty event loop. Our processing pipeline after get() is CPU intensive (it deserializes binary avro documents). My question is:

1. Should we switch the scheduler using publishOn operator to avoid CPU intensive work on netty event loop thread pool? Even if, the work wasn’t CPU intensitve, wouldn’t it be a bad practice to allow business logic downstream to execute on a scheduler which was meant for couchbase client?

We have also noticed that one cb-io-kv thread was being significantly more utilized than other threads under heavy load (~60k gets per second). The data which we were fetching was evenly distributed across nodes. When we used publishOn after get(), all threads started being utilized evenly. So the second question would be:

2. What could be the reason of the uneven distribution of work between cb-io-kv threads? Have you heard about this problem before?

[mico.piira]

Hello, here is a very similar question that I asked earlier Java SDK V 3.0 threading/scheduler question - #2 by david.nault

I received a good response discussing this topic on that thread by david.nault:

@david.nault

By default, the downstream operators are executed in the Netty Event Loop thread for the socket connected to the Couchbase Server node. If you had more than one Couchbase Server node, and the documents you were fetching lived on different nodes, you might see different threads being used.

If your processing is expensive and you have more CPUs than Couchbase nodes, it might make sense to distribute the work among all CPUs. You can do this by applying the publishOn operator to the get result. This specifies the scheduler to use for executing all downstream operators.