Bucket get operations causing inconsistency and timeout exception

Hi,

We are using couchabse java client 2.6.2 and we are getting stange behaviour:
We are executing bucket.get opeartion for around 100 document in a loop:
our code look like this:

RetryWhenFunction retryWhenRuntimeException = RetryBuilder.anyOf(Exception.class)
			.delay(Delay.exponential(TimeUnit.SECONDS, 64)).max(10)
			.doOnRetry((t1, t2, t3, t4) -> _log.warn("Retrying due to a runtimeexception. retry number :{}:", t1, t2))
			.build();
JsonDocument doc = _cbBucket.async().get(key).retryWhen(retryWhenRuntimeException).toBlocking().singleOrDefault(null);

Our thread get blocked before completing the loop (in the ~20 document) and we are getting continiously those errors:

[RxComputationScheduler-2] WARN com.couchbase.client.core.endpoint.AbstractGenericHandler - [/10.244.66.130:11210][KeyValueEndpoint]: Got error while consuming KeepAliveResponse.
java.util.concurrent.TimeoutException
	at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber.onTimeout(OnSubscribeTimeoutTimedWithFallback.java:166)
	at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber$TimeoutTask.call(OnSubscribeTimeoutTimedWithFallback.java:191)
	at rx.internal.schedulers.EventLoopsScheduler$EventLoopWorker$2.call(EventLoopsScheduler.java:189)
	at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

The keyvalue timeout is set to 60 seconds.

Our retry policy is not logging any thing (not called!!).

When switching to synchronous code (bucket.get),I’m able to get kv timeout exception and with a manuel retry policy code , I’m getting better result :

Caused by: java.util.concurrent.TimeoutException: {"b":"dev_module_1","s":"kv","t":60000000,"i":"0x10"}
	at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:131)
	at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:127)
	at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
	at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber.onTimeout(OnSubscribeTimeoutTimedWithFallback.java:166)
	at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber$TimeoutTask.call(OnSubscribeTimeoutTimedWithFallback.java:191)
	at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

What is wrong with the asynchronous version and why this performance issue with a small test?

Thanks

Any one from couchbase who can help on this?