Couchbase client library, AWS and timeouts

connections

#1

We are using couchbase which is behind the ELB running in AWS env. couchbase client 2.1.4 running with application in tomcat. Application using spring but its not using spring couchbase integration. application creates its own cluster and bucket bean and adds it in to spring context.

After running for certain period of time we suddenly started seeing the following exception and eventually we have to restart the application.
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at com.couchbase.client.java.util.Blocking.blockForSingle(Blocking.java:93) ~[java-client-2.1.4.jar:2.1.4]
at com.couchbase.client.java.CouchbaseBucket.get(CouchbaseBucket.java:120) ~[java-client-2.1.4.jar:2.1.4]
at com.couchbase.client.java.CouchbaseBucket.get(CouchbaseBucket.java:115) ~[java-client-2.1.4.jar:2.1.4]

  • I would like to understand how the client library maintains the connections with couchbase server?
  • Does it maintain the connection pool and check the health every often? How frequently and how many times it does that? Where can I find more documentation about connection management?
  • If I use the hostname instead of IP to connect to couchbase then how does existing connections behaves when IP changes? Does the connection by any chance do IP lookup ?
  • Currently we give ELB name for connecting to CB since we cant connect to CB directly, primarily because of infrastructure issues. We will be changing it but meantime need to support this config. How do we recover if the ELB IP changes?

#2

Hi @mmoghe

In the Couchbase client, Netty is in charge of network IO. The client receives updates from the cluster in terms of node removals and additions, by talking to it through one of the IP/hostnames you provided for bootstrap (CouchbaseCluster.create("IP1", "IP2", "hostname3")). If there is a disconnect there, the client tries to re-establish the connection, with an exponential back-off delay.

There is a keep-alive that is performed on every TCP connection (1 per service per node), every 30 seconds by default. You can configure the delay at which it occurs (it will trigger after nothing was sent/received during that duration):

CouchbaseEnvironment = DefaultCouchbaseEnvironment.builder()
                .keepAliveInterval(TimeUnit.MINUTES.toMillis(3)) //3min, in milliseconds
                .build();

I’m not entirely sure about the scenario where the IP behind a hostname changes, @daschl any insight on that?

PS: You should upgrade to at least the latest bugfix release in the 2.1 branch (2.1.6) and preferably 2.2.7, several bugfixes that could impact connectivity have been fixed.