Socket connect took longer than specified timeout

Java SDK
1

Hey, all
Trying to run a simple example.
I have a 4 nodes cluster on Amazin EC2 running CB Version: 4.0.0-4051.
Newest java SDK 2.2.5
My Firewall is disabled as the Cluster security group is open.
I can access to port 8091 from my local machine to the CB interface.
Yet when i run:
Cluster cluster = CouchbaseCluster.create(“54.201.13.77”);
System.out.println(“A”);
Bucket bucket = cluster.openBucket(“beer-sample”,90,TimeUnit.SECONDS);
System.out.println(“B”);

N1qlQueryResult queryResult =
	bucket.query(N1qlQuery.simple(PrepareStatement.prepare("CREATE PRIMARY INDEX `beer-sample-primary-index` ON `beer-sample` USING GSI")));
//N1qlQueryResult queryResult =	bucket.query(N1qlQuery.simple(PrepareStatement.prepare("select * from `beer-sample` limit 10")));
	
System.out.println(queryResult.errors());
while(queryResult.rows().hasNext())
    System.out.println(queryResult.rows().next());
cluster.disconnect();

i get:

Mar 07, 2016 4:27:42 PM com.couchbase.client.core.CouchbaseCore
INFO: CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile=‘null’, sslKeystorePassword=‘null’, queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=4, computationPoolSize=4, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-jvm-core/1.2.5 (git: 1.2.5), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, queryTimeout=75000, viewTimeout=75000, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}
A
Mar 07, 2016 4:27:44 PM com.couchbase.client.core.node.CouchbaseNode signalConnected
INFO: Connected to Node ec2-54-201-13-77.us-west-2.compute.amazonaws.com
Mar 07, 2016 4:27:49 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.43 is slow, took 4503ms
Mar 07, 2016 4:27:53 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.44 is slow, took 4503ms
Mar 07, 2016 4:27:58 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.45 is slow, took 4501ms
Mar 07, 2016 4:28:02 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.46 is slow, took 4501ms
Mar 07, 2016 4:28:02 PM com.couchbase.client.core.config.DefaultConfigurationProvider$8 call
INFO: Opened bucket beer-sample
Mar 07, 2016 4:28:02 PM com.couchbase.client.core.node.CouchbaseNode signalDisconnected
INFO: Disconnected from Node ec2-54-201-13-77.us-west-2.compute.amazonaws.com
B
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][KeyValueEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise notifyListener0
WARNING: An exception was thrown by com.couchbase.client.core.endpoint.AbstractEndpoint$2.operationComplete()
rx.exceptions.OnErrorNotImplementedException: connection timed out: /172.31.8.43:11210
at rx.Observable$26.onError(Observable.java:7954)
at rx.observers.SafeSubscriber._onError(SafeSubscriber.java:158)
at rx.observers.SafeSubscriber.onError(SafeSubscriber.java:119)
at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48)
at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71)
at rx.internal.operators.OperatorSingle$ParentSubscriber.onError(OperatorSingle.java:139)
at rx.internal.operators.OperatorTakeLastOne$ParentSubscriber.onError(OperatorTakeLastOne.java:164)
at rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:239)
at rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:774)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:532)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:521)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onError(OperatorMerge.java:808)
at rx.internal.operators.OperatorSingle$ParentSubscriber.onError(OperatorSingle.java:139)
at rx.internal.operators.OperatorTakeLastOne$ParentSubscriber.onError(OperatorTakeLastOne.java:164)
at rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:239)
at rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:774)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:532)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:521)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onError(OperatorMerge.java:808)
at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48)
at rx.internal.operators.OperatorSingle$ParentSubscriber.onError(OperatorSingle.java:139)
at rx.internal.operators.OperatorTakeLastOne$ParentSubscriber.onError(OperatorTakeLastOne.java:164)
at rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:239)
at rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:774)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:532)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:521)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onError(OperatorMerge.java:808)
at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48)
at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48)
at rx.internal.operators.OperatorSingle$ParentSubscriber.onError(OperatorSingle.java:139)
at rx.internal.operators.OperatorTakeLastOne$ParentSubscriber.onError(OperatorTakeLastOne.java:164)
at rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:239)
at rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:774)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:532)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:521)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onError(OperatorMerge.java:808)
at rx.subjects.SubjectSubscriptionManager$SubjectObserver.onError(SubjectSubscriptionManager.java:226)
at rx.subjects.AsyncSubject.onError(AsyncSubject.java:115)
at com.couchbase.client.core.endpoint.AbstractEndpoint$2.operationComplete(AbstractEndpoint.java:318)
at com.couchbase.client.core.endpoint.AbstractEndpoint$2.operationComplete(AbstractEndpoint.java:284)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:415)
at com.couchbase.client.deps.io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:87)
at com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler$1.operationComplete(KeyValueFeatureHandler.java:140)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:415)
at com.couchbase.client.deps.io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:87)
at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler$3.operationComplete(KeyValueAuthHandler.java:316)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:222)
at com.couchbase.client.deps.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at com.couchbase.client.deps.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.couchbase.client.deps.io.netty.channel.ConnectTimeoutException: connection timed out: /172.31.8.43:11210
at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:220)
… 7 more

Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][QueryEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][QueryEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][ViewEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][ViewEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][QueryEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][ViewEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][ViewEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][KeyValueEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][KeyValueEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][KeyValueEndpoint]: Socket connect took longer than specified timeout.
Mar 07, 2016 4:28:03 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
WARNING: [null][QueryEndpoint]: Socket connect took longer than specified timeout.
Exception in thread “main” java.lang.RuntimeException: java.util.concurrent.TimeoutException
at com.couchbase.client.java.util.Blocking.blockForSingle(Blocking.java:75)
at com.couchbase.client.java.CouchbaseBucket.query(CouchbaseBucket.java:656)
at com.couchbase.client.java.CouchbaseBucket.query(CouchbaseBucket.java:582)
at App.main(App.java:31)
Caused by: java.util.concurrent.TimeoutException
… 4 more

I tried to extend in the environment socketConnectTimeout, but i get the same results…
Can’t think why is this happening… any suggestions?

2

What kind of latency do you have between your client node and the cluster node you’re trying to connect to? and can you share the code you used to increase the socketConnectTimeout?

You should also try to increase the connectTimeout as well (one is pure network IO tuning, the other is more of an applicative timeout but it is taken into account as well during init).

Can you confirm that port 11210 is also reachable? Couchbase uses a certain number of ports (hence my previous recommendation to check the firewall configuration).

3

Thanks for the response.

What do you mean by what kind?

And as for the other questions:
The code to increase timeouts:
CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder() .queryTimeout(10000000).kvTimeout(10000000).searchTimeout(1000000).socketConnectTimeout(1000000000).managementTimeout(1000000000).connectTimeout(10000000).build();

I’ve confirmed that port 11210 is also reachable.

When i really increase the timeouts i get the following:
INFO: CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile=‘null’, sslKeystorePassword=‘null’, queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=4, computationPoolSize=4, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-jvm-core/1.2.5 (git: 1.2.5), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000000000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, queryTimeout=10000000, viewTimeout=75000, kvTimeout=10000000, connectTimeout=10000000, disconnectTimeout=25000, dnsSrvEnabled=false}
A
Mar 07, 2016 6:03:43 PM com.couchbase.client.core.node.CouchbaseNode signalConnected
INFO: Connected to Node ec2-54-201-13-77.us-west-2.compute.amazonaws.com
Mar 07, 2016 6:03:48 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.43 is slow, took 4525ms
Mar 07, 2016 6:03:52 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.44 is slow, took 4520ms
Mar 07, 2016 6:03:57 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.45 is slow, took 4523ms
Mar 07, 2016 6:04:01 PM com.couchbase.client.core.node.CouchbaseNode
WARNING: DNS Reverse Lookup of 172.31.8.46 is slow, took 4524ms
Mar 07, 2016 6:04:01 PM com.couchbase.client.core.config.DefaultConfigurationProvider$8 call
INFO: Opened bucket TwittterCampaignsStatistics
Mar 07, 2016 6:04:01 PM com.couchbase.client.core.node.CouchbaseNode signalDisconnected
INFO: Disconnected from Node ec2-54-201-13-77.us-west-2.compute.amazonaws.com
Exception in thread “main” com.couchbase.client.core.RequestCancelledException: Could not dispatch request, cancelling instead of retrying.
at com.couchbase.client.core.retry.RetryHelper.retryOrCancel(RetryHelper.java:51)
at com.couchbase.client.core.service.AbstractPoolingService.dispatch(AbstractPoolingService.java:60)
at com.couchbase.client.core.service.AbstractDynamicService.send(AbstractDynamicService.java:129)
at com.couchbase.client.core.node.CouchbaseNode.send(CouchbaseNode.java:197)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:210)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:76)
at com.couchbase.client.deps.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)

4

By what kind I meant what latency do you see eg. when you do a simple ping.

Interesting lines in the log: WARNING: DNS Reverse Lookup of 172.31.8.43 is slow, took 4525ms
I bet your client machine is a Windows one? Each IP in the cluster is reverse looked up so that we have the hostname for various reasons (one of which is meaningful logging), but we noticed that often on a windows host the operation’s performance is very poor.

Here it takes 4.5 seconds for 1 IP, which adds up… Can you maybe try to configure a hostfile so that kind of resolution becomes trivial for windows? (not sure how to do that in that environment)

That said the timeouts you set are huge and 6 nodes reverse lookup shouldn’t cause a timeout with that kind of duration…

5

Latency:
Ping statistics:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 205ms, Maximum = 216ms, Average = 209ms

My cluster machines are all Ubuntu while indeed i’m reaching from Windows.
Timeouts are huge only for testing and obviously wont stay.

6

Updating that to solve all problems i needed to name each node of the cluster under the public Amazon EC2 DNS name and neither the private/public ip or the private DNS name. Thanks for answering and paying attention to my problems: @simonbasle

7

@simonbasle, I run into the same problem wihile developing on Windows laptop (Eclipse). The problem started to appear when I converted a stand-alone community edition server into a cluster.

I specify nodes with their public AWS DNS names. I also increased openBucket() timeout to 60 seconds. Reverse DNS lookup of each node in the cluster take 4.5 seconds. Eventually I get:

rx.exceptions.OnErrorNotImplementedException: connection timed out: /PRIVATE_IP:11210

Any idea how to deal with that?

8

@alex1 if you’re on a recent release of the SDK, you might want to increase the socketConnectTimeout as well, in addition to the connectTimeout. The former is specifically for establishing the socket connection (pure I/O level) while the later includes client side and server side operations like authentication, establishing local resources, thread pools, etc…

Edit: the socketConnectTimeout can be tuned only via the CouchbaseEnvironment that you can pass to CouchbaseCluster.create(...) (not tunable in the openBucket method call).

9

Thank you!

I am going through the client JavaDoc - it looks like creating a Custom CouchbaseEnvironment is not an easy thing. Can you please share some sample code that creates a Custom CouchbaseEnvironment with a non-default connectionTimeOut?

10

Sure thing! This happens at the point where you initialize a connection to the Couchbase cluster:

CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder()
    //this set the IO socket timeout globally, to 45s
    .socketConnectTimeout((int) TimeUnit.SECONDS.toMillis(45))
    //this sets the connection timeout for openBucket calls globally (unless a particular call provides its own timeout)
    .connectTimeout(TimeUnit.SECONDS.toMillis(60))
    .build();

//once you've prepared an environment configuration, use it when connecting to the cluster
Cluster cluster = CouchbaseCluster.create(env, "127.0.0.1"); //adapt your IP/hostname

//then you open the bucket. this operation will use the 60s timeout you configured above
//and the 45s network IO timeout internally
Bucket bucket = cluster.openBucket("myBucket");

Note that these values do not necessarily make sense, to be tuned according to your environment.
Most notably, if you’ve noticed that the DNS name resolution takes 4.5 seconds per node, then your connectTimeout should at least be greater than numberOfNodes x 4.5 x 1000 milliseconds…

11

Thank you @simonbasle, it solved the connection problem. Now I have a new one - I am using Vert.x and connecting to CouchBase cluster from a Worker Verticle. I get:

io.vertx.core.impl.DeploymentManager
SEVERE: java.util.concurrent.TimeoutException
java.lang.RuntimeException: java.util.concurrent.TimeoutException

Any idea why CouchBase Java client is soooo slow on Windows with regards to reverse DNS lookup?

Is there anyone in CouchBase team who might have experience with Vert.x?

I didn’t have this problem with a stand-alone CB server - it appeared only after introduction of a 2-nodes CB cluster…

12

It’s a Windows problem, not a Couchbase problem, unfortunately… IIRC things can be improved by adding entries in the hosts file, but that defeats the purpose of using hostnames for a cloud setup :confounded:

We saw it before in the wild, and we even added a dedicated log to help diagnose this issue (see this commit). Always on Windows :’(

Maybe we’ll be able to avoid reverse DNS lookups in a future release, but it’s not the case right now.

I have a very partial exposure to Vertx, and I’m not aware of someone else at Couchbase having done any work with it.

13

FYI: I ran into the same issue for me the Couchbase Console was reachable, while the sample Java program was not working. And I realized that it was the proxy server which was not allowing the connections.

Mine was a t2.medium instance with 8091, 8092, 8093, 8094 & 11210 ports open.

I tried another network connection and worked so I was able to round it to the proxy server blocking.

-Thanks

14

@simonbasle Thank you for all your suggestions. I followed all your solutions but I am facing below timeout issues when trying to connect to couch base from Jenkins. It works fine when connecting to couch base in my local machine. Can you please suggest?
com.couchbase.client.core.CouchbaseCore - CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile=‘null’, sslTruststoreFile=‘null’, sslKeystorePassword=false, sslTruststorePassword=false, sslKeystore=null, sslTruststore=null, sslHostnameVerificationEnabled=false, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=8091, bootstrapCarrierSslPort=11207, ioPoolSize=4, computationPoolSize=4, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=12, queryServiceEndpoints=12, searchServiceEndpoints=12, configPollInterval=2500, configPollFloorInterval=50, networkResolution=NetworkResolution{name=‘auto’}, ioPool=NioEventLoopGroup, kvIoPool=null, viewIoPool=null, searchIoPool=null, queryIoPool=null, analyticsIoPool=null, coreScheduler=CoreScheduler, memcachedHashingStrategy=DefaultMemcachedHashingStrategy, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.7.11 (git: 2.7.11, core: 1.7.11), retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, continuousKeepAliveEnabled=true, keepAliveErrorThreshold=4, keepAliveTimeout=2500, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=45000, callbacksOnIoPool=false, disconnectTimeout=25000, requestBufferWaitStrategy=com.couchbase.client.core.env.DefaultCoreEnvironment$4@6e3dd5ce, certAuthEnabled=false, coreSendHook=null, forceSaslPlain=false, compressionMinRatio=0.83, compressionMinSize=32, compressionEnabled=true, operationTracingEnabled=true, operationTracingServerDurationEnabled=true, tracer=ThresholdLogTracer, orphanResponseReportingEnabled=true, orphanResponseReporter=DefaultOrphanResponseReporter, keyValueServiceConfig=KeyValueServiceConfig{minEndpoints=1, maxEndpoints=1, pipelined=true, idleTime=0}, queryServiceConfig=QueryServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, searchServiceConfig=SearchServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, viewServiceConfig=ViewServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, analyticsServiceConfig=AnalyticsServiceConfig{minEndpoints=0, maxEndpoints=12, pipelined=false, idleTime=300}, queryTimeout=75000, viewTimeout=75000, searchTimeout=75000, analyticsTimeout=75000, kvTimeout=2500, connectTimeout=60000, dnsSrvEnabled=false, propagateParentSpan=true}

[RxComputationScheduler-1] WARN com.couchbase.client.core.endpoint.Endpoint - [XX-01.us.XX…com:8091][KeyValueEndpoint]: Socket connect took longer than specified timeout: Connect callback did not return, hit safeguarding timeout.

nested exception is java.lang.RuntimeException: java.util.concurrent.TimeoutException