Unable to connect to Couchbase cluster - Couchbase kafka connector

shivraj · November 21, 2019, 12:14pm

I am using couchbase-kafka-connector for sending mutation/deletion events to kafka topic. Everything working fine in my local machine. But I am getting below exception when trying to run the connector in DEV env the connector is able to connect to kafka but not to couchbase cluster because of below exception.

org.apache.kafka.connect.errors.ConnectException: com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket
at com.couchbase.connect.kafka.CouchbaseSourceTask.poll(CouchbaseSourceTask.java:194)
at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:259)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:226)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket
at com.couchbase.client.dcp.Client$6.call(Client.java:346)
at com.couchbase.client.dcp.Client$6.call(Client.java:343)
at rx.Completable$26$1.onError(Completable.java:1742)
at rx.internal.operators.CompletableOnSubscribeConcatArray$ConcatInnerSubscriber.onError(CompletableOnSubscribeConcatArray.java:63)
at rx.Completable$19$1.onError(Completable.java:1393)
at rx.internal.operators.CompletableOnSubscribeTimeout$2.onError(CompletableOnSubscribeTimeout.java:99)
at rx.Completable$26$1$1.onError(Completable.java:1765)
at com.couchbase.client.dcp.conductor.HttpStreamingConfigProvider$3$1.operationComplete(HttpStreamingConfigProvider.java:193)
at com.couchbase.client.dcp.conductor.HttpStreamingConfigProvider$3$1.operationComplete(HttpStreamingConfigProvider.java:176)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:512)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:505)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:484)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:425)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:113)
at com.couchbase.client.deps.io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:89)
at com.couchbase.client.dcp.transport.netty.StartStreamHandler.channelRead0(StartStreamHandler.java:84)
at com.couchbase.client.dcp.transport.netty.StartStreamHandler.channelRead0(StartStreamHandler.java:39)
at com.couchbase.client.deps.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921)
at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:135)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
… 1 more
Caused by: com.couchbase.client.core.CouchbaseException: Unknown error code during connect: 400 Bad Request
at com.couchbase.client.dcp.transport.netty.StartStreamHandler.channelRead0(StartStreamHandler.java:81)
… 24 more

Note : Firewall has been opened for port 8092,8093 and 11210 ports .

david.nault · November 21, 2019, 7:41pm

Hi Shivraj,

The connector also talks to Couchbase on port 8091 to fetch info about the cluster configuration. Add that to the list and you should be fine.

Thanks,
David

shivraj · November 22, 2019, 2:55am

Hi. Sorry my bad… forgot to mention 8091 port… even with 8091 port open I am facing above issue.

david.nault · November 22, 2019, 4:42am

I have a hunch the firewall is interfering with the long poll requests. Let’s see if we can verify that.

First let’s try a basic test. If you take this curl command and replace “BUCKET_NAME” with the name of the your bucket, and replace “Administrator:password” with your Couchbase credentials, what kind of response do you get?

curl -v -u Administrator:password http://localhost:8091/pools/default/b/BUCKET_NAME

If everything is working, you should see a JSON object representing the cluster config, and then the server should close the connection.

If that worked, let’s try the same thing against the streaming endpoint the connector actually uses! Use the same curl command as before, but replace /b/ with /bs/.

This time you should get the same JSON object with the cluster config followed by a few blank lines. The server should leave the connection open (it will push new JSON objects if/when the cluster changes). Kill it with control-c.

I’d be curious to hear what happens when you run each of those curl commands.

Thanks,
David

shivraj · November 25, 2019, 11:59am

HI David,

Sorry for the delay,

Please find the below output for curl coommand. Both /b and /bs are giving the response.

curl -v -u user:pwd http://X.X.X.X:8091/pools/default/bs/bucket

Hostname was NOT found in DNS cache * Trying X.X.X.X… * Connected to X.X.X.X (X.X.X.X) port 8091 (#0) * Server auth using Basic with user ‘user’ > GET /pools/default/bs/bucket HTTP/1.1 > Authorization: Basic cHJpY2VPbW5pOm9tbmlAMTIzJA== > User-Agent: curl/7.38.0 > Host: X.X.X.X:8091 > Accept: / > < HTTP/1.1 200 OK < x-xss-protection: 1; mode=block < x-permitted-cross-domain-policies: none < x-frame-options: DENY < x-content-type-options: nosniff * Server envoy is not blacklisted < server: envoy < pragma: no-cache < expires: Thu, 01 Jan 1970 00:00:00 GMT < date: Mon, 25 Nov 2019 09:58:54 GMT < content-type: application/json; charset=utf-8 < cache-control: no-cache,no-store,must-revalidate < x-envoy-upstream-service-time: 3 < transfer-encoding: chunked < {“rev”:2196,“name”:“bucket”,“uri”:“/pools/default/buckets/bucket?bucket_uuid=9a3cce493f15c2296d7c57943cXXXX”,“streamingUri”:“/pools/default/bucketsStreaming/bucket?bucket_uuid=9a3cce493f15c2296d7c579XXXXXX”,“nodes”:[{“couchApiBase”:“X.X.X.X curl -v -u user:pwd$ http://X.X.X.X:8091/pools/default/bs/bucketi * Hostname was NOT found in DNS cache * Trying X.X.X.X… * Connected to X.X.X.X (X.X.X.X) port 8091 (#0) * Server auth using Basic with user ‘user’ > GET /pools/default/bs/bucket HTTP/1.1 > Authorization: Basic cHJpY2VPbW5pOm9tbmlXXXA== > User-Agent: curl/7.38.0 > Host: X.X.X.X:8091 > Accept: / > < HTTP/1.1 200 OK < x-xss-protection: 1; mode=block
< x-permitted-cross-domain-policies: none < x-frame-options: DENY < x-content-type-options: nosniff * Server envoy is not blacklisted < server: envoy < pragma: no-cache < expires: Thu, 01 Jan 1970 00:00:00 GMT < date: Mon, 25 Nov 2019 10:01:10 GMT < content-type: application/json; charset=utf-8 < cache-control: no-cache,no-store,must-revalidate < x-envoy-upstream-service-time: 3 < transfer-encoding: chunked < {“rev”:2196,“name”:“bucket”,“uri”:”/pools/default/buckets/bucket?bucket_uuid=9a3cce493f15c2296d7c57943cXXXXXX",“streamingUri”:“/pools/default/bucketsStreaming/bucket?bucket_uuid=9a3cce493f15c2296XXXXXXXX”,“nodes”:[{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7cXXXXX",“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}},{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7c57943c4e2183”,“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}},{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7c57943c4e2183”,“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}},{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7c57943c4e2183”,“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}},{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7c57943c4e2183”,“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}},{“couchApiBase”:“http://X.X.X.X:8092/bucket%2B9a3cce493f15c2296d7c57943c4e2183”,“hostname”:“X.X.X.X:8091”,“ports”:{“proxy”:11211,“direct”:11210}}],“nodesExt”:[{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“hostname”:“X.X.X.X”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“hostname”:“X.X.X.X”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“thisNode”:true,“hostname”:“X.X.X.X”},{“services”:{"mgm
t”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“hostname”:“X.X.X.X”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“hostname”:“X.X.X.X”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“capiSSL”:18092,“capi”:8092,“kvSSL”:11207,“projector”:9999,“kv”:11210,“moxi”:11211},“hostname”:“X.X.X.X”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“indexAdmin”:9100,“indexScan”:9101,“indexHttp”:9102,“indexStreamInit”:9103,“indexStreamCatchup”:9104,“indexStreamMaint”:9105,“indexHttps”:19102,“n1ql”:8093,“n1qlSSL”:18093},“hostname”:“X.X.X.X2”},{“services”:{“mgmt”:8091,“mgmtSSL”:18091,“indexAdmin”:9100,“indexScan”:9101,“indexHttp”:9102,“indexStreamInit”:9103,“indexStreamCatchup”:9104,“indexStreamMaint”:9105,“indexHttps”:19102,“n1ql”:8093,“n1qlSSL”:18093},“hostname”:“X.X.X.X3”}],“nodeLocator”:“vbucket”,“uuid”:“9a3cce493f15c2296d7c57943c4XXXX”,“ddocs”:{“uri”:“/pools/default/buckets/bucket/ddocs”},“vBucketServerMap”:{“hashAlgorithm”:“CRC”,“numReplicas”:1,“serverList”:[“X.X.X.X:11210”,“X.X.X.X:11210”,“X.X.X.X:11210”,“X.X.X.X:11210”,“X.X.X.X:11210”,“X.X.X.X:11210”],“vBucketMap”:[[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[0,4],[5,2],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[0,5],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[2,4],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[1,4],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,1],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[1,5],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[5,2],[2,4],[2,4],[2,4],[2,4],[2,4],[2,4],[2,4],[2,4],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[0,1],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[3,4],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,2],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0],[4,0

david.nault · November 26, 2019, 3:07am

Hi Shivraj,

Just to confirm, you ran those curl commands from the same machine where you’re running the Kafka connector?

Thanks,
David

shivraj · November 26, 2019, 3:32am

Yes David. I ran from the machine where couchbase-kafka-connector is running.

david.nault · November 26, 2019, 8:00am

Hi Shivraj,

Thanks for confirming. Next thing I can recommend is to run SDK Doctor against your dev cluster (from the same machine that runs the Kafka connector) and see it it identifies any irregularities.

Thanks,
David

shivraj · November 26, 2019, 11:47am

Thanks David for the suggestion. Could you please help me how to install and run the sdk-doctor application. Currently am deploying and running the kafka connector application on GCP(Google Cloud) pods. Seems like the application is written in go language.

david.nault · November 26, 2019, 6:55pm

A pre-built Linux binary is available on the SDK doctor releases page. Just download and execute.

shivraj · November 30, 2019, 5:32am

Hi David,

I ran the sdk-doctor from couchbase-kafka-connector machine, but seem like no issues connecting to Couchbase. I ran for both single IP and cluster url but getting same response. Please find the response below,

Note : there is a failure while fetching cluster info. Is that causing an issue. If so could you please help me how to fix it.

09:46:09.681 INFO Parsing connection string couchbase://X.X.X.1/bucket
09:46:09.734 INFO Connection string identifies the following CCCP endpoints:
09:46:09.734 INFO 1. X.X.X.1:11210
09:46:09.734 INFO Connection string identifies the following HTTP endpoints:
09:46:09.734 INFO 1. X.X.X.1:8091
09:46:09.734 INFO Connection string specifies bucket bucket
09:46:09.734 WARN Your connection string specifies only a single host. You should consider adding additional static nodes from your cluster to this list to improve your applications fault-tolerance
09:46:09.734 INFO Performing DNS lookup for host X.X.X.1
09:46:09.734 INFO Attempting to connect to cluster via CCCP
09:46:09.734 INFO Attempting to fetch config via cccp from X.X.X.1:11210 09:46:09.739 INFO ▶ Selected the following network type: default 09:46:09.739 INFO ▶ Identified the following nodes: 09:46:09.739 INFO ▶ [0] X.X.X.2 09:46:09.739 INFO ▶ moxi: 11211, mgmt: 8091, mgmtSSL: 18091 09:46:09.739 INFO ▶ capiSSL: 18092, capi: 8092, kvSSL: 11207 09:46:09.739 INFO ▶ projector: 9999, kv: 11210 09:46:09.739 INFO ▶ [1] X.X.X.1 09:46:09.739 INFO ▶ mgmtSSL: 18091, capiSSL: 18092, capi: 8092 09:46:09.739 INFO ▶ kvSSL: 11207, projector: 9999, kv: 11210 09:46:09.739 INFO ▶ moxi: 11211, mgmt: 8091 09:46:09.739 INFO ▶ [2] X.X.X.3 09:46:09.739 INFO ▶ capiSSL: 18092, capi: 8092, kvSSL: 11207 09:46:09.739 INFO ▶ projector: 9999, kv: 11210, moxi: 11211 09:46:09.739 INFO ▶ mgmt: 8091, mgmtSSL: 18091 09:46:09.739 INFO ▶ [3] X.X.X.4 09:46:09.739 INFO ▶ kvSSL: 11207, projector: 9999, kv: 11210 09:46:09.739 INFO ▶ moxi: 11211, mgmt: 8091, mgmtSSL: 18091 09:46:09.739 INFO ▶ capiSSL: 18092, capi: 8092 09:46:09.739 INFO ▶ [4] X.X.X.5 09:46:09.739 INFO ▶ mgmt: 8091, mgmtSSL: 18091, capiSSL: 18092 09:46:09.739 INFO ▶ capi: 8092, kvSSL: 11207, projector: 9999 09:46:09.739 INFO ▶ kv: 11210, moxi: 11211 09:46:09.739 INFO ▶ [5] X.X.X.7 09:46:09.739 INFO ▶ mgmt: 8091, mgmtSSL: 18091, capiSSL: 18092 09:46:09.739 INFO ▶ capi: 8092, kvSSL: 11207, projector: 9999 09:46:09.739 INFO ▶ kv: 11210, moxi: 11211 09:46:09.739 INFO ▶ [6] X.X.X.5 09:46:09.739 INFO ▶ indexAdmin: 9100, indexStreamCatchup: 9104, indexStreamMaint: 9105 09:46:09.739 INFO ▶ indexHttps: 19102, n1qlSSL: 18093, mgmt: 8091 09:46:09.739 INFO ▶ [6] X.X.X.5 09:46:09.739 INFO ▶ indexAdmin: 9100, indexStreamCatchup: 9104, indexStreamMaint: 9105 09:46:09.739 INFO ▶ indexHttps: 19102, n1qlSSL: 18093, mgmt: 8091 09:46:09.739 INFO ▶ mgmtSSL: 18091, indexScan: 9101, indexHttp: 9102 09:46:09.739 INFO ▶ indexStreamInit: 9103, n1ql: 8093 09:46:09.739 INFO ▶ [7] X.X.X.6 09:46:09.739 INFO ▶ mgmt: 8091, mgmtSSL: 18091, indexAdmin: 9100 09:46:09.739 INFO ▶ indexScan: 9101, indexStreamInit: 9103, indexHttps: 19102 09:46:09.739 INFO ▶ n1qlSSL: 18093, indexHttp: 9102, indexStreamCatchup: 9104 09:46:09.739 INFO ▶ indexStreamMaint: 9105, n1ql: 8093 *09:46:09.739 INFO ▶ Fetching config from X.X.X.2:8091* 09:46:09.745 INFO ▶ *Failed to retreive cluster information (status code: 401)* 09:46:09.748 INFO ▶ Successfully connected to KV service at X.X.X.2:1121009:46:09.751 INFO ▶ Successfully connected to MGMT service atX.X.X.2:809109:46:09.753 INFO ▶ Successfully connected to CAPI service atX.X.X.2:809209:46:09.755 INFO ▶ Successfully connected to KV service atX.X.X.1:1121009:46:09.760 INFO ▶ Successfully connected to MGMT service atX.X.X.1:809109:46:09.762 INFO ▶ Successfully connected to CAPI service atX.X.X.1:809209:46:09.765 INFO ▶ Successfully connected to KV service atX.X.X.3:1121009:46:09.769 INFO ▶ Successfully connected to MGMT service atX.X.X.3:809109:46:09.770 INFO ▶ Successfully connected to CAPI service atX.X.X.3:809209:46:09.774 INFO ▶ Successfully connected to KV service atX.X.X.4:1121009:46:09.778 INFO ▶ Successfully connected to MGMT service atX.X.X.4:809109:46:09.780 INFO ▶ Successfully connected to CAPI service atX.X.X.4:809209:46:09.783 INFO ▶ Successfully connected to KV service atX.X.X.8:1121009:46:09.786 INFO ▶ Successfully connected to MGMT service atX.X.X.8:809109:46:09.788 INFO ▶ Successfully connected to CAPI service atX.X.X.8:809209:46:09.790 INFO ▶ Successfully connected to KV service atX.X.X.7:1121009:46:09.794 INFO ▶ Successfully connected to MGMT service atX.X.X.7:809109:46:09.795 INFO ▶ Successfully connected to CAPI service atX.X.X.7:809209:46:09.818 INFO ▶ Successfully connected to MGMT service atX.X.X.5:809109:46:09.819 INFO ▶ Successfully connected to N1QL service atX.X.X.5:809309:46:09.824 INFO ▶ Successfully connected to MGMT service atX.X.X.6:809109:46:09.825 INFO ▶ Successfully connected to N1QL service atX.X.X.6:809309:46:09.831 INFO ▶ Memd Nop PingedX.X.X.2:1121010 times, 0 errors, 0ms min, 0ms max, 0ms mean 09:46:09.837 INFO ▶ Memd Nop PingedX.X.X.1:1121010 times, 0 errors, 0ms min, 0ms max, 0ms mean 09:46:09.843 INFO ▶ Memd Nop PingedX.X.X.3:1121010 times, 0 errors, 0ms min, 0ms max, 0ms mean 09:46:09.850 INFO ▶ Memd Nop PingedX.X.X.4:1121010 times, 0 errors, 0ms min, 0ms max, 0ms mean 09:46:09.853 INFO ▶ Memd Nop PingedX.X.X.8:1121010 times, 0 errors, 0ms min, 0ms max, 0ms mean 09:46:09.856 INFO ▶ Memd Nop PingedX.X.X.7:11210` 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.856 INFO Diagnostics completed

09:46:09.831 INFO Memd Nop Pinged X.X.X.2:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.837 INFO Memd Nop Pinged X.X.X.1:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.843 INFO Memd Nop Pinged X.X.X.3:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.850 INFO Memd Nop Pinged X.X.X.4:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.853 INFO Memd Nop Pinged X.X.X.8:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.856 INFO Memd Nop Pinged X.X.X.7:11210 10 times, 0 errors, 0ms min, 0ms max, 0ms mean
09:46:09.856 INFO Diagnostics completed

Summary:
[WARN] Your connection string specifies only a single host. You should consider adding additional static nodes from your cluster to this list to improve your applications fault-tolerance

Found multiple issues, see listing above.

david.nault · December 2, 2019, 11:24pm

HTTP status code 401 (“Unauthorized”) indicates there might be a problem with the user’s credentials. Maybe the user lacks the necessary role to access the cluster config info? Since this is your DEV environment, try connecting as a user with full admin privileges. If that works then we know this is a permissions issue.

Thanks,
David

shivraj · December 3, 2019, 9:39am

Hi David,

I tried sdk-doctor with admin credentials, but still getting the same output as above.

Thanks,
Shivraj

david.nault · December 3, 2019, 9:23pm

Hi Shivraj,

What if you run it from inside the firewall?

Thanks,
David

shivraj · December 4, 2019, 1:08pm

Sorry, I did not get you.

david.nault · December 4, 2019, 9:24pm

Can you run it on the same machine where Couchbase Server is running?

shivraj · December 5, 2019, 1:03pm

Hi David,

Thanks a lot for the help!

I ran the same connector in Stage environment, there the functionality is working fine as expected. So the issue is with DEV environment.

Thanks,
Shivraj