Unable to connect cbq-engine in VM


#1

Environment:

  • Application client in Win7,
  • Java7,
  • couchbase java-client 2.1.1
  • VM (centOS) with couchbase server 3.0.2 (IP: 192.168.56.200), and cbq-engine running
 public static void main(String[] args){
    Cluster cluster=CouchbaseCluster.create(DefaultCouchbaseEnvironment
    .builder()
    .queryEnabled(true)
    .build(),"192.168.56.200");

    System.out.println("Before");
    Bucket bucket=cluster.openBucket("sampleBucket",100,TimeUnit.SECONDS);
    System.out.println("After");

    QueryResult queryResult=bucket.query(Query.simple("SELECT * FROM sampleBucket LIMIT 10"));
    System.out.println("I'm here");
}
    Mar 13, 2015 1:19:36 PM com.couchbase.client.core.CouchbaseCore <init>
    INFO: CouchbaseEnvironment: {, queryTimeout=5000, viewTimeout=5000, kvTimeout=5000, connectTimeout=2147483647, disconnectTimeout=5000, dnsSrvEnabled=false}
    Before
    Mar 13, 2015 1:19:42 PM com.couchbase.client.core.node.CouchbaseNode$1 call
    INFO: Connected to Node 192.168.56.200
    Mar 13, 2015 1:19:42 PM com.couchbase.client.core.config.DefaultConfigurationProvider$6 call
    INFO: Opened bucket sampleBucket
    Mar 13, 2015 1:19:42 PM com.couchbase.client.core.node.CouchbaseNode$1 call
    INFO: Disconnected from Node 192.168.56.200
    After
    Mar 13, 2015 1:19:43 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
    WARNING: [null][KeyValueEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: 
    java.net.ConnectException: Connection refused: no further information: localhost/127.0.0.1:11210
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
        at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:744)
    
    Mar 13, 2015 1:19:43 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
    WARNING: [null][ViewEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: 
    java.net.ConnectException: Connection refused: no further information: localhost/127.0.0.1:8092
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
        at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:744)
    
    Mar 13, 2015 1:19:43 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
    WARNING: [null][QueryEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: 
    java.net.ConnectException: Connection refused: no further information: localhost/127.0.0.1:8093
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
        at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:744)

Why does it disconnect from the remote couchbase after open bucket, and then trying to connect to localhost? Please help me, not sure which part was wrong


#2

I tried to downgrade to java client 2.0.3 (using bucket.query() ) has no disconnection issue, but have the result parsing exception.


#3

Can you please share DEBUG or FINEST logs to see what config we get back from the server (ideally from 2.1.1 and 2.0.3 to compare)?

So you are saying connecting works in 2.0.3 but not in 2.1.1 - can you also try 2.1.0?

Also please double check that the host of the server is not configured to 127.0.0.1 (so even while it responds to the request, the config contains 127.0.0.1 as the node addr).


#4

Sure daschl., please find the logs below
https://gist.github.com/anonymous/1a179c6393b168d9b789

im not quite sure how to check this, could you guide me (im using Linux 2.6.32) please?
Also please double check that the host of the server is not configured to 127.0.0.1 (so even while it responds to the request, the config contains 127.0.0.1 as the node addr).


#5

thanks for reporting that, indeed it looks a little fishy. I’ll dig into it as soon as possible.


#6

@chinkang would it be possible that you enable TRACE logging for the com.couchbase namespace so we see the raw config arriving from the server?


#7

@daschl Here it is :slight_smile: https://gist.github.com/anonymous/08da1f6d93edecd880e6
Two trace logs in the gist


#8

Hi,

we’ve published a developer preview of 2.1.2 to get quicker feedback if the bug is properly fixed and your issues are addressed. You can get it from our couchbase repository like all pre-releases, the 2.1.2 GA release will be published to maven central as usual.

You can get it like this:

<dependencies>
    <dependency>
        <groupId>com.couchbase.client</groupId>
        <artifactId>java-client</artifactId>
        <version>2.1.2-dp</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>couchbase</id>
        <name>couchbase repo</name>
        <url>http://files.couchbase.com/maven2</url>
        <snapshots><enabled>false</enabled></snapshots>
    </repository>
</repositories>

It includes this change which should resolve the issue: https://github.com/couchbase/couchbase-jvm-core/commit/f6d5dd3f62f01a0d2ce48d5f925eefadf00c77d6

Keep in mind that this is not an officially tested release since it has not gone trough extensive QE testing yet. Please let us know if it works and upgrade to 2.1.2 as soon as it is released.

Thanks!


#9

@daschl I just tried with your proposed solution, and it works! Thank you very much!


#10

great to hear, please go to 2.1.2 as soon as it’s out.


#11

I am getting same connection error inspite of using the 2.1.3 Java SDK version


#12

@badrictl if you are using the 4.0 releases, don’t use queryEnabled at all, it will pick it up automatically. So use 2.1.3 with 4.0-beta. If the error still exists, can you please share TRACE level logs of the full run?


#13

Hi,
@daschl I’m using using couchbase Java SDK version 2.1.5 and couchbase server version 3.0.2 on a single node cluster and still getting the same error.

As mentinoed above, I have not touched the "queryEnabled’ feature and not using any N1QL queries. Here is my console output:-

    Oct 18, 2015 4:40:19 PM com.couchbase.client.core.CouchbaseCore 
    INFO: CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=8, computationPoolSize=8, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.1.5 (git: 2.1.5), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, socketConnectTimeout=3600000, queryTimeout=75000, viewTimeout=75000, kvTimeout=2500, connectTimeout=3600000, disconnectTimeout=25000, dnsSrvEnabled=false}
    ********** setting bucket
    Oct 18, 2015 4:40:20 PM com.couchbase.client.core.node.CouchbaseNode$1 call
    INFO: Connected to Node 192.168.25.144
    Oct 18, 2015 4:40:22 PM com.couchbase.client.core.config.DefaultConfigurationProvider$6 call
    INFO: Opened bucket demobucket
    Oct 18, 2015 4:40:22 PM com.couchbase.client.core.node.CouchbaseNode$1 call
    INFO: Disconnected from Node 192.168.25.144
    ********** end of set bucket
    Oct 18, 2015 4:40:23 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
    WARNING: [null][ViewEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: 
    java.net.ConnectException: Connection refused: no further information: /127.127.0.1:8092
    	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    	at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
    	at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
    	at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    	at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
    	at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    	at java.lang.Thread.run(Unknown Source)

    Oct 18, 2015 4:40:23 PM com.couchbase.client.core.endpoint.AbstractEndpoint$2 operationComplete
    WARNING: [null][KeyValueEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: 
    java.net.ConnectException: Connection refused: no further information: /127.127.0.1:11210
    	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    	at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
    	at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
    	at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    	at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    	at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
    	at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    	at java.lang.Thread.run(Unknown Source)

Also, why does it show Connection refused: no further information: /127.127.0.1:8092 ??
Shouldn’t it try to connect to 192.168.25.144:8092 ??
My node is configured to run on 192.168.25.144 and I can get the docs and perform queries through the browser and REST calls on this IP.

Trying to connect with Java SDK 2.0.3 also gives the same error.

Here is my code

private void setEnvironment(){
		CouchBaseConnector.environment =  DefaultCouchbaseEnvironment.builder()
        .connectTimeout(3600000) 
        .socketConnectTimeout(3600000)
        .build();
		setCluster();
	}
	
	private void setCluster(){
		CouchBaseConnector.cluster = CouchbaseCluster.create(CouchBaseConnector.environment,CouchBaseConnector.ip);
		setBucket();
	}
	private void setBucket(){
		System.out.println("********** setting bucket");
		CouchBaseConnector.bucket = CouchBaseConnector.cluster.openBucket(bucketName,15,TimeUnit.MINUTES);
		System.out.println("********** end of set bucket");
	}

setBucket() is where the issue is arising.


#14

I see what’s going on in your system, check out the first logs you posted.

You are connecting to 192.168.25.144, but the server config returns 127.127.0.1, which is then not accessible by your application.

You need to make sure that your cluster listens on externally visible addresses, otherwise the server will return a config to the client which it can’t do anything with it. See the discrepancy between the first and second one


#15

@daschl, thanks for the response. I tried changing the server IP to 192.168.25.144 in the “ip_start” file and the server stopped recognizing the buckets. After a long and painful procedure, we have the server back on line, but the same issue persists. How do I change the IP of my already running server?
From what I have understood, we will have to backup the current data, create a totally new instance of the server which is running on the external IP, and cbrestore the old data. Even if we want to change the server to run on a hostname instead of an IP, we will still have to instantiate a new instance of the node. Is this the right way? It would be much easier if there was a way of changing this IP without going through the whole lengthy process. Please provide some tips for the same. Thank you!
EDIT: Also, is it safe to run the server on an external IP this way? Will there be any security lapses due to this?