Problem with Timed out while reading configuration over HTTP

Java SDK
1

Hi,

I’m trying to benchmark couchbase server 2.2 with the YahooCloudServingBenchmark (YCSB).

I’m using the connector couchbase2.0 of Thumbtack which can be found here: https://github.com/thumbtack-technology/ycsb

I’m running two client instances in the cloud an one instance of couchbase server with one data bucket.

When I’m trying to start my workloads in parallel I get the following Exception:

2014-01-29 14:02:05.394 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Timed out while reading configuration over HTTP 2014-01-29 14:02:05.395 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Timed out while reading configuration over HTTP java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)

This problem only happens when I’m running the workloads in parallel (with ClusterSSH or a Fabric Script).

If i’m starting the clients manually (each with 2 seconds delay) all works fine.

I can access the pools over the URL without any problems.

Can you give me any hints why I’m running in this problem ?

Thanks in advance

2

I would recommend you to fork the code and upgrade to 1.3.2, 1.1.0 is rather old.

Also, you can turn on debug logging to get more detailed information on what’s going on. Also make sure that the hostnames are correctly available and you are not having DNS requests that take a long time (because misconfigured) and then will time it out.

If you still see issues, you can go ahead and create an issue on our bugtracker, thanks!

3

Hello,

thank you for your hint.

I have upgraded the project to the current version of the client.

But I’ve I’m trying to start a benchmark I got the following exception:

2014-02-20 15:52:03.325 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Timed out while reading configuration over HTTP java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:423) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:209) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:146) at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:313) at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:252) at com.yahoo.ycsb.couchbase.CouchbaseClient2_0.createCouchbaseClient(CouchbaseClient2_0.java:95) at com.yahoo.ycsb.couchbase.CouchbaseClient2_0.init(CouchbaseClient2_0.java:54) at com.yahoo.ycsb.DBWrapper.init(DBWrapper.java:79) at com.yahoo.ycsb.ClientThread.run(Client.java:317) 2014-02-20 15:52:03.338 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Connection problems with URI http://109.231.122.236:8091/pools ...skipping java.io.IOException: Timed out while reading configuration over HTTP at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:448) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:209) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:146) at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:313) at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:252) at com.yahoo.ycsb.couchbase.CouchbaseClient2_0.createCouchbaseClient(CouchbaseClient2_0.java:95) at com.yahoo.ycsb.couchbase.CouchbaseClient2_0.init(CouchbaseClient2_0.java:54) at com.yahoo.ycsb.DBWrapper.init(DBWrapper.java:79) at com.yahoo.ycsb.ClientThread.run(Client.java:317) Caused by: java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300) at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:423) ... 8 more com.yahoo.ycsb.DBException: com.couchbase.client.vbucket.ConfigurationException: Configuration for bucket "test" was not found in server list ([http://109.231.122.236:8091/pools]).

Do you have any hints for this ?