Hi,
Since the last couple of days, I am noticing a strange new error on the client side. The app is started and runs ok for a while. Then suddenly the logs start showing:
46442 INFO | jvm 1 | 2013/09/01 05:09:38 | 2013-09-01 05:09:38.568 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@52f0684e
46443 INFO | jvm 1 | 2013/09/01 05:09:38 | 2013-09-01 05:09:38.569 INFO net.spy.memcached.MemcachedConnection: Reconnecting due to failure to connect to {QA sa=edc-cache-app5.myOrg.com
46443 /10.63.52.31:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
46444 INFO | jvm 1 | 2013/09/01 05:09:38 | java.net.ConnectException: Connection refused
46445 INFO | jvm 1 | 2013/09/01 05:09:38 | at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
46446 INFO | jvm 1 | 2013/09/01 05:09:38 | at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
46447 INFO | jvm 1 | 2013/09/01 05:09:38 | at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:407)
46448 INFO | jvm 1 | 2013/09/01 05:09:38 | at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:275)
46449 INFO | jvm 1 | 2013/09/01 05:09:38 | at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:2030)
46450 INFO | jvm 1 | 2013/09/01 05:09:38 | 2013-09-01 05:09:38.569 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=edc-cache-app5.prod.gecwalmart.com/10.63.52.31:11210, 46450 #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}, attempt 1.
And then it repeats every 30 seconds (just to give a sample):
51042:INFO | jvm 1 | 2013/09/01 05:36:36 | java.net.ConnectException: Connection refused
51101:INFO | jvm 1 | 2013/09/01 05:37:06 | java.net.ConnectException: Connection refused
51191:INFO | jvm 1 | 2013/09/01 05:37:36 | java.net.ConnectException: Connection refused
51256:INFO | jvm 1 | 2013/09/01 05:38:06 | java.net.ConnectException: Connection refused
51348:INFO | jvm 1 | 2013/09/01 05:38:36 | java.net.ConnectException: Connection refused
51425:INFO | jvm 1 | 2013/09/01 05:39:06 | java.net.ConnectException: Connection refused
51514:INFO | jvm 1 | 2013/09/01 05:39:36 | java.net.ConnectException: Connection refused
51591:INFO | jvm 1 | 2013/09/01 05:40:06 | java.net.ConnectException: Connection refused
51694:INFO | jvm 1 | 2013/09/01 05:40:36 | java.net.ConnectException: Connection refused
51775:INFO | jvm 1 | 2013/09/01 05:41:06 | java.net.ConnectException: Connection refused
When I check on the cluster side, everything looks ok. Any ideas? We are running CB 2.0.1.
regards,
-Piyush