Was there a bug associated with this issue ?
I have been experiencing similar issues with 1.4.4 where netty IO threads are in WAIT state. Its frustrating that we cannot reproduce this issue in LnP but in production during heavy traffic hours we are experiencing this issue. When this issue occurs the application server will no longer be taking any traffic and just restarting app server won’t help instead we had to reboot the VM to kill those dangling connections.
Here is the stack from the thread dump we took when this issue happened.
Thread Name
Couchbase View Thread for node cbnibslc02-289848/10.120.159.104:8092
State
Waiting on condition
Java Stack
at sun/nio/ch/EPollArrayWrapper.epollWait(Native Method)
at sun/nio/ch/EPollArrayWrapper.poll(EPollArrayWrapper.java:228(Compiled Code))
at sun/nio/ch/EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81(Compiled Code))
at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:87(Compiled Code))
at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:98(Compiled Code))
at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:305)
at com/couchbase/client/http/AsyncConnectionManager.execute(AsyncConnectionManager.java:89)
at com/couchbase/client/ViewNode$1.run(ViewNode.java:89)
at java/lang/Thread.run(Thread.java:780)
Native Stack
(0x00007F0D06106052 [libj9prt26.so+0x13052])
(0x00007F0D061136CF [libj9prt26.so+0x206cf])
(0x00007F0D06105D9B [libj9prt26.so+0x12d9b])
(0x00007F0D06105E97 [libj9prt26.so+0x12e97])
(0x00007F0D061136CF [libj9prt26.so+0x206cf])
(0x00007F0D061059BB [libj9prt26.so+0x129bb])
(0x00007F0D060FF812 [libj9prt26.so+0xc812])
(0x00007F0D07252B40 [libpthread.so.0+0xfb40])
pthread_cond_wait+0xca (0x00007F0D0724EA9A [libpthread.so.0+0xba9a])
(0x00007F0D0634D0CF [libj9thr26.so+0x80cf])
(0x00007F0D064BA859 [libj9vm26.so+0x63859])
(0x00007F0D052DE6D9 [libj9jit26.so+0x5b96d9])
thanks