Thanks for that, I’m able to mostly mitigate this problem after implementing client retry procedures (at 1000ms intervals). I’m still curious as to why the failures are happening though, and whether there are anything we can do to reduce their occurrence. Both the client and the Couchbase cluster are hosted on AWS within the same region, so network connection and speeds shouldn’t be an issue.
Another note is that I sometimes get the response status of
None during the periods of error, which I’ve also included as a status that warrants retry.
I’ve enabled logging and can see the following during the periods of which errors occur:
2017-09-04 10:07:30,727  WARN Couchbase.IO.ConnectionBase - Handling disconnect for connection 72322f00-8a4a-488d-9924-d351daac6a3f: System.IndexOutOfRangeException: Index was outside the bounds of the array.
at Couchbase.IO.Converters.DefaultConverter.CopyAndReverse(Byte src, Int32 offset, Int32 length)
at Couchbase.IO.Converters.DefaultConverter.ToInt32(Byte buffer, Int32 offset)
2017-09-04 10:07:30,758  WARN Couchbase.IO.ConnectionBase - Handling disconnect for connection 72322f00-8a4a-488d-9924-d351daac6a3f: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
at System.Net.Sockets.Socket.Send(Byte buffer, Int32 offset, Int32 size, SocketFlags socketFlags, SocketError& errorCode)
at System.Net.Sockets.Socket.Send(Byte buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at Couchbase.IO.MultiplexingConnection.SendAsync(Byte request, Func`2 callback)
I have the logging level on WARN at the moment as I couldn’t see anything useful under DEBUG during brief testing, but the DEBUG level logs were huge so I might’ve missed something.