Intermittent failures in couchbase server

Hi,
I am using community version of couchbase 3.0.1 version and I having cluster of 3 machines. In the couchbase UI, I can see multiple errors:

  1. Control connection to memcached on ‘ns_1@xx.xx.xx.xx’ disconnected: {badmatch, {error,closed}} (repeated 5 times)

  2. Port server memcached on node ‘babysitter_of_ns_1@127.0.0.1’ exited with status 134. Restarting. Messages: Tue Apr 4 18:05:49.389098 IST 3: (No Engine) Spawning 4 readers, 4 writers, 1 auxIO, 1 nonIO threads
    Tue Apr 4 18:05:49.685178 IST 3: (fastlane) metadata loaded in 292 ms
    Tue Apr 4 18:05:50.826981 IST 3: (fastlane) warmup completed in 1434 ms
    Tue Apr 4 18:05:53.036698 IST 3: (fastlane) Warning: failed to fetch data from database, vBucket=44 key=064773d4754a563dc8d85152dfa04d7e5b5366ba error=document not found [none]
    asssertion failed [fileRev] at /home/buildbot/buildbot_slave/debian-7-x64-301-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1823

In the error logs of couchbase machine, I can getting error as:
[ns_server:error,2017-04-04T18:12:38.687,ns_1@10.85.125.51:ns_doctor<0.1151.0>:ns_doctor:update_status:229]The following buckets became not ready on node ‘ns_1@10.85.125.51’: [“fastlane”], those of them are active [“fastlane”]
[stats:error,2017-04-04T18:12:44.358,ns_1@10.85.125.51:<0.1549.0>:stats_collector:handle_info:124]Exception in stats collector: {exit,
{{badmatch,{error,closed}},
{gen_server,call,
[‘ns_memcached-fastlane’,
{stats,<<>>},
180000]}},
[{gen_server,call,3,
[{file,“gen_server.erl”},{line,188}]},
{ns_memcached,do_call,3,
[{file,“src/ns_memcached.erl”},
{line,1401}]},
{stats_collector,grab_all_stats,1,
[{file,“src/stats_collector.erl”},
{line,84}]},
{stats_collector,handle_info,2,
[{file,“src/stats_collector.erl”},
{line,116}]},
{gen_server,handle_msg,5,
[{file,“gen_server.erl”},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,“proc_lib.erl”},{line,239}]}]}

The above error in UI and error logs are happening intermittently and at same time one of couchbase machine is also going down intermittently. Though it gets up instantly within 30seconds after it goes down, but I can’t figure out why this is happening.
Can someone please point out the issue here.
Thanks in advance

3.0.1 is pretty ancient now (released October 2014). There’s been many fixes since then - see https://developer.couchbase.com/documentation/server/3.x/admin/rel-notes/rel-notes3.0.html - in fact 3.x it went End-Of-Life earlier this year

Suggest you upgrade to the current CE release 4.5.0.

Hi drigby,
Thanks for the reply. We plan to move to latest CE release 4.5.0 later this month but is this bug anyway specific to version 3.0.1 ?
Thanks,
Aman

Suggest you check the aforementioned 3.x release notes - or search for fixed issues on issues.couchbase.com

Found this issue under https://issues.couchbase.com/browse/MB-14496 :slight_smile: