We are using Couchbase 2.0.1 with 4 nodes in an environment.
We got an incident recently that 1 of the nodes was down and we found some data loss after the auto fail-over.
From the stats log, we grabbed the total “curr_items” and “vb_replica_curr_items” per log entry and found that the number of “vb_replica_curr_items” are few thousand less than “curr_items”.
After re-adding the failed node and rebalance (with a rebalance success returned), we used “couchbase-cli server-info” to get the numbers again. Still, the number of “vb_replica_curr_items” is 640 less than the “curr_items”.
First, may I know if I am correct using these 2 values to calculate the number of keys and replicas?
Second, may I know the reason why the number of replicas will be less than the number of keys even after a success rebalance? Is this a known bug in 2.0.1?
Thanks a lot.