CbBackup Tool gets interrupted after 30s of inactivity for specific buckets in a cluster

Couchbase Server
1

I have a cluster with 4 buckets in it. Whenever I try to take backups for the buckets individually, two of the buckets get backed up while the other two do not get backed up.

For the buckets which do not get backed up, I got this message in the console.

w0 no response for 30 seconds while there 1024 active streams

This is the command I’m running for each bucket.
./cbbackup http://localhost:8091 /datadrive/cb-backups/ -u <USERNAME> -p '<PASSWORD>' -b <BUCKET_NAME> -m full

These are the specs for those two buckets which were not getting backed up.

Bucket 1 - 4GB RAM, currently has around 400,000 documents.
Bucket 2 - 4GB RAM, currently has around 150,000 documents.

It’s worth noting that we first had 2GB ram in both the buckets. After increasing the RAM for both the buckets, backups started working again but the same error occured from the next day

Is there an inherent problem with the CbBackup tool? Does anyone know how the backups are actually taken? That would give more insight into why this error might occur.

DISTILLED TECHNICAL INFORMATION:

  • Couchbase Server - Community Edition 5.0.1 build 5003
  • Command used - ./cbbackup http://localhost:8091 /datadrive/cb-backups/ -u <USERNAME> -p '<PASSWORD>' -b <BUCKET_NAME> -m full
    *Bucket Specs -
    * Bucket 1 - 4GB RAM, currently has around 400,000 documents.
    * Bucket 2 - 4GB RAM, currently has around 150,000 documents.

Thanks for your valuable time.

2

Hello @Sriram_R, welcome to the Couchbase Forums.

w0 no response for 30 seconds while there 1024 active streams

This warning message means that cbbackup has not processed any data from the cluster for over 30 seconds. There are a number different of root causes, to name just a few:

  • The backup machine is overloaded and is busy writing the backup to disk
  • The cluster is overloaded and it can send any more data
  • There was a network issue causing data to be delayed

Before starting to debug there issue further it would be good to know what version of Couchbase Server is being used? There is a known issue where the server would not correctly close the connection.

3

We’re using Community Edition 5.0.1 build 5003. All the details are mentioned in the question. Thank you.

4

Given that you’re on Couchbase Server 5.0.1, it’s possible that you are hitting MB-31481 where the end snapshot is not being sent to cbbackup. I would recommend upgrade to Couchbase Server 6.0.0 and seeing if you end the problem again.

5

I’m using couchbase community Edition 6.6.0. But same error facing while getting cbbackup. still it’s a bug or other issue?

6

I’m using Couchbase Server Community Edition 6.0.0 build 1693 and I’m facing the same issue.
Can anyone help us?