I’m having issues taking a backup (using cbbackup) from a live production cluster. I’ve previously posted about this a couple months ago (thread link below) on the Couchbase Server category. After further investigation, it doesn’t seem like the couchbase processes are failing (from what I can tell), so I’m focusing on the Java client.
In a nutshell, we’re on AWS EC2, 5 instance cluster. As soon as cbbackup is started, the Java client immediately begins to fail - timeouts from reads, Temporary failure (ERR_TEMP_FAIL) on writes. I’ve pared the workload back to nearly zero (load avg < 0.25 on all instances) and the client still fails. Our app (i.e. the client) is running on the same 5 instances as the couchbase cluster. I realize (now) that’s not ideal, but we migrated from a similar memcached configuration.
We’re running Couchbase Server 2.2.0, Java SDK 1.4.2. The cbbackup is running on a separate EC2 instance from the cluster. Our EC2 instances are m3.larges - I realize they are only 2 virtual cpus, but at such low cpu, I would think I wouldn’t see issues.
I’m happy to post more info about our setup, but looking for ideas on what could be going wrong and steps to take to further debug and isolate the problem.
Previous post from Couchbase Server category