Cbtransfer very slow

Couchbase Server
1

Hi,

I am running cbtransfer to copy a bucket from one server to another, but it takes ages even though there are only 12 records. There were milllions but they were deleted (may be relevant).

my command:

./cbtransfer http://server1:8091 http://server2:8091 -b b1 --bucket-destination b1 -u username -p password

The status keeps updating and going up (currently: 4156391/estimated 4156395 msgs)

Thanks

2

cbtransfer does move the tombstones (deleted documents). In this case it’s transferring 4,156,395 tombstones and documents.

3

Thanks, that would explain it. The natural question then arises which is how to avoid that either by:

  1. completely deleting the records (as well as the tombstones) from the source
    OR
  2. only moving real records to the the target (without tombstones)

Thanks

4

In some use cases such as XDCR, backup/restore and indexing, having the tombstones are important for consistency.

completely deleting the records (as well as the tombstones) from the source

Tombstones are automatically removed according to the metadata purge interval, which by default is set to 3 days and can be reduce.

  1. only moving real records to the the target (without tombstones)

All of the tools are design to transfer both the documents and the tombstones. One option is to use XDCR which will also move the tombstones but it is much faster than cbtransfer.

5

Thanks for the information.

Both seem like viable solutions, though not very straight forward. I was hoping to export and import from the web console, so hopefully that will be added at some stage. The export is there, but the import not.