Cbbackupmgr for community-7.0.0-beta FULL now INCR

Hi,

A couple of days ago, whenever I ran the Cbbackupmgr backup it always did a full backup no matter how often I ran it.
I liked it, as I could cleanup easily in the backups.

Now a few days ago it seems like the Cbbackupmgr backup rutine have shifted to do FULL and then Incremental backups.
This is of course super cool as I can run it more often, but cleanup is now very difficult as I can’t “merge” as that’s a enterprise feature …

What is the take on this ? Will merge be available in community 7.0.0 ?
Also I would love a way to force Cbbackupmgr to do FULL backup, not INCR.

Hi @dabjo,

Welcome to Couchbase Forums!

cbbackupmgr has been made available in our community edition with restrictions. Merge will only be available in the enterprise edition. There is no way to force a full backup with cbbackupmgr though. It will do an incr backup everytime you run it unless you delete all the previous backups. If you deleted the backups, then it will start with a full backup again.

But you bring up a good point though. The cbbackup tool has a mode option which let the community users specify if they want a full or incr or accumulated backup (cbbackup | Couchbase Docs) everytime. But with cbbackupmgr, they can only do incr backup always. I will check with the team on this and get back to you.

Best,
Arun

Yes I do know about the cbbackup been using it. I’m looking at the cbbackupmgr in the beta specifically as it’s made available for the community version and cbbackup is being deprecated. Just want to future proof our production backup solution so it’s ready to make the change.

Yes, you need to make sure that all features of cbbackup is to find to some extend in cbbackupmgr as it’s being deprecated from 7.0.0!

It looks like the reason I was seeing the “FULL” backup every time I ran a simulated backup from before was when I’m starting out backing up a empty cluster with only a couple of buckets created no data. Then each run with cbbackupmgr returns a “FULL” backup apparently. See example log output.

Name            | Size    | # Backups |
backup_saturday | 16.61KB | 21        |
+  Backup                         | Size | Type | Source                    | Cluster UUID    | Range | Events | Aliases | Complete |
+  2021-01-30T00_15_29.562759128Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T00_30_33.51883174Z  | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T00_45_35.794521494Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T01_00_18.464644455Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T01_15_21.151625957Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T01_30_25.172566181Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T01_45_28.675859373Z | 810B | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |

What I would expect was this, that even though it’s the same structure/data/no-data that it would do 1 FULL and rest would be INCR.
When i then fill in some data to one of the buckets, it then changes behavior to using INCR … As you can see in the below output log

+  2021-01-30T11_46_14.485702968Z | 810B   | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T12_01_18.522300574Z | 810B   | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T12_16_21.650098628Z | 810B   | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T12_31_25.208793503Z | 3.70MB | FULL | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T12_46_19.20958751Z  | 1.67KB | INCR | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |
+  2021-01-30T13_01_23.021046641Z | 1.67KB | INCR | couchbase://node1.cluster | 6bd5e...b73905c | N/A   | 0      | 0       | true     |

The changing behavior (will of it’s own) depending on the structure/no data etc makes it an issue to cleanup, as now I have to think about two scenarios when scripting.
It’s like cbbackupmgr is very focused on that you have the “merge” feature so it don’t care how it’s creating it’s backups as the user can always run a merge script and clean everything up to only 1 full backup. But for community this is not an option!

Hello @dabjo ,

There is a hidden option that forces a full backup --full-backup as such this is not supported, I have opened MB-44184 to look at making this option documented and supported in the future.

What I would expect was this, that even though it’s the same structure/data/no-data that it would do 1 FULL and rest would be INCR .

What decides if a backup is Full or INCR are the documents in the buckets and not the metadata (indexes, views, etc). Empty buckets are always seen as full, while CE does not have merge it does have the remove option which can be used to do clean up.

The explanation on why the overview stated FULL on all backup snapshots makes perfectly sense. Thanks!

Yes since the MERGE feature is not present the only way for community users is to use the remove function do delete ALL backup snapshots FULL+Any Incrementals and then do a new FULL backup in order to “cleanup”. Where as if you had the merge you could simply merge full+incrementials into a new “FULL” that would cover everything up until now and then the next would simply be a incremental.

So it’s a bit heavy on the community solution as we have to delete/remove everything and then do a completely new FULL backup. Meaning etc deleting 50-70GB of data then do backup again of 50-70GB of data :slight_smile: This is very time consuming heh …