How can I backup documents with all old revision?


#1

I am using Couchbase 3.0.2 as database of Sync Gateway 1.1.0
As I know, I can backup documents with all old revision by XDCR when I set up XDCR while bucket is empty. but XCDR is not safe when my documents in the source bucket is broken (The broken documents will be replica to target bucket without warning or fallback mechanism).
So I want to know that how can I backup documents with all old revision which I want to backup using cbbackup or other commend/tools/ways, to backup documents without XCDR .
And, How can I restore the documents which I want to restore(SG can use it without any changed).


#2

As I tested, If I restore bucket by cbrestore which backup by cbbackup. I will lose the old revision documents.


#3

My test scene as following:

  1. create bucket dev4s;
  2. config SG to connect the bucket dev4s;
  3. put some docs by SG;
  4. update some docs by SG;
  5. backup full bucket dev4s by cbbackup;
  6. create bucket dev4d;
  7. restore bucket dev4d by cbrestore;
  8. Change SG to connect the bucket dev4d;
  9. get old revision docs by SG;
    return 401.
    Note: at the same time, I can not get old revision docs by SG which connect to the bucket dev4s;

#4

I tested it again,I have 3 old revision docs. but I can only get 1 old revision documet, and ** I lose 2 old revision documents.**


#5

What do you mean when you say “3 old revision docs”? Non-conflicting revision documents aren’t permanently archived by Sync Gateway, regardless of backup/restore. They are set to auto-expire after 5 minutes.

If it’s conflicting revisions, those bodies get stored in the sync metadata for the main document, and so shouldn’t be stored as separate docs either.

Can you share some more details on the ‘old revision docs’ you’re expecting to see?


#6

for example, I create a doc using:

curl -X PUT -H "Content-Type:application/json" http://testuser:password@127.0.0.1:4984/sgdb/doc1 --data '{"key":"value1"}'

I got the following return:

{"id":"doc1","ok":true,"rev":"1-9b22d08abd210d0876c2f7c1a0afd70f"}

then I update the doc with:

curl -X PUT -H "Content-Type:application/json" http://testuser:password@127.0.0.1:4984/sgdb/doc1?rev=1-9b22d08abd210d0876c2f7c1a0afd70f --data '{"key":"value2"}'

I got doc with:

{"id":"doc1","ok":true,"rev":"2-cba53be1da1f15e7730c60c56bfd6d6d"}

Then I updated again:

curl -X PUT -H "Content-Type:application/json" http://testuser:password@127.0.0.1:4984/sgdb/doc1?rev=2-cba53be1da1f15e7730c60c56bfd6d6d --data '{"key":"value3"}'

I got doc with:

{"id":"doc1","ok":true,"rev":"3-078f68ab617effe29019b27466e1594f"}

If I didn’t get the old revision docs from source bucket.after I using cbbackup to backup the bucket,I can not get old revision docs from target bucket.


#7

I am not quite sure what is this mean. after I modify the docs some times but don’t get them, such as using:

curl -X GET -H "Content-Type:application/json" http://testuser:password@127.0.0.1:4984/sgdb/doc1?rev=2-cba53be1da1f15e7730c60c56bfd6d6d

after I backup the bucket and restore.I can not get doc1 with rev=2-cba53be1da1f15e7730c60c56bfd6d6d


#8

Old, non-conflicting revisions aren’t retained by Sync Gateway. In the example above, the body of your rev 2-cba53be1da1f15e7730c60c56bfd6d6d revision will be compacted away by Sync Gateway after a period of time, independent of whether you’ve done a backup/restore or not.

Some relevant details on the revision model are documented here, including a discussion on revisions being compacted away:
http://developer.couchbase.com/documentation/mobile/current/develop/guides/couchbase-lite/native-api/revision/index.html


#9

but why I got the old revision docs before I backup the bucket, I can obtain the docs which I got it from source bucket?


#10

My guess would be the timing of the retrieval on the pre-backup bucket. Old revisions are set to expire 5 minutes after the new revision arrives. Were you retrieving in that window?


#11

Yes, I tested again, I can not get old revision docs any more both in source and target bucket.
How Can SyncGateway store all old revision docs in CB? I think restore docs by revision is fascinating feature.