New couchbase document gets high revision

I am using Couchbase 5.0.1. I have a bucket which contains about 25,000 docs. If I create a new Document through the “Add Document” button in the Couchbase Server web GUI, I would expect it to get a revision ID of “1-…” (assuming a document with this ID hasn’t existed before). However, it has a much larger revision ID (e.g. “457-…”)

If I flush out all the documents in this bucket then make a new document, it gets revision ID “1-…” as expected. However, if I manually delete all the documents (using the Bucket.remove() method of the CB API) rather than flushing, then creating new documents still get the high random numbers, so there’s something strange about the bucket contents.

Why would a new document get these seemingly random high revision numbers?

Thanks,
Giles

Hi @giles,

I’m not sure why specific numbers are assigned. I guess to answer your question with a question, why does it matter? Are you relying on revision numbers for some sort of business logic? It’s likely you would be better served by the CAS values, or storing your own revision number inside the document.

Hi @matthew.groves,

I don’t really care what the revision numbers are (we don’t use them directly) but it matters because it stops XDCR from working. We create the document and it gets a high revision number, however the XDCR-replicated document on another server has revision 1. This means that if the document is modified on the other server, it does not get replicated back again as it has revision 2 which is less than 457 (or whatever).

This behaviour was first noticed after migrating from CB4 to CB5. It’s also only seen in some buckets and if we flush the bucket it doesn’t happen. However if we manually delete all the documents rather than flushing the situation is the same. Also, in a bucket that causes the issue, if we replicate it with XDCR to a different server, then the bucket on the new server has the same issue, so it follows the data.

Thanks,
Giles

Note: this sounds similar to the issue mentioned in Weird revision for new documents in CB 5

As per that thread; I’d recommend raising an issue at issues.couchbase.com so it can be investigated further.

2 Likes

Giles,
When you created a document and its revision was 457, was replication running at that time? If so I would expect this created document to get replicated to target with the same revision, 457. This is what I saw in my own environment when I tried the scenario you described.

2 Likes

Thanks @drigby, I’ll do some more investigation and raise an issue when I have more details. It doesn’t happen on every bucket so trying to find out if there’s some kind of pattern first.

Hi @ysui6888, yes this problem only happens if replication is running at the time - the document gets replicated with revision 1.

If replication is not running and the document is created, then when you start replication the document gets replicated with the correct number (457 in this case).

When you just tried it, did you have a random high revision number similar to this?

When I tried it, I got a revision number like 3 - since I deleted a doc with the same key only once before. Not sure if I reproduced the same issue with revision as yours.
I had replication running when i created the doc, deleted it, and recreated it. The ending snapshot of the document on target had revision of 3, which was the expected and correct value. The same happened when I stopped replication in the middle. I was not able to reproduce the issue with replication.
If you have a reproducible case/environment, please turn replication log level to Debug, reproduce the issue, do cbcollect, send us the logs, and turn replication log level back to Info.

Hi @ysui6888, I think your situation with the revision number 3 is working corretly. In my case I am not using a document ID that had previously been used. The explanation for higher numbers provided by @drigby in the issue that he linked to does make sense (i.e. that higher numbers can come about as a result of any deleted documents, they don’t have to have the same ID). However, that would still mean that all new docs would start with the same higher revision which also isn’t happening.

If I start a new bucket and add and delete a few documents, I do see exactly the behaviour that you describe - numbers increase if a document with the same ID is deleted then added again (as you would expect) and this replicates correctly.

This problem is specific to certain buckets after a certain amount of use. Something happens to the data or there is some kind of historical sync data that causes it. The problem still happens if you start a new bucket and either XDCR or use cbrestore to populate the new bucket with the same data. So it is definitely somehow linked to something in the data.

I think the key, that I may not have mentioned before, is be that the bucket was migrated from CB 4.5 to CB 5.0. I didn’t see the behaviour on CB 4.5, and I haven’t seen it on any bucket created on CB 5.0, at least until I restore/replicate the migrated data.

The bucket in question contains confidential customer data so unfortunately I can’t send you the backup. However, I can reproduce this behaviour every time in that bucket, so I can try changing the replication log level and using cbcollect to send you logs. Can you tell me how to do this please, or send me a link to an appropriate doc. I’ve looked in the “Logs” page on the Couchbae GUI, but I don’t see anything obvious to change the log level.

@drigby, when I have these logs, as well as sending them to CB, I will raise a Jira with these.

Thanks,
Giles

In replication creation/edit view, there is a “Show Advanced Settings” switch. Click on the switch, and more fields will be displayed under the switch. The last field/setting is named “XDCR Logging Level”, which is defaulted to “Info”. You can change it to “Debug”, repro the issue, and do cbcollect. Just remember to change the log level back to “Info” afterward. Otherwise there may be significant performance degradation.