Doc size limited in push replication?


#1

In my Sync Gateway log I see messages like this:
2016-09-27T17:50:10.626-07:00 BulkDocs: Doc “MostRecentBluetoothDeviceRecords95_kO6cYg6v-4F7xD9Q0AOoDegqxPLdpgnODJdBmVFw” --> 413 Too Large: Too large (MCResponse status=E2BIG, opcode=SET, opaque=0, msg: Too large)

Does that mean my document will not get replicated to the server ?
Or will the system fallback on another replication method for large documents ?

In my case the document becomes large not because of its application data, but due to a very deep revision history (100’s of thousands of revisions)

How should I handle this ? Thanks.


#2

Couchbase Server has a 20MB limit for documents.

The revision history is not supposed to grow indefinitely … there may have been a bug in earlier versions of Sync Gateway that prevented it from being pruned, but I’m pretty sure this has been fixed as of 1.3. (@adamf?)


#3

@jens: I verified that the document does not grow indefinitely. However it does get larger than 20MB. From my server I retrieved the last successfully pushed rev of this large document. It has 180 entries in _sync.history.bodymap. That array is by far the largest piece of the document.

Should I use the sync_gateway property ‘revs_limit’ to reduce upper bound for this array ?
I tried that, but it does not seem to have an effect on the size of _sync.history.bodymap.
Attached is the large document: TooBigToPush.zip (582.8 KB)


#4

The attachment didn’t work … it downloads as a 0-byte file. Try Dropbox maybe?

180 revisions in the doc history won’t push the doc nearly to 20MB; each revision only occupies a few hundred bytes at most.


#5

Uploaded here: https://drive.google.com/drive/folders/0BxH32PUWxhEebEJvTFNIU3BiVkk?usp=sharing
You’ll see that each of the 180 entries in _sync.history.bodymap is a full copy of my doc’s user properties (~110KByte in my case).
Thanks.
Rainer.


#6

Ah. Those are all conflicting revisions that haven’t been resolved. That’s why SG saves their bodies. It sounds like none of the client apps (nor any server-side processes) are detecting or resolving conflicts.


#7

Thank you. That points me in the right direction.