High memory usage


#1

I am seeing a behavior where if I update a document
(last updated date) every time an event happens, the amount of memory
and # of documents used in sync gateway grows very large.

It appears that a “sync document” is created whenever a change is made to a document.

I need to find out whether this is expected. I wonder if sync gateway will eventually compact this.

If this is expected behavior, it will limit how we can use sync gateway.
For instance, we may need to calculate the last message date on the fly
on the client side.


#2

I did a proof of concept where I have a device document and I processed 1000 events (about 1 year of data). Each time I processed an event, I updated the device document. The database had ~2000 documents because there were 1000 events, 1 device document, and 1000 sync documents. Is this expected?


#3

Couchbase Mobile uses MVCC, so it’s expected that each update to the device document is going to generate a new revision. Previous revisions get stored as _sync documents. (more details here on MVCC http://developer.couchbase.com/mobile/develop/guides/couchbase-lite/native-api/revision/index.html)

Older revisions are temporarily stored in Couchbase server with the _sync prefix (as you’ve seen). When using Couchbase Server, these old revisions should get removed automatically after 5 minutes. If you’re using walrus for your proof of concept, these won’t get automatically cleaned up - walrus doesn’t have auto-expiration support.

In either case, you can also call _compact on the admin API to remove the old revisions. There’s some (admittedly sparse) documentation on the _compact call here: http://developer.couchbase.com/mobile/develop/references/sync-gateway/admin-rest-api/index.html