After deleting -> purging -> compacting documents is there always data left in the database, meaning the database grows on disk indefinitely? Is there some other cleanup that happens after some time?
I noticed this behavior on OSX with SQLite. In this use case I don’t need syncing/revisioning. I want to temporarily store logging data on a mobile device and after sending the data to a remote API, the data can be removed.
If I wanted to completely get rid of data on disk, would the proper pattern be to remove the database and recreate it?
Purging a document will remove all data associated with its revisions, and then compacting the database will garbage-collect any unreferenced attachment data. So there shouldn’t be anything left over.
On the master branch of CBL/iOS (for the 1.3 release) we have a class that implements the behavior you’re describing: collecting small timestamped pieces of data and pushing them to a server. It takes care of purging the data after it’s been successfully pushed. The class (CBLTimeSeries) isn’t exposed in the public API yet, but of course you can use it anyway.
It seems that the docs table is never emptied in Couchbase Lite, which ends up as minor amounts of data on disk. That plus the fact that the SQLite Write-Ahead Log (WAL) file never seems to be truncated on OSX lead me to believe that the database sizes grew unreasonably fast and were not completely emptied.
On Android/iOS the WAL file seems to behave nicer and it is truncated with compaction.
Currently I need support for both Android and iOS using Cordova, so I’ll have to come back to CBLTimeSeries some time later.
The docs table size is pretty minor; it just stores the docID and a primary key.
The WAL is maintained by SQLite itself. I haven’t looked at its size, but I’ve been assuming it would behave itself, i.e. shrink back down some time after a commit. Also, the Cocoa implementation doesn’t configure the SQLite database any differently on Mac OS vs. iOS, so it’s surprising that the WAL would behave differently.
Are there specific scenarios on Mac OS that cause the WAL to end up too large, or do you find that this just happens through normal usage?
I was just fiddling around, trying to figure out how Couchbase Lite works and noticed that the WAL file does not shrink ever (at least during my testing, maybe after some time X?). It seemed like SQLite reserves whatever amount of space it needs for the WAL file and never shrinks it.
I’m remembering vaguely that the WAL may get cleaned up when SQLite closes a database. But from what I’ve seen, apps generally don’t bother to close CBL databases on quit, and it’s generally not a problem (it can’t cause data loss.) It might be contributing to the WAL growth, though?
Try closing the CBLManager when your app quits, and see if it makes a difference.