Hi, thanks for the response!
We just upgraded everything to CBL 2.1.1, and SGW 2.1.0
CBServer is 5.5.1
I haven’t re-tested yet, and I wanted to play with the SGW config to try to optimize our replication-- but again here is the main question:
Can we stop clients from pulling tombstone revisions, for documents they don’t have locally?
The main issue is that now the clients are pulling all of these tombstone revisions for documents they never even had.
But going backwards, the only reason this is a concern is because these tombstone revisions take a while to sync. Our entire replication process, for about 500K documents (which ostensibly includes tombstones), takes ~10 minutes? But with the users constantly deleting documents and reuploading new versions, we could easily end up with millions of documents/tombstones.
But we only have maybe 200K documents at any given time, that aren’t tombstones.
And it’s not clear how long it takes to push either, because our users could update 100K documents at once, we ideally want that push replication to happen as fast as possible. If the user quits the application while the push is still happening, we currently don’t have any support for handling that scenario
**EDIT:
So actually maybe our SGW config could be optimized as well? For our MAX 300K documents at any given time, if that could sync within 30 seconds, that would be our goal.
Here is an example of a config for one of our databases:
"live-partDB": {
"server": "http://db:8091",
"username": "Administrator",
"password": "password",
"bucket": "live-partBucket",
"users": {
"GUEST": { "disabled": false, "admin_channels": ["*"] }
},
"allow_conflicts": false,
"replications":[
{
"changes_feed_limit": "10000"
}
],
"num_index_replicas": 0
},
Our Bucket settings:
100MB Memory,
Bucket Type: Couchbase,
Replicas Enabled, 1
Replicate View Indexes: enabled
Compression Mode: Passive
Ejection Method: Value Only
Bucket Priority: High (we set them all to high)
Couchbase Server Settings:
Data 2GB
Index 2GB
Search 500MB
Analytics 1GB
Eventing 500MB
I think everything else is default
We just want to sync everything as fast as possible, but it’s unclear why it seems to take so long to replicate right now. Our server seems strong enough, the thought is maybe our configuration isn’t ideal. We’ll do whatever it takes to replicate 300K documents within 30 seconds