I have a Couchbase bucket that receives approximately 31mm documents / day. It is flushed 3 times / day.
The problem is that I have a cron that runs every 5 minutes and queries couchbase - I am at risk for losing up to 5 minutes of data during the flush.
Is there a way to put a condition on FLUSH, ie “only flush items with c_time < 30 minutes”?
Alternatively, if I replaced “flush” with a conditional bulk delete, how will performance compare over such a large number of documents?