I am using Couchbase Server Community Edition 6.6.0 build 7909 on a Ubuntu server and a few times in the past month, the indexes on my server suddenly getting thousands of mutations and this increases into the millions.
The indexes are showing as ‘ready’ and are queryable.
This does eventually go back to normal but when it is processing all the mutations any new documents aren’t returned when the index is queried.
The sync gateway requests timeout when there are loads of mutations and then once it comes back up the changes feed starts processing a load of deleted objects that have been deleted previously.
We previously had 4 servers but 3 were on Docker and 1 wasn’t, the 1 that wasn’t also had all the indexes on due to only recently adding the Docker servers. I ran failover on that server and spread the indexes out on the other servers and thought that was it, but it happened today on the Docker servers.
I’ve attached the logs from the 17th at about 10:20 it happened from the none Docker server.
CBServerLogs.zip (3.1 MB)
Screenshot is from today on the Docker server but similar thing happened on the other server
I did have the ejection mode on full and have changed this back to value only today and also reduced the memory quota for the indexes as the server was on ~21GB/~23GB which doesn’t help.