I am using Couchbase server with Sync Gateway.
I have a cluster with 3 nodes.
Mobile devices are adding data to couchbase server through Sync_gateway.
For few day it went smoothly. Data got accumulated in CB server.
It was around 15 million records, out of that 8 million records are not useful, so I have removed those records using N1QL delete query.
After this delete operation,
The ‘cbq-engine’ process is always showing high CPU usage and high memory usage.
The memory usage by ‘cbq-engine’ keeps on growing and it takes almost entire free memory, at this point this process is terminated by OS (shows ‘defunct’ if observed using ‘top’ command in Linux) then the ‘cbq-engine’ gets restarted.
This cycle continues and at some point that server node goes down.
Also it is observed that ‘Operation per second’ are very high, more that 12 K per second when there is not that much activity (insert recs through Sync_gateway or N1QL query etc.).
Is this problem is related to deleting records using ‘N1QL’ query ?
What can be cause of this and how to remove it ?
I have also tried by drop and create the GSI indexes. But it did not helped.