I have tested different sync_gateway configurations and found out that in our use case the absolute winning configuration is to use separate SG for pull and push. And configure them so that push sg process has high priority and pull low.
We are seeing 10x performance improvements using this setup, because now push-events will occur at higher priority than pull and they stop causing more “entropy” to the whole system later on when every client is just pulling. I am using about 500 clients to test this.
I am wondering if there is already some configuration option in SG that could be used to adjust push & pull priority according to the load application use case is causing?
One other interesting observation in this test is that sync_gateway takes about 80% of CPU all the time and couchbase server is doing very little work (like 7% cpu)! This is also interesting to me, that why is it actually that sync_gatweway is such a bottleneck?