I have been load testing the beta 3 version. It has certainly improved performance. Although I still see Memcached gets 5 hits per .aspx hit, but still gives better performance. I can go over 400 req/sec, which is as good as SQL Server session performance:
However, there’s a new problem:
You can see from the below graph that suddenly cache hit rate goes down, and I can see pages are unable to retrieve their sessions. This problem wasn’t there in the previous version.
This also happens randomly, even under small load.
When the problem starts, I can see ops/sec goes down to zero. It no longer sends any request to Couchbase server.
My Jmeter tests start failing when this problem happens:
When I restart the website, it starts working again. So, I believe this is a problem on the code, not on the Couchbase server.
Another observation is, if I do not warmup the site after restart, and run the load test on a cold site, where many requests get queued, the session retrieval problem starts right from the beginning. Not a single time Home page is able to retrieve session after login. So, I believe somewhere there’s a race condition shutting down all access to couchbase server.