We have a spring boot rest application using spring data couchbase, the application fetch data from couchbase and returns json response. Our service is designed to fetch 2700+ documents from a couchbase bucket that holds around 1,000,000 documents.
The expected response time of the service is 600ms and we could achieve average response as 800ms while performing load testing using Jmeter with 10 users. However the response time have increased dramatically when the test load was increased to 30 users - the average response time rises to 1720 ms; with 500 ms as best case and 3272 ms as worst case.
The application uses Rx API for async operations and in-order to achieve high throughput. The code snippet is as below.
Observable.from(couchbaseRefs).flatMap(id -> bucket.async().get(id)).toList() .toBlocking().single();
Upon analysing the thread dump report we could notice considerable CPU wait within Couchbase API and Rx below breakdown results shows the same. Kindly help in understanding this behaviour, Also we would appreciate with any better performance tuning suggestion within Java SDK or Couchbase setup.
Application uses : Spring Data Couchbase 2.1.2.RELEASE
Couchbase : 4.5.0-2601 Enterprise Edition with 2 nodes