I’m having a strange problem with the speed of data transfer out of Couchbase.
I’ve tried various different configurations but here is my current setup:
5-Node cluster on EC2.
All Ubuntu 13.10 with 8Gb RAM each.
Approx 4m documents in the bucket (not the default bucket)
A View (map/reduce) which returns approx 800,000 records.
When I query the View with reduce=true I get the ‘count’ which is approx 800,000 - this is returned instantly. So this tells me that the View itself is executing extremely quickly.
When I query the View with reduce=false I get each of the 800,000 lines. I’ve done the calculations and the data transfer amounts to around 60 Mb of data.
THIS IS TAKING BETWEEN 20-35 SECONDS.
I’ve installed iperf on all machines and tested the network transfer rate to be anywhere from 100 Mb/sec to 1000 Mb/sec depending on which servers I’m using. This is megabytes per second not megabits. So transferring 60 Mb of data should not be a problem but it is taking about 30 seconds.
This is completely killing my use case.
I’ve independently (not using iperf) confirmed that transferring 60 Mb of a file takes less than a second but when I query the View which returns 60Mb worth of data it takes 30 seconds and the CPU usage of the Node in question is about 60% for the full 30 seconds.
What’s going on ?
Your help would be greatly appreciated - I’m completely stuck with this.