Unable to scale index scan

We have a system which requires intensive read. The entire data set is small(<500mb) The system only perform a few write every hour. We can scale our system (with separate data, index and query nodes) only up to ~250 N1QL queries / sec. We notice that adding any more query nodes cannot scale up the # of queries/sec. CPU utilization on query nodes are high(close to 90%). However, we also notice that CPU utilization on the index nodes are very low(~15%). We know that our indexes are being utilized(with explain and # of index scan/sec). Any advise is highly appreciated.

which version of Couchbase Server do you use?
And how do you use N1QL?(By SDK or Prepared REST)

I’m using 4.5.1 with java SDK 2.4.1

do you use Global Secondary Indexes or Memory-Optimized Global Indexes?
How the memory usage of Index service?

I use memory-optimized global indexes. memory usage on index node is very low. around 15%

can you post explain of top 5 N1QL which took more time and the index definition?
may be you should turn the debug log on to check which part took more time