We are using Community Edition 6.0.0 build 1693 ‧ IPv4.
We have a large number of documents about 23 million of them
Below mentioned is an index that we have created
Here is the n1ql which we are using
SELECT APID,LastUpdate FROM
apbucket WHERE segment IN [“ABC”] AND status=“ACT” AND LastUpdate BETWEEN
“1970-01-01T00:00Z[UTC]” AND “2020-06-15T15:54:42.359Z[UTC]” OFFSET 12000000 LIMIT 1000
Issue that we have is : we see a severe drop in performance and also timeouts as we increase the Offset value.
Query performs well with smaller values
As we increase the offset value to 1,100,1000,10000 ,100000,1000000,10000000 we see a drop in performance and timeouts.
We have seen an suggestion listed in this blog about use of meta().id for these kind of scenarios.
we would like our consumers to make multiple calls to fetch this information in parallel and having stateful transactions to have context of meta().id maintained is not really a choice that we were willing to consider
Are there any other possibilities for solving this issue with performance.