GSI Indexing never completes. Index has negative fragmentation

40-preview

#1

When creating a GSI index on a dataset of 3million documents the indexer process seems to use all available memory on the server and then restart. The memory used by the indexer seems to higher than then limit set in the node config. The index fragmentation for the index appears in the metrics as a negative value.

Indexer memory size set to 500m

From top

14586 couchbas 20 0 3495m 2.2g 3760 S 50.2 59.4 3:17.20 indexer

Shows resident memory as 2.2G


#2

Thanks for the feedback. obviously developer preview has still a few issues. There is a known issue where if you set the index ram quota larger than what’s available on the node, we may not be able to start the indexer on the node. I recommend using the following guidelines for now until we tune the product for compactness: http://docs.couchbase.com/4.0/getting-started/resource-reqs-dp.html

There is a known issue on the index fragmentation. It looks similar to this: https://issues.couchbase.com/browse/MB-14689 but I noted your condition as well there.
thanks
-cihan