Trouble creating index


#1

We have problems with creating index.
Create Index command with “defer_build” timed out, but Index still appears in Index UI section with status Created and progress at 0%. Trying to start Build or Drop the index returns Index not found response, while other attempts to Create Index results in “Index already exists”.

Index does not appear in index list returned by query.

Failovering the node with faulty index did not help.

Server version is: 4.1.0-5005 Enterprise Edition (build-5005).

Is there any other additional information we should provide?


#2

We have the same problems. The only workaround we found was removing the node from the cluster, then readding it and recreating all indexes from scratch. Needless to say this is not acceptable with big databases. At the moment our database contains 20 million documents and creating indexes takes several days.


#3

This sounds like a bug. Do you have your indexer.log file? could you send that to cihan@couchbase.com. I’ll take a look.
thanks
-cihan


#4

@p_lab_adform, I got the log. I see a number of issues initializing the index service.

2016-03-22T17:01:09.485+01:00 [Error] ClustMgrAgent::NewClustMgrAgent Error In Init Invalid service
2016-03-22T17:01:09.485+01:00 [Fatal] Indexer::NewIndexer ClusterMgrAgent Init Error%!(EXTRA *indexer.MsgError=Invalid service)
2016-03-22T17:01:09.485+01:00 [Warn] Indexer Failure to Init Invalid service

this typically happen when someone else grabs the ports indexer needs on the machines. do you have any of the following ports occupied by another process or a virus scanner that may be preventing indexer from grabbing them on the nodes? -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105
thanks
-cihan


#5

Those ports are available for indexer to use. One of the things I tried was removing the problematic index from @2i folder but it would reappear again after service restart.

Although the indexer is running on all three nodes, we are still seeing the error message: “Warning: We are having troubles communicating to the indexer process. The information might be stale.”


#6

This certainly isn’t usual and we are not seeing these issues on our end. I have both indexer.logs and I’ll take a deeper look at dev and test env you sent.
thanks
-cihan