Error 5000 while running query after indexing service crash

Couchbase Server
1

Our Couchbase server 4.5.1 sometimes stops working due to index service not responding. Most times restart server restores its behavior, sometimes we need to drop and rebuild some index to have our apps to work again. Today, the restore procedure we usually follow was not enough to restore server operation. Running a query gives the following error:

{
    "code": 5000,
    "msg": "WSARecv tcp 127.0.0.1:51752: An existing connection was forcibly closed by the remote host. - cause: WSARecv tcp 127.0.0.1:51752: An existing connection was forcibly closed by the remote host."
  }

In server’s logs I find this error when query fails:

Service 'indexer' exited with status 1. Restarting. Messages:    0fd0   31 36 34 34 30 5d 2c 5b  31 35 33 35 30 2c 31 35   16440],[15350,15
   0fe0   33 35 30 5d 2c 5b 31 36  32 31 35 2c 31 36 32 31   350],[16215,1621
   0ff0   00 00 00 00 00 00 84 d5  00 07 00 00 00 00 00 dd   ................
assertion failed [rsize && rsize <= trie->chunksize] at ..\forestdb\src\hbtrie.cc:84 (0000000000000000 != 00000000161734E0)
[goport] 2019/04/18 19:21:20 d:/FF3D/Couchbase/bin/indexer.exe terminated: exit status 3

Any suggestion how to fix this problem?

2

@paolo.morgano, The assertion is coming from forestdb storage engine. It would be worthwhile to upgrade to latest version as some of these problems have been addressed and switch to circular compaction. I am assuming you are on CE version.

3

Explaining query execution plan I found that two indexes are involved: one of them is the primary index, dropping and rebuilding it fixed the query problem.

Might it be a bug in the indexer of my server version? Might be that low server resources (high cpu load, shared disk with OS and other programs) causes index corruption?

4

You are right, we are running CE version. Wich version do you recommend to upgrade to?

5

I think the latest one should be a better choice - CE 6.0.

6

Thanks @deepkaran.salooja for the suggestion, I’ll consider to switch to the 6.0 CE. Is the upgrade straight forward or I have to take in account to migrate the SDK & sync gateway version too?

EDIT: this link contains the answers to my questions