My Couchbase, running on Google Compute Engine, became completely unresponsive to Couchbase access and SSH . I restarted the server and Couchbase works again. How can I diagnose this?
At the time, Couchbase was building indexes that were previously created with
I don’t think I was overloading the server. The Dashboard (last I checked) showed plenty of unused memory quota left from the 10 GB RAM; and only 1% of the disk is in use. (There are 2 (2 vCPUs).
The bucket has 1.7 million documents. 185 indexes were created. Almost all of these actually only index a small fractio of the documents, as they have a
WHERE _type= filter. After restart, approx 70 indexes are not yet fully built; and the indexing GUI says “Warning: Cannot communicate with indexer process.”
The logs show “invalid memory address”, but I don’t know if that gets at the root cause.
2019-05-01T11:45:50.623+00:00 WARN CBAS.cbas restarting metakv.RunObserveChildren on /cbas/ due to error: Get http://127.0.0.1:8091/_metakv/cbas/?feed=continuous: dial tcp 127.0.0.1:8091: getsockopt: connection refused 2019-05-01T11:45:50.624+00:00 WARN CBAS.cbas restarting metakv.RunObserveChildren on /cbas/ due to error: Get http://127.0.0.1:8091/_metakv/cbas/?feed=continuous: dial tcp 127.0.0.1:8091: getsockopt: connection refused 2019-05-01T11:45:50.751+00:00 WARN CBAS.cbas TLS config has been refreshed by ns server panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x7f7ea2] goroutine 72010 [running]: main.postTLSConfig(0x0, 0x0) goproj/src/github.com/couchbaselabs/cbas/cbas/security.go:115 +0x422 main.notifyTLSConfigChange() goproj/src/github.com/couchbaselabs/cbas/cbas/security.go:87 +0x62 created by main.registerTLSRefreshCallback.func ...show ns_log 000 firstname.lastname@example.org 11:45:51 AM Wed May 1, 2019 Couchbase Server has started on web port 8091 on node 'email@example.com'. Version: "6.0.1-2037-enterprise". menelaus_sup 001 firstname.lastname@example.org 11:45:50 AM Wed May 1, 2019 Shutting down bucket "fromds" on 'email@example.com' for server shutdown ns_memcached 000 firstname.lastname@example.org 11:45:49 AM Wed May 1, 2019 Shutting down bucket "metricsBucket" on 'email@example.com' for server shutdown ns_memcached 000 firstname.lastname@example.org 11:45:48 AM Wed May 1, 2019``` ![failure|398x500](upload://uV6sDDQigiTD778z00xomrboLup.png)![failure|398x500](upload://uV6sDDQigiTD778z00xomrboLup.png)