I am testing a 2 node cluster and I’m having problems generating workloads when one server is down but not failed over. Even when executing a connection against the server that is still up and running I am getting an error. Only when I’ve gone to the live server and failed over the down node am I able to successfully connect and send data to couchbase.
- Both servers (node1, node2) are alive and cbworkloadgen is functioning normally
- Shutdown couchbase-server service on node2
- cbworkloadgen fails to connect with the following:
[node1]# /opt/couchbase/bin/tools/cbworkloadgen -n node1:8091 -r .95 -i 100000 -s 100 -t 1 -u Administrator -p password
s0 error: CBSink.connect() for send: error: SASL auth exception: node2:11210, user: default
s0 error: async operation: error: SASL auth exception: node2:11210, user: default on sink: http://node1:8091(default@N/A-0)
error: SASL auth exception: node2:11210, user: default
- Execute failover of node2 on node1 UI
- cbworkloadgen functions normally
Autofailover only works with 3 nodes so obviously I could add another node and hope the autofailover happens fast enough but we don’t have another server allocated yet.
Is this normal behavior when a node goes down?