I ran into a rather odd issue this morning.
We have an internal testing environment for couchbase of 1 cluster containing 3 server nodes. This morning when I checked out the couchbase console, 1 node was stuck on pending and another was down. I checked the two machines in question and they were doing fine. So I checked the logs and all it mentioned was something like: ‘Shutting down bucket “XXXXX” on
’email@example.com’ for server shutdown’. So I failed that node over and immediatly the pending node came back up and running perfectly.
I figured it might have been something like a scheduled task that hogged too much memory and had the ‘down’ node crash or something so I just went ahead and rebalanced the cluster and then tried to re-add that 3rd node but all I’m getting is an:
Attention - Prepare join failed. Could not connect to “xxx.xxx.xxx.xxx” on port 8091. This could be due to an incorrect host/port combination or a firewall in place between the servers.
All firewalls are deactivated on all 3 machines internally to avoid those issues - this has never happened before. I’m unsure if I should try uninstalling couchbase and then reinstalling it instead.
Any help is appreciated.