No, this is not automatic. Here is what should happen when a node crashes, on a cluster where inter-node replication has been set to 1, 2 or 3:
- A node crashes: SDK sees exceptions (most probably timeouts or connection exceptions)
- Someone or something should failover the node (mark it as bad). It can be the ops, an in-house monitoring scripts or (in the case of a first failure and with a minimum delay of 30s) the auto-failover feature baked into couchbase…
- From that point, one of the replicas is promoted and start serving the dataset of the crashed node in place of the crashed node (no more errors in the SDK)
- The node should be repaired or replaced and the cluster rebalanced by the ops so that it is in a homogenous healthy state again (all data is replicated at configured level, all subsets of data are equally replicated, etc…)
The use for getFromReplica() is to manually cover for the window at the beginning where a node crashes but hasn’t yet been failed over (so it still has the full range of replicas, none of them having been promoted). So you catch errors, you fallback to a getFromReplica().
@daschl any gotchas or anything to add?