We met with a strange issue in our production environment while trying to create a new index which ended up in multiple nodes going down and instance becoming inaccessible through ssh.
3 node clustered environment
Number of buckets : 7
Out of which 6 buckets are having only few data - less 50K
1 bucket with more than 2M data and having 6 indexes on it.
We tried to build a new index on the same bucket and within minutes a node went down and subsequently instance became inaccessible.
Iam attaching cb_collectinfo dump here.
After 4-5 hours and multiple start-stop of instance everything became normal.
Also we tried to remove that new index in between.
Please help us finding any clue whats happening and how we can avoid the same in future.
Any help would be greatly appreciated.