Cluster become Un responsive


#1

Hey Guys,
We currently have a 4 nodes cluster holding about ~337M documents, with average of: 100 sets per seconds and get rate of 3000-4000 per seconds.
We experience some issue that causes the cluster (or some nodes in the cluster) to get un responsive each time compaction is being executed.
We set compaction to run once a day in our “off-time” (low traffic hours) - and during the process we get alerts that nodes are not responsive (we have checks for each node separately)

We tried some solution - including running the compaction more frequently - but with not help.

Are you guys aware of this issue ? or is there something we are doing wrong ?

We are running on Amazon Cloud on r3.2xlarge machines, Server version 2.2.0 with replication factor of 1.

Thanks,
Idan.