Couchbase connector high memory utilization

hello,
couchbase version: 6.5.0, community edition
couchbase to elasticsearch connector version: 4.2.2
elasticsearch version: 7.6

whenever the couchbase connector sends updates to elasticsearch, it causes high memory and cpu utilization on couchbase server side.

would you please provide me with solution?
possibilities:
1- should i define special indexes in my buckets to prevent the occurrence of such issue?
2- should the couchbase connector be installed in a different machine?

thank you in advance

Hello @ehsnsbn welcome to the group.

Your question depends on a lot of factors , including the environment setup , example Distribution mode you are using.
Ops / sec, general machine configuration, what operations are you performing etc.

hello dear Arun,
thank you very much for your reply, and appreciate your kindness,

regarding the Ops/Sec:
peaks at 4500-5000 ops/sec.

general operations from backend server:
mostly do n1ql queries from nodejs and java backend.

connector configuration:
autonomous operation mode + hashicorp consul, both are locally installed on the db server and the cosul(s) are clustered.

the problem is that, when i shutdown the connector, couchbase server works like a charm.
just some quick questions that i wonder if you can lead me to find the correct answer for:
1- apart from os tuning, and sizing guidelines, should i change the connector configuration into solo mode?
2- the statement - that i should define some indexes to speed up connector querries to couchbase - is irrelative? if not, what indexes should i define?

thank you very much! :slight_smile:

hello @ehsnsbn, thanks for your kind words

1 - solo mode as the documentation recommends is strictly for development / experimental only.

What I would think is for you to install connectors on a different machine in a distributed way.

Connectors processes mutations, so I am not sure if well defined indexes is your problem.
Regardless GSI does help in the sense if you are running operations that are on non-indexed fields.

Can you try installing them on a different machine (standalone for the connectors ? ) and see if that helps