Feature Request: Deregister inactive service definitions

Elasticsearch Connector
1

Hi,

this is a request for the consul connector. When working with multiple instances, lots and lots of duplicate service definitions are created and they remain in the consul forever. You will have this problem when you’re working with multiple nodes like in Kubernetes. I’m assuming you’re not deregistering them because somebody would like to see the failing ones. But I think there are better ways of checking the state of connectors like using Kubernetes pod metrics or CBES’s own metrics.

Other than that, if you have any other thoughts of not deregistering inactive definitions, I would be happy to hear.

Thank you

2

Hi Emre,

Thanks for this feature request, which we’re tracking as CBES-240.

Version 4.3.4 of the connector added native Kubernetes integration; since you’re deploying to Kubernetes, you might not need Consul at all. If you try it, we’d appreciate any feedback.

Thanks,
David

1 Like
3

Hey David,

we’re using CBES in multiple clusters. I took a look at the native Kubernetes integration and I understood it works for a single cluster. If it works for multiple clusters, I would analyze it deeply and may use it.

Thank you

4

Hi Emre,

Could you explain the requirements in more detail, please?

With the native Kubernetes integration, each Elasticsearch connector group is deployed as a StatefulSet. Each group has its own connector config; to connect to multiple Couchbase clusters, deploy multiple groups. The number of connector groups is limited only by the Kubernetes cluster resources.

Thanks,
David

5

Hey David,

what I mean by multiple clusters is kubernetes clusters. There is only a single connector group we use, but we run it in multiple kubernetes clusters.

1 Like
6

Hi Emre,

Thanks for clarifying. The native Kubernetes integration does indeed require all members of the connector group to run in the same k8s cluster.

Thanks,
David

1 Like
7

I should also say that feature requests made through the Couchbase Forum are handled on a “best effort” basis. If your company has a Couchbase Enterprise Subscription License Agreement, please follow up with our support organization (mentioning this forum post and CBES-240) to ensure this gets triaged appropriately.

1 Like
8

Version 4.4.1 of the connector is now available, with two new config options in consul.toml:

  • deregisterServiceOnGracefulShutdown - if true, the last thing the connector does during a graceful shutdown is deregister the corresponding Consul service. (default true)

  • deregisterCriticalServiceAfter - Time after which a failed connector service is automatically deregistered from Consul. (default 7 days)

Thanks,
David

1 Like
9

Very happy to hear that. Great job.