Internal Couchbase Prometheus memory leak

Couchbase Server
1

I’ve deployed a three node Couchbase 7 cluster. When its is running then the consumed memory keeps increasing day by day. Htop reveales that it is solely consumed by some Prometheus process that seems to run internally within Couchbase.

grafik
grafik1901×265 249 KB

Today it reached a consumption of ~45% of the nodes memory which is ~12.8GB.
After restarting the node and letting it join the cluster again, the prometheus memory consumption is at ~300mb again. Is this some kind of known issue. What could causes it, how can It be prevented?

Total Memory consumption of my nodes during couchbase node restarts and rebalance:

grafik
grafik697×196 8.06 KB

2

Hello @aki

Thank you for using Couchbase and participating in the community.
There was a known memory resource issue with prometheus in Couchbase Server 7.0.0, which was resolved in 7.0.1 and later. https://issues.couchbase.com/browse/MB-47502

The recommendation would be to use a later version of Couchbase Server.

Thanks,
Ian McCloy

1 Like