Not sure how much detail is required but the thumbnail sketch is that reaching out from a pod to couchbase from an Azure AKS cluster with CNI enabled results in the specific pod IP being DNSMASQ’d to the node IP when the request travels out of the VNet (even if it is to services in a peered VNet). For whatever reason (and I call this an Azure AKS issue - apparently I cannot apply a configmap that would alter this behavior), this means that when I have a couchbase cluster residing outside of the VNet, requests reach the cluster, but a response never makes it back to the microservice. We have worked around this issue in our dev environments my making sure our couchbase environment is on the same VNet. However, in our high availability configuration, with redundant AKS clusters in different regions on separate VNets accessing a high-availability couchbase cluster spread across 3 regions, we cannot run this way.
Finally to my question! Is there any way to place HAProxy between our microservice and our couchbase cluster? If I can pull this off, that means I can place an HAProxy server on the same vnet as each AKS cluster and it can forward requests coming from specific pod IPs and route responses back to that same pod IP. Hoping this makes sense and/or that someone has experience with this and can help us out. Thanks!
Sincerely,
Lee Roder