Hi CB team ,
Can you help where should I add list of ports (for data/ query / event services ) in Couchbase provided 2.0 Operators Yaml and Cluster Yaml file ?
I don’t see that ports tags available in values.yaml or Charts.yaml either.
Thanks
Any update on this ?
I am new to the CB community but was thinking the community is strong and helpful to address concern and questions and being very active . Please help if you know the answer how I can have 8093 port added to Yaml definition on the couchbase shipped packages for cluster ?
Hi @eldorado
You can expose all the service ports by setting exposedFeatures to client:
exposedFeatures:
- client
1 Like
thanks you @tommie . this is great help and this works great ! Apprec . iate it
I have other challenges with this . I want to spin up K8s PODS with MDS (Multi Dimensional scaling) and want to have 3 POD’s with Data Services , 2 PODs’s with Index and with this generic YAML config how I will define the services and exposed the related POD’s .As -client will load all POD’s with all services. Can you help because looks like you have lot insights on Yaml definitions .
thank again.
@tommie - Hi Tommy ,
may be you are the right person to address my concern . Can you please help here on how to config the values YAML for Multidimensional scaling ?
I have managed to do it here in my custom values YAML :
servers:
- size: 3
name: dataservices
services:
- data
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: indexservices
services:
- index
pod:
resources:
limits:
cpu: "40"
memory: 75Gi
requests:
cpu: "30"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: queryservices
services:
- query
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 2
name: otherservices
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
when I run the installer to install cluster in --dry-run --debug it throws below warning >
2019/12/15 03:12:39 Warning: Merging destination map for chart ‘couchbase-cluster’. Cannot overwrite table item ‘servers’, with non table value: map[all_services:map[pod:map serverGroups: services:[data index query search eventing analytics] size:5]]
REVISION: 1
When I want to continue install it throws below error :
Error: release filled-beetle failed: admission webhook “couchbase-admission-controller-couchbase-admission-controller.default.svc” denied the request: validation failure list:
data in spec.servers[1].services is required
data in spec.servers[2].services is required
data in spec.servers[3].services is required
I am pretty sure I am missing somewhere the loop condition in …\templates\couchbase-cluster.yaml as it is not liking multiple entries for MDS :
And this section seems to be the culprit :
----------------
servers:
{{- range $server, $config := .Values.couchbaseCluster.servers }}
- name: {{ $server }}
{{ toYaml $config | indent 4 }}
{{- end }}
{{- if .Values.couchbaseTLS.create }}
----------------
Any clue what I am missing to get my MDS installation goes well ?
thanks for your help.
@deniswsrosa - Hi Dennis , I am reaching you to get some help on K8s setup for MDS .
I saw your post on : https://blog.couchbase.com/databases-on-kubernetes/#comment-19547
and Tried below settings too in CB 6 EE version : Not working . Am I missing anything .
Sorry I am lost because I didn’t find a comprehensive Couchbase documentation to get the details :
servers:
- size: 3
name: data
services:
- data
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: index
services:
- index
pod:
resources:
limits:
cpu: "50"
memory: 75Gi
requests:
cpu: "40"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: query
services:
- query
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 2
name: others
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
With Helm installation I am getting issue as ;
helm install couchbase-cluster -f couchbase-cluster/tc_values_rate_ilcb.yaml --tiller-namespace=bi-cb
Error: release original-dog failed: admission webhook “couchbase-admission-controller-couchbase-admission-controller.default.svc” denied the request: validation failure list:
data in spec.servers[1].services is required
data in spec.servers[2].services is required
data in spec.servers[3].services is required
Hi @eldorado
When using helm for MDS, the server spec is expressed as a map where each key is the name of the config.
For instance, the overrides you are attempted to create should be expressed as:
couchbaseCluster:
servers:
dataservices:
size: 3
services:
- data
indexservices:
size: 1
services:
- index
all_services: null
Note also all_services is set to null so that it is removed from spec.
By expressing this as a map scaling becomes easier for you with MDS. ie
helm upgrade --set couchbaseCluster.indexservices.size=2 <chart_name>
rather than guessing indexes.
This is great to know … @tommie I am going this out and let you know …
for 2nd part with helm upgrade are you saying its another way of override existing cluster to change index node sizes rather than editing the cbc ?
thanks
Hi @tommie - Here is what exactly I did and out of luck …
servers:
all_services: null
dataservices:
size: 3
services:
- data
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
indexservices:
size: 1
services:
- index
pod:
resources:
limits:
cpu: "50"
memory: 75Gi
requests:
cpu: "40"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
queryservices:
size: 1
services:
- query
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
otherservices:
size: 2
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
I have tried moving all_services: null at bottom as well no luck .
When I run this with my helm chart it throws below error : (Note that the couchbase-cluster.yaml shipped with helm repo has not been changed ) . It seems still there is mapping issue with …/templates/couchbase-cluster.yaml file .
helm install couchbase-cluster -f couchbase-cluster/tc_values_rate_ilcb.yaml --tiller-namespace=bi-cb --dry-run --debug
[debug] Created tunnel using local port: ‘36064’
[debug] SERVER: “127.0.0.1:36064”
[debug] Original chart version: “”
[debug] CHART PATH: ~/couchbase/couchbase-cluster
Error: failed to parse couchbase-cluster/tc_values_rate_ilcb.yaml: error converting YAML to JSON: yaml: line 22: mapping values are not allowed in this context
@tommie - Any help directing to the right Path . Without clear documentation and direction I am kind of blocked and stuck on MDS setup . Can you help ? Sorry to bother you .
@arungupta @tommie @deniswsrosa
Can you please help me to write the MDS parameters as I don’t see anyways to get around my cluster config to work properly in K8s .
thanks
Looks like you have an indentation problem around line 22
error converting YAML to JSON: yaml: line 22
ref -> https://github.com/kubernetes/kubernetes/issues/30961#issuecomment-439194422
Check out the quickstart guide for specifying cluster overrides, you will find the section on persistent volumes helpful. There isn’t anything special about MDS besides adding an additional server map to the yaml.
@tommie - Maybe I have overwhelmed you with information :
can you just get me this working ?
servers:
- size: 3
name: data
services:
- data
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: index
services:
- index
pod:
resources:
limits:
cpu: "50"
memory: 75Gi
requests:
cpu: "40"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 1
name: query
services:
- query
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
- size: 2
name: others
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
There is no syntax error in above yaml but below :
when I run the installer to install cluster in --dry-run --debug it throws below warning >
2019/12/15 03:12:39 Warning: Merging destination map for chart ‘couchbase-cluster’. Cannot overwrite table item ‘servers’, with non table value: map[all_services:map[pod:map serverGroups: services:[data index query search eventing analytics] size:5]]
REVISION: 1
When I want to continue install it throws below error :
Error: release filled-beetle failed: admission webhook “couchbase-admission-controller-couchbase-admission-controller.default.svc” denied the request: validation failure list:
data in spec.servers[1].services is required
data in spec.servers[2].services is required
data in spec.servers[3].services is required
This seems to be a COMPATIBILITY ISSUE with MDS and obviously with couchbase-cluster.yaml … I have listed this as below how helm ship with this yaml file inside templates:
apiVersion: "couchbase.com/v1"
kind: "CouchbaseCluster"
metadata:
name: {{ template "couchbase-cluster.clustername" . }}
spec:
baseImage: {{ .Values.couchbaseCluster.baseImage }}
version: {{ .Values.couchbaseCluster.version }}
authSecret: {{ template "couchbase-cluster.secret.name" . }}
exposeAdminConsole: {{ .Values.couchbaseCluster.exposeAdminConsole }}
adminConsoleServices: {{ .Values.couchbaseCluster.adminConsoleServices }}
exposedFeatures: {{ .Values.couchbaseCluster.exposedFeatures }}
adminConsoleServiceType: {{ .Values.couchbaseCluster.adminConsoleServiceType }}
exposedFeatureServiceType: {{ .Values.couchbaseCluster.exposedFeatureServiceType }}
logRetentionTime: {{ .Values.couchbaseCluster.logRetentionTime }}
logRetentionCount: {{ .Values.couchbaseCluster.logRetentionCount }}
disableBucketManagement: {{ .Values.couchbaseCluster.disableBucketManagement }}
{{- if .Values.couchbaseCluster.dns }}
dns:
domain: {{ .Values.couchbaseCluster.dns.domain }}
{{- end }}
{{- if .Values.couchbaseCluster.platform }}
platform: {{ .Values.couchbaseCluster.platform }}
{{- end }}
cluster:
{{ toYaml .Values.couchbaseCluster.cluster | indent 4 }}
buckets:
{{- range $bucket, $config := .Values.couchbaseCluster.buckets }}
-
{{ toYaml $config | indent 4 }}
{{- end }}
servers:
{{- range $server, $config := .Values.couchbaseCluster.servers }}
- name: {{ $server }}
{{ toYaml $config | indent 4 }}
{{- end }}
{{- if .Values.couchbaseTLS.create }}
tls:
static:
member:
serverSecret: {{ template "couchbase-cluster.secret.tls-server" . }}
operatorSecret: {{ template "couchbase-cluster.secret.tls-operator" . }}
{{- end }}
securityContext:
{{ toYaml .Values.couchbaseCluster.securityContext | indent 4 }}
volumeClaimTemplates:
{{ toYaml .Values.couchbaseCluster.volumeClaimTemplates | indent 4 }}
The link you have shared is not useful because all its talking about all_services and I am not interested that on MDS context . sorry
thanks
The indentation here is causing error
size: 3
services:
- data
should be
size: 3
services:
- data
Here is the most concise way to express what you’re trying to do
couchbaseCluster:
servers:
dataservices:
size: 3
services:
- data
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
indexservices:
size: 1
services:
- index
pod:
resources:
limits:
cpu: "50"
memory: 75Gi
requests:
cpu: "40"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
queryservices:
size: 1
services:
- query
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
all_services:
size: 2
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
Note I replaced otherservices with all_services because the key cannot be removed but only be re-used. In the future it will simply be named ‘default.’
@tommie
Thanks and I exactly set to what you have suggested and getting below error :
helm install couchbase-cluster -f couchbase-cluster/tc_values_rate_ilcb_old.yaml --tiller-namespace=bi-cb
Error: release measly-kudu failed: admission webhook “couchbase-admission-controller-couchbase-admission-controller.default.svc” denied the request: validation failure list:
data in spec.servers[0].services is required
data in spec.servers[2].services is required
data in spec.servers[3].services is required
Debug mode has no error but this is how it is interpreting from cluster yaml ::
# Source: couchbase-cluster/templates/couchbase-cluster.yaml
apiVersion: "couchbase.com/v1"
kind: "CouchbaseCluster"
metadata:
name: tccbldc-cluster
spec:
baseImage: couchbase/server
version: enterprise-6.0.1
authSecret: olfactory-jaguar-tccbldc-cluster
exposeAdminConsole: true
adminConsoleServices: [data]
exposedFeatures: [client]
adminConsoleServiceType: NodePort
exposedFeatureServiceType: NodePort
logRetentionTime: 500000s
logRetentionCount: 20
disableBucketManagement: true
cluster:
analyticsServiceMemoryQuota: 4096
autoFailoverMaxCount: 3
autoFailoverOnDataDiskIssues: true
autoFailoverOnDataDiskIssuesTimePeriod: 120
autoFailoverServerGroup: false
autoFailoverTimeout: 120
dataServiceMemoryQuota: 20480
eventingServiceMemoryQuota: 1024
indexServiceMemoryQuota: 51200
indexStorageSetting: plasma
searchServiceMemoryQuota: 2048
buckets:
-
compressionMode: passive
conflictResolution: seqno
enableFlush: true
enableIndexReplica: false
evictionPolicy: fullEviction
ioPriority: high
memoryQuota: 128
name: default
replicas: 1
type: couchbase
servers:
- name: all_services
pod:
resources:
limits:
cpu: "5"
memory: 10Gi
requests:
cpu: "2"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
serverGroups: []
services:
- search
- eventing
- analytics
size: 2
- name: dataservices
pod:
resources:
limits:
cpu: "10"
memory: 30Gi
requests:
cpu: "5"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
services:
- data
size: 3
- name: indexservices
pod:
resources:
limits:
cpu: "50"
memory: 75Gi
requests:
cpu: "40"
memory: 50Gi
volumeMounts:
data: couchbase
default: couchbase
services:
- index
size: 1
- name: queryservices
pod:
resources:
limits:
cpu: "10"
memory: 10Gi
requests:
cpu: "5"
memory: 5Gi
volumeMounts:
data: couchbase
default: couchbase
services:
- query
size: 1
tls:
static:
member:
serverSecret: olfactory-jaguar-tccbldc-cluster-server-tls
operatorSecret: olfactory-jaguar-tccbldc-cluster-operator-tls
securityContext:
fsGroup: 1000
volumeClaimTemplates:
- metadata:
name: couchbase
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 250Gi
storageClassName: pure-block
Would you mind to send couchbase-cluster.yaml from your end if you have that handy … I am not sure if Couchbase recommends to change this YAML .
thanks
Glad that helped resolved indentation problem.
Since I’m not actually running the cluster here I suggest going through docs as a way to double check syntax and values as what I’m providing is more for example.
In this specific case, when using persistent volumes you can only mount paths that correspond to services used. Which means
size: 1
services:
- index
Must be
volumeMounts:
index: couchbase
default: couchbase
Note the use of index instead of data for the mount.
Mounts are provided for data/index/analytics services (not eventing, query). See
https://docs.couchbase.com/operator/current/couchbase-cluster-config.html#default
Also note analytics is multi-disk which uses array syntax as seen in the cluster example at top of docs page. ie…
volumeMounts:
analytics:
- couchbase
default: couchbase
1 Like
@tommie - Wow ! this is final config which works … Listing down here for others benefit if required .
Do you see any issue there … Sorry I want to isolate query service but lookslike it needs VolumeMounts otherwise throw errors …Can this still be null ?
thanks for your help for finally crack it down …
servers:
dataservices:
size: 3
services:
- data
pod:
resources:
limits:
cpu: "4"
memory: 30Gi
requests:
cpu: "2"
memory: 20Gi
volumeMounts:
data: couchbase
default: couchbase
indexservices:
size: 1
services:
- index
pod:
resources:
limits:
cpu: "5"
memory: 75Gi
requests:
cpu: "1"
memory: 50Gi
volumeMounts:
index: couchbase
default: couchbase
queryservices:
size: 1
services:
- query
pod:
resources:
limits:
cpu: "1"
memory: 5Gi
requests:
cpu: "1"
memory: 2Gi
volumeMounts:
default: couchbase
all_services:
size: 2
services:
- search
- eventing
- analytics
pod:
resources:
limits:
cpu: "2"
memory: 10Gi
requests:
cpu: "1"
memory: 5Gi
volumeMounts:
default: couchbase@tommie - Little challenge here with Query services :
I have one POD running just with query service and getting below error when click on “Query” in UI : Is that expected ? or I should be able to access query from same Admin UI interface :
The query interface is only available on Couchbase nodes running the query service.
The query service was found on the following nodes:
http://cluster-0007.-cluster.bi-cb.svc:8091/ui/index.html#!/query/workbench
Note that the above links are provided as a convenience. They may not work if you are accessing the Web Console via a web proxy or if Couchbase Server has been deployed in a split horizon DNS environment, with public and private hostnames and IPs. In the event the links don’t work, log directly into the node.
queryservices:
size: 1
services:
- query
pod:
resources:
limits:
cpu: "1"
memory: 5Gi
requests:
cpu: "1"
memory: 2Gi@tommie - Also when I tried to edit the cbc in commandline and remove query services from yaml its saving this w/o error . But as soon as I add : - query to be part of my all_services after - analytics I get below error :
servers:
- name: all_services
pod:
resources:
limits:
cpu: "2"
memory: 10Gi
requests:
cpu: "1"
memory: 5Gi
volumeMounts:
default: couchbase
services:
- search
- eventing
- analytics
- query
error: couchbaseclusters.couchbase.com "cb-cluster" could not be patched: admission webhook "couchbase-admission-controller-couchbase-admission-controller.default.svc" denied the request: validation failure list:
spec.servers[0].services in body cannot be updated
You can run `kubectl replace -f /tmp/kubectl-edit-p251n.yaml` to try this update again.Yes this is correct, Couchbase server does not allow services to be enabled/disabled online.
As a work around until 2.0, you can add/remove services and change the server class name (spec.servers[].name) and it will work as you intend.