Q: how to set the ulimited when deploy docker contianer with swarm


#1

hi
I use docker stack depoly couchbase server docker container. I created a docker container from offical container. when use docker run I can assign --ulimit nofile set the container open file max number. but docker stack depoly not support this argument.
I enter couchbase container, use ulimit -a can see the open file max number is 65535, but switch to couchbase user, the number is 1024.
when I deploy the container,it prompt the nofile not allow below 70000.
how can I set ulimit nofile number?

thanks for you reply.

#2

It is true that Docker Swarm doesn’t have a way to set ulimits - it’s a known missing feature that’s been around for quite some time: https://github.com/moby/moby/issues/25209

So what happens is that the ulimits for all containers will fall back to the defaults that dockerd is running with on each host in the swarm. That will vary from system to system, but the default systemd configuration file “docker.service” sets those limits quite high. I just started a one-node Couchbase cluster on a swarm here with a super-trivial stack file:

version: '3.5'

services:
  couchbase:
    deploy:
      replicas: 1
    image: couchbase:6.0.0
    ports:
      - 8091:8091

The server started up just fine. If I “docker exec” into the container, as root at least, I see the ulimit is 1048576. This is with a Docker 18.06 node running on an Ubuntu 18.04 host.

You said you saw a warning about the ulimit being less than 70000 - where did you see that output?


#3

hi @ceej:
thanks for your reply.
I saw a warning about ulimit when running ‘docker logs couchbase’.

The maximum number of open files for the couchbase user is set too low.
It must be at least 70000.

Normally this can be increased by adding the following lines to
/etc/security/limits.conf:

couchbase              soft    nofile                  <value>
couchbase              hard    nofile                  <value>

 Where <value> is greater than 70000. The procedure may be totally
 different if you're running so called "non-root/non-sudo install" or
 if you've built Couchbase Server from source.

I google this issue,hanv’t found resolve.


#4

“docker logs couchbase” wouldn’t be a valid command to run if you’re using Docker Swarm… did you mean “docker service logs couchbase”?

What OS are you running Docker on, and which version of Docker?


#5

hi @ceej:
I use docker swarm before,but I can’t resolve ‘ulimit’ problem.now I use k8s with operator deploy couchbase cluster.but when I run ‘kubectl logs couchbasepod’ ,it still have warning message about ulimit nofile.
my docker version is : 18.06
OS: centos 7|| ubuntu 16.04
when I run container with --ulimit nofile argument, then logs have not warning message.
now I learn how to define the k8s pod deploy script to avoid this warning.


#6

Interesting. I don’t have this problem when launching a Couchbase service on a Docker Swarm on a cluster of Ubuntu 18.04 hosts (also running Docker 18.06.1). I wonder if they changed something regarding ulimits in Ubuntu 18?

I suspect if you changed the default ulimits on the various hosts of your Docker Swarm, you could get past the problem there as well. I’m not sure exactly how one would do that, and there’s unfortunately a lot of conflicting and confusing information about it on the net.

I haven’t used Kubernetes myself, so I can’t suggest anything for that environment.


#7

thanks @ceej. let me try adjust docker host machine settings.I’ll report it. thanks again.