How to create preconfigured couchbase docker image with data


#1

Hi,
For integration tests I’d like to run pre-configured Couchbase server (community edition 5.0.1) docker container.
One way is to run init script to start server, services, buckets and indexes (see below) and then run another docker to fill in data or use cbrestore tool to restore data. This takes some time.
I tried to configure the server and create buckets and data and then create docker image from this container but when running container from the new image I’m always getting uninitialized Couchbase server.

Question:

Any ideas how to create docker image with pre-configured, ready to use Couchbase server without running init scripts (again the init process takes time and I need this server to be up and running with data immediately)?

Scripts I’m currently using to init container:

(This script plus the process to fill data takes about 4 minutes)

Dockerfile:

FROM couchbase/server:community-5.0.1

COPY configure.sh /configure.sh
CMD ["/configure.sh"]

configure.sh:

#!/bin/bash

# Enables job control
set -m

# Enables error propagation
set -e

# Run the server and send it to the background
/entrypoint.sh couchbase-server &

# Check if couchbase server is up
check_db() {
  curl --silent http://127.0.0.1:8091/pools > /dev/null
  echo $?
}

# Variable used in echo
i=1
# Echo with
log() {
  echo "[$i] [$(date +"%T")] $@"
  i=`expr $i + 1`
}

# Wait until it's ready
until [[ $(check_db) = 0 ]]; do
  >&2 log "Waiting for Couchbase Server to be available ..."
  sleep 1
done

# Setup index and memory quota
log "$(date +"%T") Init cluster ........."
couchbase-cli cluster-init -c 127.0.0.1 --cluster-username Administrator --cluster-password password \
  --cluster-name myCluster --cluster-ramsize 1024 --cluster-index-ramsize 512 --services data,index,query,fts \
  --index-storage-setting default

# Create the buckets
log "$(date +"%T") Create buckets ........."
couchbase-cli bucket-create -c 127.0.0.1 --username Administrator --password password --bucket-type couchbase --bucket-ramsize 250 --bucket bucket1
couchbase-cli bucket-create -c 127.0.0.1 --username Administrator --password password --bucket-type couchbase --bucket-ramsize 250 --bucket bucket2

# Create user
log "$(date +"%T") Create users ........."
couchbase-cli user-manage -c 127.0.0.1:8091 -u Administrator -p password --set --rbac-username sysadmin --rbac-password password \
 --rbac-name "sysadmin" --roles admin --auth-domain local

couchbase-cli user-manage -c 127.0.0.1:8091 -u Administrator -p password --set --rbac-username admin --rbac-password password \
 --rbac-name "admin" --roles bucket_full_access[*] --auth-domain local

# Need to wait until query service is ready to process N1QL queries
log "$(date +"%T") Waiting ........."
sleep 20

# Create bucket1 indexes
echo "$(date +"%T") Create bucket1 indexes ........."
cbq -u Administrator -p password -s "CREATE PRIMARY INDEX idx_primary ON \`bucket1\`;"
cbq -u Administrator -p password -s "CREATE INDEX idx_type ON \`bucket1\`(_type);"
cbq -u Administrator -p password -s "CREATE INDEX idx_account_id ON \`bucket1\`(id) WHERE (_type = \"account\");"

# Create bucket2 indexes
echo "$(date +"%T") Create bucket2 indexes ........."
cbq -u Administrator -p password -s "CREATE PRIMARY INDEX idx_primary ON \`bucket1\`;"
cbq -u Administrator -p password -s "CREATE INDEX idx_event_ruleId ON \`bucket1\`(ruleId) WHERE (_type = \"event\");"
cbq -u Administrator -p password -s "CREATE INDEX idx_event_startTime ON \`bucket1\`(startTime) WHERE (_type = \"event\");"

fg 1


#2

Hey Motty,

I’ve spent my day mucking about with this. It’s not the configuration that takes the time, it literally takes ~11 seconds for the couchbase server to start and become available, even if it’s already configured. I don’t know exactly why, but my best wild guess is that it’s waiting in an attempt to autodiscover peers before it declares itself the master.

Below is a link to a nearly complete CRUD app with scripts that configure a docker image with the couchbase server preconfigured, including with a bucket and a user. The CRUD app still needs to loop on connecting to the db until the server is up.


#3

I moved the preconfig to a branch

Waiting for the ~11s twice (once for config and once for startup) was annoying.