Docker Compose Notes

Russell Bateman
September 2018
last update:

Quick Docker Compose notes in preparation for beginning to make use of this technology.

Docker Compose is a tool for defining and running multi-container Docker applications. You use a YAML file, docker-compose.yml, to configure your application's services. With a single command, docker-compose up, you create and start all the services from your configuration.

Docker Swarm is a tool for a clustering and scheduling of Docker containers. With Swarm, administrators and developers can establish and manage an entire cluster of Docker nodes as a single, virtual system.


Installation

Install docker-compose. Install this Docker aid that makes life easier in terms of Docker command lines. You can see this in the YAML file which obviates the starting of Docker with gazillions of rather lengthy options:

$ wget https://github.com/docker/compose/releases/download/1.22.0/docker-compose-Linux-x86_64
$ chmod a+x docker-compose-Linux-x86_64
# cp docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
$ vim ./docker-compose.yml
elk:
  image: sebp/elk
  ports:
    - "5601:5601"
    - "9200:9200"
    - "9300:9300"
    - "5044:5044"
  ulimits:
    nofile:
      soft: "65536"
      hard: "65536"

Now try to run Docker via docker-compose:

$ docker-compose up elk

This doesn't work complaining that there was no known port. I googled the fool out of this problem (and wondered why I had not encountered it last week when I did this). The solution is ugly and I don't feel super confident that this is even the best way to proceed; however, it works.


Docker-compose YAML for exercise above

At this point, we're looking at Part 3 of the Get Started Docker introductory exercises.

First, install docker-compose:

[email protected]:~/Downloads$ wget https://github.com/docker/compose/releases/download/1.22.0/docker-compose-Linux-x86_64
[email protected]:~/Downloads$ chmod a+x docker-compose-Linux-x86_64
[email protected]:~/Downloads$ sudo cp docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
[email protected]:~/Downloads$ which docker-compose
/usr/local/bin/docker-compose
[email protected]:~/Downloads$ docker-compose --version
docker-compose version 1.22.0, build f46880fe
[email protected]:~/Downloads$ rm docker-compose-Linux-x86_64

Here's the docker-compose YAML file for the contained application we created earlier (see note above).

docker-compose.yml:
version: "3"
services:
  web:
    image: windofkeltia/get-started:part2
    deploy:
      replicas: 5
      resources:
        limits:
          cpus: "0.1"
          memory: 50M
      restart_policy:
        condition: on-failure
    ports:
      - "4000:80"
    networks:
      - acme
networks:
  acme:

Repeating the useful, guiding explanations, this is what the YAML file accomplishes:

  1. Pull the image, the one we uploaded ourselves to the registry, from that registry.
  2. Run 5 instances of that image as a service called web, limiting each one to use, at most, 10% of the CPU (across all cores), and 50MB of RAM.
  3. Immediately restart containers if one fails.
  4. Map port 4000 on the host to web's port 80.
  5. Instruct web's containers to share port 80 via a load-balanced network called acme. (Internally, the containers themselves publish to web's port 80 at an ephemeral port.)
  6. Define the acme network with the default settings (which is a load-balanced overlay network).

Labels on nodes in Docker Swarm

To apply labels to a node (my host node is named moria):

$ docker node update            \
    --label-add acme.ks=true    \
    --label-add acme.nginx=true \
    --label-add acme.elk=true   \
    moria
moria    (reply from Docker)

To remove a label from a node (my host node is named moria):

$ docker node update --label-rm acme.elk=true moria
moria    (reply from Docker)

To see what labels are on node, do this (my host node is named moria):

$ docker node ls -q | xargs docker node inspect \
>   --format '{{ .ID }} [{{ .Description.Hostname }}]: {{ range $k, $v := .Spec.Labels }}{{ $k }}={{ $v }} {{end}}'
zkxs95a465yzp68egf98rf9xj [moria]: acme.deploy=true acme.elk=true acme.ks=true acme.nginx=true

Handling mounted volumes in docker-compose.yml

Imagine docker-compose.yml (or most of it) looking like below. Here we're mostly interested in postgres, but we've included other services too. What interests us is highlighted. Contemplate the section PGDATA at Docker postgres (Docker Official Images) which is an excellent container and place to start.

version: '3.5'
services:
  .
  .
  .
  postgres:
    image: "postgres"
    restart: always
    networks:
      acme:
    volumes:            ("volume-name:path", see below)
      - "postgres-data:/var/lib/postgresql/data"
    environment:
      POSTGRES_DB:       acme
      POSTGRES_USER:     postgres
      POSTGRES_PASSWORD: acme123
    deploy:
      mode: global

  elk:
    image: "sebp/elk:642"
    ports:
      - "5601:5601"
      - "9200:9200"
      - "9300:9300"
      - "5044:5044"
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    networks:
      acme:
    volumes:
      - "elastic-search-data:/var/lib/elasticsearch"
    deploy:
      mode: global
      placement:
        constraints:
          # Only deploy on Docker nodes labeled with acme.elk=true, to wit:
          # $ docker node update --label-add elk=true <nodename>
          - node.labels.acme.elk==true

  filebeat:
    image: "filebeat:6.4.2"
    networks:
      acme:
    volumes:
      - "logs:/opt/logs"
    # Deploy filebeat on _all_ nodes
    deploy:
      mode: global

  .
  .
  .
volumes:
  postgres-data:        (volume-name/what volume is called in this file)
    external: true      (means the volume is created outside of Compose)
    name: postgres-data (what volume name is called in host filesystem)
  logs:
    name: log-data
  elastic-search-data:
    name: elk-data

More notes (and better perhaps than the comments above)...