Dockerize your IT

(1)

HAL Id: cel-01998899

https://hal.archives-ouvertes.fr/cel-01998899

Submitted on 29 Jan 2019

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Dockerize your IT

Yoann Juet

To cite this version:

(2)

Dockerize Your IT !

Centrale Nantes

Information Technology Department

Yoann Juet

(3)

A Brief History of Containers

1979

2000

2001

2002

2006

2008

UNIX CHROOT BSD JAIL LINUX VSERVER LINUX NAMESPACES

2005

LINUX OPENVZ LINUX CGROUPS LINUX LXC

2013

DOCKER

2015

KUBERNETES

(4)

Docker Containers != Virtual Machines

3

VMs have a full copy of an OS, leading

to cpu, memory, network overhead ;

very good isolation between VMs

Containers share host ressources (kernel, cpu,

memory, network) ; good isolation between

Containers

(5)

Docker architecture

● docker : the Docker user CLI ● dockerd : engine daemon

○ Create image, pass it to containerd

● containerd : runtime daemon ○ Core container runtime for

Docker

○ Manage the complete container lifecycle (stop, start, transfer, supervision, storage, network)

● This model gives the ability to restart or upgrade Docker Engine without breaking the running containers

(6)

Docker images,

An image is a

read-only

group of

layers of other

images

. It includes everything an application needs

to run: binaries, libraries, config files…

Each image is made of a

base image

(e.g. debian,

ubuntu, alpine) plus a collection of diffs -

intermediate images/layers

- that adds the required

features (e.g. emacs, apache).

Images can be stored on

public, private repos

, on

any host machine that has previously pulled the

package from a repo

(7)

Docker Containers

A container is a

running instance

- read-write - of

an image

You can run containers on Linux, Windows 10,

Windows Server 2016, Cloud (AWS, Google…)

Containers should be as ephemeral as possible.

You should expect them to go down at any time

and lose all data stored inside:

● Don’t store data in containers

● Don’t run more than one process in a single

container

● Use custom created volumes or system

mounts

(8)

Docker Image != Docker Container

App Image App 1 Container Public or Private Registry

Remember…

● A Docker Image is similar to a

read-only template

● A Docker Container is an

writable instance

of a

Docker Image

● Each Docker Container has its own read-write

layer -

thus its own data

- that sits on one or more

Docker image

App 2 Container pull r/w Instance Commit r/w Instance push

(9)

Lightweight, low resources consumptions

Isolation model, default set of capabilities

Low attack surface

Networking can be tricky

Fast boot, removal, reproducibility : the same code runs everywhere

Orchestration complexity

Large ecosystem - lot of official and unofficial images -

Ideal for development team (test, pre-prod, prod)

(10)

Docker repository (version 18.09)

user@host:~$ sudo apt-get update &&

apt-get install apt-transport-https ca-certificates curl

gnupg2 software-properties-common

user@host:~$ sudo curl fsSL https://download.docker.com/linux/debian/gpg | aptkey add -user@host:~$ sudo apt-key fingerprint 0EBFCD88

pub rsa4096 2017-02-22 [SCEA]

9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88

uid [ unknown] Docker Release (CE deb) <[email protected]> sub rsa4096 2017-02-22 [S]

user@host:~$ sudo add-apt-repository "deb https://download.docker.com/linux/debian stretch stable" user@host:~$ sudo apt-get update && apt-get install docker.ce

user@host:~$ sudo usermod -aG docker <user> && newgrp docker

Installing Docker Community Edition (CE)

(11)

-Check for correct installation

user@host:~$ docker info ...

Server Version: 18.09.0 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true ...

Runtimes: runc

Default Runtime: runc

Kernel Version: 4.9.0-8-amd64

Operating System: Debian GNU/Linux 9 (stretch) ...

Docker Root Dir: /var/lib/docker ...

Product License: Community Engine

Installing

Docker Community Edition (CE)

(12)

-Search for an official nginx image...

user@host:~$ docker search nginx

NAME DESCRIPTION STARS OFFICIAL AUTOMATED nginx Official build of Nginx. 10386 [OK] jwilder/nginx-proxy Automated Nginx reverse proxy for docker con… 1473 [OK] richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of… 652 [OK] ...

Download the image from the official Docker Registry

user@host:~$ docker pull nginx Using default tag: latest

latest: Pulling from library/nginx a5a6f2f73cd8: Pull complete

1ba02017c4b2: Pull complete 33b176c904de: Pull complete

Digest: sha256:5d32f60db294b5deb55d078cd4feb410ad88e6fe77500c87d3970eca97f54dba Status: Downloaded newer image for nginx:latest

Run

Your First Docker Application

Let’s say an nginx http server !

In the OFFICIAL column, OK indicates an image built by the

(13)

Verify

user@host:~$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 568c4670fa80 46 hours ago 109MB

Run a container based on this image

user@host:~$ mkdir ~/mydir && echo "Hello, France!" > ~/mydir/index.html && docker run --name myweb -v ~/mydir:/usr/share/nginx/html -p 8080:80 -d nginx

8807f82f280b6dde9848029c414b22b9ac7b77362b39d7c58c94e15d2eedc905 user@host:~$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

8807f82f280b nginx "nginx -g…" 40 seconds ago Up 39 seconds 0.0.0.0:8080->80/tcp myweb

Run

Your First Docker Application

(14)

Connect to your App

user@host:~$ wget -q localhost:8080 -O /dev/stdout Hello, France!

Enter in your App

user@host:~$ docker exec -it myweb /bin/bash root@8807f82f280b:/# ls /etc/nginx/ && exit

conf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf ...

Stop and Remove your App

user@host:~$ docker rm -f myweb myweb

root@host:/tmp# docker ps --all

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Run

Your First Docker Application

(15)

Focus on Docker Images...

docker push image[:TAG]

docker image push image[:TAG] docker images

docker image ls [--all]

List all installed images

Pull an image from a registry

docker pull image_{docker image pull image}

Push an image to a registry

Remove an image

docker rmi imagedocker image rm image

docker image prune [--all]

Backup and restore an image (tarball)

docker save -o archive.tar image_{docker load < archive.tar}

(16)

Focus on Docker Containers...

docker stop container

docker container stop container docker ps [--all]

docker container ls [--all]

List all containers

[re]Start an inactive container

docker [re]start container_{docker container [re]start container}

Stop an active container

[stop and] Remove a container

docker rm [-f] container

docker container rm [-f] container

Execute a command in an active

container / Get a shell console

docker exec container command

docker exec -it container /bin/bash

Inspect the content of a container

docker inspect container

(17)

Focus on Docker Containers...

docker create image docker run image

docker commit container [REPO[:TAG]]

Create an image from a container

Show the logs of a container

docker logs container

Create a new container

Backup and restore a container (tarball)

docker export container > archive.tar docker import - container < archive.tar

(18)

17

1 - Get alpine image

user@host:~$ docker pull alpine

2 - Run it

user@host:~$ docker run -d -it --name myapp alpine

3 - Edit a file in the container

user@host:~$ docker exec myapp \

sh -c 'echo "Hello, France!" > /root/msg.txt'

4 - Stop the container

user@host:~$ docker stop myapp

5 - Start the container

user@host:~$ docker start myapp

6 - Check

user@host:~$ docker exec myapp less /root/msg.txt Hello, France

What about my data ?

7 - Stop then remove the container

user@host:~$ docker rm -f myapp

8 - Run again a container

user@host:~$ docker run -d -it --name myapp alpine

9 - Check

user@host:~$ docker exec myapp less /root/msg.txt

more: can't open '/root/msg.txt': No such file or directory

Remember that each container has is own read-write layer

When you instantiate an image, the

new container starts with a clean

filesystem

(19)

Run it

user@host:~$ echo "Hello, Nantes!" > ~/msg.txt && docker run -d -it -v

~/msg.txt:/root/hop.txt --name mydata alpine

Check

user@host:~$ docker exec mydata ls /root hop.txt

user@host:~$ docker exec mydata less /root/hop.txt

Hello, Nantes!

What about my data ?

Run another docker based on alpine

user@host:~$ docker run -d -it -v

~/msg.txt:/root/hop.txt --name mydatb alpine

Edit the message

user@host:~$ echo "Hello, Centrale!" > ~/msg.txt

Check

user@host:~$ docker exec mydata less /root/hop.txt

Hello, Centrale!

user@host:~$ docker exec mydatb less /root/hop.txt

Hello, Centrale!

Sharing data in your docker host with containers

Use the -v option : -v [HOST-DIR]:[CONTAINER-DIR]

(20)

Get information about volumes

user@host:~$ docker inspect mydata ... "Mounts": [ { "Type": "bind", "Source": "/home/user/msg.txt", "Destination": "/root/hop.txt", "Mode": "", "RW": true, "Propagation": "rprivate" } ], ...

What about my data ?

By default, Docker mounts the volume in

read-write mode

Sharing data in your docker host with containers

Instruction to share data in read-only mode

user@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt:ro --name mydata alpine

(21)

20

Run the yoyo container

user@host:~$ docker run -d -it -v /sharing --name yoyo alpine

user@host:~$ docker inspect yoyo ... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c4032d49...", "Source": "/var/lib/docker/volumes/fdc5bd0de90...a54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],

What about my data ?

If the host directory/file is omitted, a data

container (read-write mode per default) is created

The volume specified, here /sharing, is created

inside the container

user@host:~$ touch

/var/lib/docker/volumes/fdc5bd0d.../_data/msg.txt user@host:~$ docker exec yoyo ls /sharing

msg.txt

(22)

21

Run the dong container

user@host:~$ docker run -d -it --volumes-from yoyo --name dong alpine

user@host:~$ docker inspect dong ... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c40...4a54b", "Source": "/var/lib/docker/volumes/fdc5bd0de90...54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],

What about my data ?

user@host:~$ docker exec yoyo ls /sharing Msg.txt

user@host:~$ docker exec dong ls /sharing Msg.txt

user@host:~$ docker exec dong touch /sharing/msg2.txt

user@host:~$ docker exec yoyo ls /sharing msg.txt

Msg2.txt

user@host:~$ ls

/var/lib/docker/volumes/fdc5bd0de90688d5.../_data msg2.txt msg.txt

(23)

user@host:~$ docker image ls --all

REPOSITORY TAG IMAGE ID CREATED SIZE alpine latest 196d12cf6ab1 2 months ago 4.41MB user@host:~$ docker run -d -it --name ding alpine

user@host:~$ docker exec -it ding /bin/sh / # apk update && apk upgrade

fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz v3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]

v3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/community] OK: 9546 distinct packages available

OK: 4 MiB in 13 packages / # exit

user@host:~$ docker commit ding alpine:201812

sha256:4acef3925b22e668e0e1755a3fa5ab6004889cacee28c8a487c5754ea105be70 user@host:~$ docker image ls --all

REPOSITORY TAG IMAGE ID CREATED SIZE alpine 201812 4acef3925b22 27 seconds ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB

Committing Changes in a Container

(24)

user@host:~$ docker diff ding C /lib C /lib/apk C /lib/apk/db ... C /var/cache C /var/cache/apk A /var/cache/apk/APKINDEX.adfa7ceb.tar.gz A /var/cache/apk/APKINDEX.efaa1f73.tar.gz C /root A /root/.ash_history

Committing Changes in a Container

First column: A means that the directory/file

was added, C means that a change was

made, D means that it was removed.

You see all changes applied to the

read-write layer, the container itself - useful

before executing the commit instruction

(25)

Docker Networking Services

Bridge

Host

Share host interfaces

No more network isolation between the

host and containers

Should be used very carefully

Macvlan Bridge

Unique MAC address

Allows you to configure slave/sub-interfaces of a parent, physical ethernet interface, each with its

own unique MAC address

None

No network !

Overlay

Multiple hosts with only L3 connectivity

Combines local bridges and VXLAN, GRE to

overlay container-to-container Useful to offer network

connectivity between containers on multiple

hosts using their own IP addresses

(26)

user@host:~$ docker run --name alice -p 8080:80 -d nginx

user@host:~$ docker run --name bob -p 8081:80 -d nginx

user@host:~$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ffe72798ac2d nginx "nginx -g 'daemon of…" … … 0.0.0.0:8081->80/tcp bob 1a2f2f83e044 nginx "nginx -g 'daemon of…" … … 0.0.0.0:8080->80/tcp alice

Going deeper in docker bridge default mode

Expose unique ports on the host - here

8081 and 8080

(27)

-user@host:~$ brctl show

bridge name bridge id STP enabled interfaces

docker0 8000.02422d1a2630 no veth856faee

vetha53c999

user@host:~$ ip add ...

3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:2d:1a:26:30 brd ff:ff:ff:ff:ff:ff

inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever

7: vetha53c999@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default

link/ether be:71:06:64:64:8f brd ff:ff:ff:ff:ff:ff link-netnsid 1

11: veth856faee@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default

link/ether b2:6c:95:08:43:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 2 Going deeper in docker bridge default mode

(28)

user@host:~$ docker inspect bob "NetworkSettings": { "Bridge": "", ... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.4", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:04", ... }

user@host:~$ docker inspect alice "NetworkSettings": { "Bridge": "", ... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.3", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:03", ... }

Going deeper in docker bridge default mode

Docker assigns a dynamic IP address for both

containers

(29)

user@host:~$ iptables -L -n -v ...

Chain DOCKER (1 references)

pkts bytes target prot opt in out source destination

0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.3 tcp dpt:80 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.4 tcp dpt:80 ...

user@host:~$ iptables -t nat -L -n -v Chain DOCKER (2 references)

pkts bytes target prot opt in out source destination 1 84 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0

0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.3:80 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8081 to:172.17.0.4:80

(30)

user@host:~$ docker image ls --all

REPOSITORY TAG IMAGE ID CREATED SIZE alpine latest 196d12cf6ab1 2 months ago 4.41MB user@host:~$ vi /tmp/Dockerfile

FROM alpine:latest

RUN apk update && apk upgrade

ENTRYPOINT ["/bin/echo","Hello, France!"]

user@host:~$ cd /tmp && docker build . -t hello

Sending build context to Docker daemon 18.94kB Step 1/3 : FROM alpine:latest

---> 196d12cf6ab1

Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fa fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64 /APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x 86_64/APKINDEX.tar.gz v3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main] v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community] OK: 9546 distinct packages available

OK: 4 MiB in 13 packages

Removing intermediate container 116d94cb96fa ---> fd105816bcb9

Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37

Removing intermediate container 8756eb9e1d37 ---> b4277a54d78b

Successfully built b4277a54d78b

Successfully tagged hello:latest

(31)

user@host:~$ docker image ls --all

REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> fd105816bcb9 5 minutes ago 5.71MB hello latest b4277a54d78b 5 minutes ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB

Sending build context to Docker daemon 18.94kB Step 1/3 : FROM alpine:latest

---> 196d12cf6ab1

Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fa fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64 /APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x 86_64/APKINDEX.tar.gz v3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main] v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community] OK: 9546 distinct packages available

OK: 4 MiB in 13 packages

Removing intermediate container 116d94cb96fa

---> fd105816bcb9

Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37

Removing intermediate container 8756eb9e1d37 ---> b4277a54d78b

Successfully built b4277a54d78b

(32)

user@host:~$ vi /tmp/Dockerfile FROM alpine:latest

RUN apk update && apk upgrade RUN apk add openssh

ENTRYPOINT ["/bin/echo","Hello, France!"]

user@host:~$ cd /tmp && docker build . -t hello

---> 196d12cf6ab1

Step 2/4 : RUN apk update && apk upgrade ---> Using cache

---> fd105816bcb9

Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8 fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz (1/6) Installing openssh-keygen (7.7_p1-r3) ... (6/6) Installing openssh (7.7_p1-r3) Executing busybox-1.28.4-r1.trigger OK: 8 MiB in 19 packages

Removing intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0

Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9ada

Removing intermediate container ead00e7c9ada ---> 1d451e42e9e2

Successfully built 1d451e42e9e2 Successfully tagged hello:latest

(33)

---> 196d12cf6ab1

Step 2/4 : RUN apk update && apk upgrade ---> Using cache

---> fd105816bcb9

Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8 fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz (1/6) Installing openssh-keygen (7.7_p1-r3) ... (6/6) Installing openssh (7.7_p1-r3) Executing busybox-1.28.4-r1.trigger OK: 8 MiB in 19 packages

Removing intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0

Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9ada

Removing intermediate container ead00e7c9ada ---> 1d451e42e9e2

Successfully built 1d451e42e9e2 Successfully tagged hello:latest

root@host:/tmp# docker image ls --all

REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 3ba8527195d0 5 minutes ago 11MB hello latest 1d451e42e9e2 5 minutes ago 11MB <none> <none> b4277a54d78b 24 hours ago 5.71MB <none> <none> fd105816bcb9 24 hours ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB root@host:/tmp# docker image prune

WARNING! This will remove all dangling images. Are you sure you want to continue? [y/N] y Deleted Images:

deleted:

sha256:b4277a54d78bc870ba2c1e971f076da6a324cc73f97dead 788eb0823592c9e5e

Total reclaimed space: 0B Build your own image

(34)