Docker
What is Docker?
1. cgroups : limits the amount of resources
1.1 blkio (Bulk IO device)
1.2 cpu
1.3 cpuacct (CPU accounting OR CPU usage report)
1.4 cpuset (Assigning CPU)
1.5 devices
1.6 freezer : Suspends and resumes tasks in cgroups
1.7 memory
1.8 net_clas : Linux tc (traffic controller) can identify packet from particular cgroups
1.9 net_prio : for bandwidth management
1.10 ns (namespace)
1.11 perf_event : To identify membership of task to specific cgroups for performance analysis.
/etc/cgconfig.d is daemon process for cgroups
Check all cgroups under folder /sys/fs/cgroups
I found additional cgroups in my syste,
hugetlb
pids
rdma
systemd
unified
2. namespace = mount, IPC, network, PID, user (user and group) and UTS (=Unix Time Sharing. Host name and domain name).
2.1 Network Namespace
2.1.1 Network name space includes : network devices, IPv4 and IPv6 protocol stacks, IP routing tables, firewalls, the /proc/net directory, the /sys/class/net directory, port numbers (sockets), and so on
2.1.2 Namespace API has 3 system calls I. clone II. unshare III. setns
Typically, "ip netns" can be used
ip netns exec $namespace_id
to run some arbitrary command inside a given namespace.
However, ip netns only knows about namespaces listed in /var/run/netns/, and neither docker nor Kuberenetes have symlink its namespaces there.
Docker and Kuberenetes can accessed network namespace using "nsenter" and specifying the process ID of the container, e.g.
nsenter -t ${PID} -n ip addr
nsenter -t ${PID} -n route -n
sudo nsenter -t ${PID} -n /bin/sh
nsenter discovers the network namespace at /proc/$PID/ns/net, where the namespace is accessible (by syscalls like setns(2)) via file descriptor.
Docker creates bridges. Docker sets routes. docker uses iptables
Docker Architecture
Docker CLI
dockerd : docker daemon
(docker-)containerd : daemon listening to on UDS, gRPC endpoint
(docker-)containerd-ctr : Light weight CLI to communicate with containerd, for debug purpose
(docker-)runc : lightwieght binary deals with namespace, cgroups etc.
(docker-)containerd-shim : Between containerd and runc
Docker Dependencies
ca-certificates package contains certificates provided by the Certificate Authorities.
apt-transport-https allows the use of repositories accessed and downloaded via https protocol
software-properties-common provides some useful scripts for adding and removing PPAs + the DBUS backends. Without it, you would need to add and remove repositories (such as PPAs) manually by editing /etc/apt/sources.list and/or any subsidiary files in /etc/apt/sources.list.d
sudo apt install docker.io
To add non-root user
sudo groupadd docker
sudo gpasswd -a $USERdocker
sudo usermod -aG docker $USER
sudo setfacl -m u:$USER:rw /var/run/docker.sock
Docker Flow
docker run puts image to container
1. docker commit creates new image from container
Here commit will also tag image. default tag value is 'latest"
2. docker tag "old name" "new name may contain registry url"
Here "tag" is optional.
Instead of 1, and 2 use single command
"docker commit "
docker image and container both have IDs. They are different.
Docker run command
docker run --memory --cpu-share --cpu-quota -p [outside port:]"inside port"[/tcp|udp] --link --name -ti --privileged=true --net="host | some name" --ip "ip address" -v [absolute local path:] --restart=always --pid=host --env KEY=VALUE --volumes-from --rm -d [:tag default is latest]
rm = remove container, once it is done.
d = detach
-p here outside port is optional
-p:/ default is tcp
-P = --publish-all For exposing all ports
--link = it will add "ip address another container name" at /etc/hosts. It will auto detect IP address of another container. Assume the IP address does not change.
-v for volume
--privileged to get full access of host OS
--pid will give more privilege to control PID of host OS
--restart : To restart if it dies.
--net : To specify network name space
docker run = docker create + docker start
https://docs.docker.com/engine/reference/commandline/run/
^d to exit container.
Attach and detach
attach using
docker attach <container name>
to detach, when you are inside container, press ^p, ^q
Run process at container
docker -exec
Cannot add ports, volumes etc.
docker -exec -ti my_container sh -c "echo a && echo b"
Running more than one process or service in a container is discouraged for maximum efficiency & isolation.
Logs
docker logs
Remove and Kill
docker kill
Container moves to stopped state.
docker rm
This will remove the container
docker ps -l
l = last running container.
Network
docker port
It will list out external v/s internal ports. Same like iptables
For dynamic link, first create network
docker network create
Then use --net
here a name server will be added to network and it will take care of new ip address.
If we specify --net=host then we can see, all the bridges inside container also with "brctl show" command
Docker inspect
This command gives various information about docker like author, pid, IP address etc.
docker inspect --format '{{ .State.Pid }}' "name"
docker inspect --format '{{ .NetworkSettings.IPAddress }}' "name"
Docker images
docker images
List downloaded image.
Docker Registry
name = registry FQDN : port /organization name/image name:version tag
short name = organization name/image name
docker rmi
docker login
docker pull
docker push "name as per used in docker tag"
docker search
https://hub.docker.com/
Amazon's ECR,
Sonatype Nexus,
JFrog Artifactory
https://quay.io/
https://cloud.google.com/container-registry/
port 5000 for push, pull etc.
https://docs.docker.com/registry/deploying/#copy-an-image-from-docker-hub-to-your-registry
The docker images can be stored at local host, docker, AWS, Google, Microsoft
docker save -o "tar.gz file" "one or more than one docker image:tag"
docker load -i "tar.gz file"
Volumes
Volumes are like virtual disc
Two types: (1) persistent (-v) v/s (2) ephemeral (--volumes-from)
Volumes are not part of image
mount -o bind "original folder" "new name of folder. May be existing folder"
Host file system over guest can be mounter. Not visa versa.
Dockerfile
* Dockerfile shall start with FROM
* Dockerfile name shall be Dockerfile. D capital. docker will search for this file, to build Docker image. Now we can have file with any name and specify that file to Docker with -f option.
* Dockerfile can have multiple lines with "CMD ". The last "CMD " will override all previous lines with "CMD "
* One can pass arguments to Entrypoint
* COPY and ADD are similar. COPY allows to add file from local file system only. ADD allows to add file from URL also.
Sample Docker File
https://blog.hasura.io/the-ultimate-guide-to-writing-dockerfiles-for-go-web-apps-336efad7012c/
https://blog.hasura.io/an-exhaustive-guide-to-writing-dockerfiles-for-node-js-web-apps-bbee6bd2f3c4/
Reference
https://docs.docker.com/engine/reference/builder/
https://docs.docker.com/engine/reference/commandline/docker/
https://jvns.ca/blog/2016/10/10/what-even-is-a-container/
Playground
https://labs.play-with-docker.com/
https://nickjanetakis.com/blog/a-linux-dev-environment-on-windows-with-wsl-docker-tmux-and-vscode
https://nickjanetakis.com/blog/using-wsl-and-mobaxterm-to-create-a-linux-dev-environment-on-windows
Explore further:
1. How to run xserver in container ?
Here are the links
https://blog.jessfraz.com/post/docker-containers-on-the-desktop/ 4.Lynx
and
https://github.com/moby/moby/issues/8710
2. How to connect using docker client to remote docker server?
3. how to expose my docker server on network? How to bind it to tcp socket instead of unix socket
4. Is LXD replacement of docker?
Ansible modules
https://docs.ansible.com/ansible/latest/modules/docker_container_module.html
https://docs.ansible.com/ansible/latest/modules/docker_network_module.html
etc.
Reference
https://linuxcontainers.org/lxd/introduction/
https://us.images.linuxcontainers.org/
https://linuxcontainers.org/lxd/getting-started-cli/
- Docker is programme written in Go
- Docker is an open source community project started in 2013
- Docker is a company that supports this project.
- Docker is a containerization technology/platform for Linux
- Docker is command line tool
- Docker is programme written in Go
- Docker needs Linux kernel version 3.10 or more.
- Docker is 2 programme = client + server. The server can be on remote machine
- Docker manage kernel features (1) cgroups (2) namespace (3) copy-on-write COW file system
- Cgroups control what a process can use. Whereas namespaces control what a process can see
1. cgroups : limits the amount of resources
1.1 blkio (Bulk IO device)
1.2 cpu
1.3 cpuacct (CPU accounting OR CPU usage report)
1.4 cpuset (Assigning CPU)
1.5 devices
1.6 freezer : Suspends and resumes tasks in cgroups
1.7 memory
1.8 net_clas : Linux tc (traffic controller) can identify packet from particular cgroups
1.9 net_prio : for bandwidth management
1.10 ns (namespace)
1.11 perf_event : To identify membership of task to specific cgroups for performance analysis.
/etc/cgconfig.d is daemon process for cgroups
Check all cgroups under folder /sys/fs/cgroups
I found additional cgroups in my syste,
hugetlb
pids
rdma
systemd
unified
2. namespace = mount, IPC, network, PID, user (user and group) and UTS (=Unix Time Sharing. Host name and domain name).
2.1 Network Namespace
2.1.1 Network name space includes : network devices, IPv4 and IPv6 protocol stacks, IP routing tables, firewalls, the /proc/net directory, the /sys/class/net directory, port numbers (sockets), and so on
2.1.2 Namespace API has 3 system calls I. clone II. unshare III. setns
Typically, "ip netns" can be used
ip netns exec $namespace_id
to run some arbitrary command inside a given namespace.
However, ip netns only knows about namespaces listed in /var/run/netns/, and neither docker nor Kuberenetes have symlink its namespaces there.
Docker and Kuberenetes can accessed network namespace using "nsenter" and specifying the process ID of the container, e.g.
nsenter -t ${PID} -n ip addr
nsenter -t ${PID} -n route -n
sudo nsenter -t ${PID} -n /bin/sh
nsenter discovers the network namespace at /proc/$PID/ns/net, where the namespace is accessible (by syscalls like setns(2)) via file descriptor.
Docker creates bridges. Docker sets routes. docker uses iptables
Docker Architecture
Docker CLI
dockerd : docker daemon
(docker-)containerd : daemon listening to on UDS, gRPC endpoint
(docker-)containerd-ctr : Light weight CLI to communicate with containerd, for debug purpose
(docker-)runc : lightwieght binary deals with namespace, cgroups etc.
(docker-)containerd-shim : Between containerd and runc
Docker Dependencies
ca-certificates package contains certificates provided by the Certificate Authorities.
apt-transport-https allows the use of repositories accessed and downloaded via https protocol
software-properties-common provides some useful scripts for adding and removing PPAs + the DBUS backends. Without it, you would need to add and remove repositories (such as PPAs) manually by editing /etc/apt/sources.list and/or any subsidiary files in /etc/apt/sources.list.d
sudo apt install docker.io
To add non-root user
sudo groupadd docker
sudo gpasswd -a $USER
sudo usermod -aG docker $USER
sudo setfacl -m u:$USER:
Docker Flow
docker run puts image to container
1. docker commit creates new image from container
Here commit will also tag image. default tag value is 'latest"
2. docker tag "old name" "new name may contain registry url"
Here "tag" is optional.
Instead of 1, and 2 use single command
"docker commit
docker image and container both have IDs. They are different.
docker run --memory
rm = remove container, once it is done.
d = detach
-p here outside port is optional
-p
--link = it will add "ip address
-v for volume
--privileged to get full access of host OS
--pid will give more privilege to control PID of host OS
--restart : To restart if it dies.
--net : To specify network name space
docker run = docker create + docker start
https://docs.docker.com/engine/reference/commandline/run/
^d to exit container.
attach using
docker attach <container name>
to detach, when you are inside container, press ^p, ^q
Run process at container
docker -exec
Cannot add ports, volumes etc.
docker -exec -ti my_container sh -c "echo a && echo b"
Running more than one process or service in a container is discouraged for maximum efficiency & isolation.
Logs
docker logs
Remove and Kill
docker kill
Container moves to stopped state.
docker rm
This will remove the container
docker ps -l
l = last running container.
Network
docker port
It will list out external v/s internal ports. Same like iptables
For dynamic link, first create network
docker network create
Then use --net
here a name server will be added to network and it will take care of new ip address.
If we specify --net=host then we can see, all the bridges inside container also with "brctl show" command
Docker inspect
This command gives various information about docker like author, pid, IP address etc.
docker inspect --format '{{ .State.Pid }}' "name"
docker inspect --format '{{ .NetworkSettings.IPAddress }}' "name"
Docker images
docker images
List downloaded image.
Docker Registry
name = registry FQDN : port /organization name/image name:version tag
short name = organization name/image name
docker rmi
docker login
docker pull
docker push "name as per used in docker tag"
docker search
https://hub.docker.com/
Amazon's ECR,
Sonatype Nexus,
JFrog Artifactory
https://quay.io/
https://cloud.google.com/container-registry/
port 5000 for push, pull etc.
https://docs.docker.com/registry/deploying/#copy-an-image-from-docker-hub-to-your-registry
The docker images can be stored at local host, docker, AWS, Google, Microsoft
docker save -o "tar.gz file" "one or more than one docker image:tag"
docker load -i "tar.gz file"
Volumes
Volumes are like virtual disc
Two types: (1) persistent (-v) v/s (2) ephemeral (--volumes-from)
Volumes are not part of image
mount -o bind "original folder" "new name of folder. May be existing folder"
Host file system over guest can be mounter. Not visa versa.
Dockerfile
| No. | Purpose | docker file | "docker run" command |
|---|---|---|---|
| 1 | Base image | FROM | |
| 2 | "Author" as output of "docker inspect" command | MAINTAINER | |
| 3 | Debug statement. Execute commands and commit the results on top of the previous file layer | RUN | |
| 4 | copy files or directories from the local (or remote) URL to the filesystem of the image. | ADD "URL | local path" "container path" | |
| 5 | During build time and run time | ENV | --env (or -e)KEY=VALUE --env-file |
| 6 | It is like CMD. It is first programe to run inside container. It makes container as executable | ENTRYPOINT | --entrypoint It will override from DockerFile |
| 7 | CMD | Last Argument is combination of ENTRYPOINT + CMD | |
| 8 | Multi project file. copy files or directories from the local (or remote) URL to the filesystem of the image. | COPY | |
| 9 | sets working dir for build and for container. it is like CD | WORKDIR | --workdir |
| 10 | binary argument | Shell Form | |
| 11 | ["bianry", "argument"] | Exec Form | |
| 12 | Opening outgoing port at iptables firewall | EXPOSE portNumber | -p [outside port:]"inside port" P=publish. Publish is for outside network. Expose is for inside network. |
| 13 | Avoid local path | VOLUME ["optional local path" "container path] | -v and --volume-from |
| 14 | the container will run as user | USER | sudo |
* Dockerfile shall start with FROM
* Dockerfile name shall be Dockerfile. D capital. docker will search for this file, to build Docker image. Now we can have file with any name and specify that file to Docker with -f option.
* Dockerfile can have multiple lines with "CMD ". The last "CMD " will override all previous lines with "CMD "
* One can pass arguments to Entrypoint
* COPY and ADD are similar. COPY allows to add file from local file system only. ADD allows to add file from URL also.
Sample Docker File
https://blog.hasura.io/the-ultimate-guide-to-writing-dockerfiles-for-go-web-apps-336efad7012c/
https://blog.hasura.io/an-exhaustive-guide-to-writing-dockerfiles-for-node-js-web-apps-bbee6bd2f3c4/
Reference
https://docs.docker.com/engine/reference/builder/
https://docs.docker.com/engine/reference/commandline/docker/
https://jvns.ca/blog/2016/10/10/what-even-is-a-container/
Playground
https://labs.play-with-docker.com/
https://nickjanetakis.com/blog/a-linux-dev-environment-on-windows-with-wsl-docker-tmux-and-vscode
https://nickjanetakis.com/blog/using-wsl-and-mobaxterm-to-create-a-linux-dev-environment-on-windows
Explore further:
1. How to run xserver in container ?
Here are the links
https://blog.jessfraz.com/post/docker-containers-on-the-desktop/ 4.Lynx
and
https://github.com/moby/moby/issues/8710
2. How to connect using docker client to remote docker server?
3. how to expose my docker server on network? How to bind it to tcp socket instead of unix socket
4. Is LXD replacement of docker?
Ansible modules
https://docs.ansible.com/ansible/latest/modules/docker_container_module.html
https://docs.ansible.com/ansible/latest/modules/docker_network_module.html
etc.
Reference
https://linuxcontainers.org/lxd/introduction/
https://us.images.linuxcontainers.org/
https://linuxcontainers.org/lxd/getting-started-cli/
0 comments:
Post a Comment