Docker Security

Hey, dear Docker DevOps enthusiast! In this post we will discuss about docker security, docker service security, docker engine-level security, etc.

SELinux is Security-Enhanced Linux

it provides a mechanism for supporting access control security policies
SELinux is a set of kernel modifications and user-space tools that have been added to various Linux distros.

The 'root' user by default owns the processes spawned by a container are run.

secgroup limits the disk quota.

Security Issue

Rotate your join-token for both worker and manager when there is a suspicion that someone might have got access to the token for adding managers to the cluster.

Secretes are immutable in a docker swarm cluster. They cannot be updated sof if you want to modify the secret then you have to create a new secret file and update that to the existing service.
step 1: First we need to Create new secret,
step 2: Attach the newly created secret with an update option the service to use this new secret.
step 3: The Service resart may require - docker swarm cluster would take car of that
step 4: and delete the old secret

 docker service update --help |grep secret

      --secret-add secret                  Add or update a secret on a service

      --secret-rm list                     Remove a secret

secrets and configs are encrypted during transit and at rest in a docker swarm.
Configs operate in a similar way to secrets, except that they are not encrypted at rest and are mounted directly into the container’s filesystem without the use of RAM disks. Configs can be added or removed from a service at any time, and services can share a config.




The default location of secrets inside a Docker container '/run/secrets/'. The default locaiton of a config file when using Docker config '/'.

References:

• Swarm Configuration
• Swarm Secret

Docker Networking

Hello, dear DevOps enthusiast, welcome back to DevOpsHunter learners site! In this post, we would like to explore the docker networking models and different types of network drivers and their benefits.

What is Docker Networking?

Understanding docker networking

Docker provided multiple network drivers plugin installed as part of Library along with Docker installation. You have choice and flexibilities. Basically the classification happen based on the number of host participation. SinbleHost WILL 

Let's see the types of network drivers in docker ecosystem.

docker network overview

Docker Contianers are aim to built tiny size so there may be some of the regular network commands may not be available. We need to install them inside containers.

Issue #1 Inside my container ip or ifconfig not working, How to resolve 'ip' command not working?

Solution:

  apt update;  apt install -y iproute2  
  

Issue #2: ping command not working how to resolve this ?? Solution: Go inside your container

  apt update;  apt install -y iputils-ping  
  

What is the purpose of Docker Network?

• Containers need to communicate with external world (network)
• Reachable from external world to provide service (web applications must be accessable)
• Allow Containers to talk to Host machine (container to host)
• inter-container connectivity in same host and multi-host (container to container)
• Discover services provided by containers automatically (search other services)
• Load balance network traffic between many containers in a service (entry to containerization)
• Secure multi-tenant services (Cloud specific network capability)

Features of Docker network

• Type of network-specific network can be used for internet connectivity
• Publishing ports - open the access to services runs in a container
• DNS - Custom DNS setting Load balancing - allow, deny with iptables
• Traffic flow - application access Logging - where logs should go # check the list of docker networking

To explore docker network object related commands you can use --help

docker network help output

Listing of docker default networks. It also shows three network driver options ready to use :

docker network ls 

# There are 3 types of network drivers in Docker-CE Installation with local scoped(Single host) which are ready to use - bridge, host, null. There is the significance in each type of driver.

docker network create --driver bridge app-net
docker network ls
# To remove a docker network use rm
docker network rm app-net

Let's experiment

docker run -d -it --name con1 alpine

docker exec -it con1 sh
# able to communicate with outbound responses
ping google.com

To check the network route table entries inside a container and on your Linux box where Docker Engine hosted:

ip route
or 
ip r

we are good to go for validate that from con1 container able to communicate other container con2 ping con2 # Bridge utility if it is not found then install

yum -y install bridge-utils

# check the bridge and its associated virtual eth cards brctl show

The 'brctl' show command output

There is the mapping of IP address shows the bridge attached to container

Default Bridge Network in Docker

The following experiment will let you understand the default bridge network that is already available when the docker daemon is in execution.

docker run -d -it --name con1 alpine
docker run -d -it --name con2 alpine

docker ps
docker inspect bridge

default bridge network connected with con1, con2

After looking into the inspect command out you will completely get the idea how the bridge works.

Now let's identify the containers con1 and con2

docker exec -it con1 sh
ip addr show
ping -c 4 google.com
ping -c 4 ipaddr-con2
ping -c 4 con2        --- should not work means dns will not work for default bridges
exit

# remove the containers which we tested docker rm -v -f con1 con2

User-defined Bridge Network

docker network create --driver bridge mynetwork
docker network ls
docker inspect mynetwork

docker run -d -it --name con1 --network mynetwork alpine
docker run -d -it --name con2 --network mynetwork alpine

docker ps
docker inspect mynetwork

Now let's see inside the container with the following:

docker exec -it con1 sh
ip addr show
ping -c 4 google.com
ping -c 4 ipaddr-con2
ping -c 4 con2        --- should WORKS, that means dns is avaialable in mynetwork
exit 

Clean up of all the containers after the above experiment

docker rm -v -f con1 con2 #cleanup

Now we can check the connection between two user defined bridge networks.

docker network create --driver bridge mynetwork1
docker network create --driver bridge mynetwork2

docker run -d -it --name con1 --network mynetwork1 alpine
docker run -d -it --name con2 --network mynetwork1 alpine

docker run -d -it --name con3 --network mynetwork2 alpine
docker run -d -it --name con4 --network mynetwork2 alpine

docker inspect mynetwork1
docker inspect mynetwork2

Now we can get the IP Address of containers of both the networks, using the 'docker inspect' command

docker exec -it con1 sh
ping -c 4 ipaddr-con3 #get the ip address using docker container inspect con3
ping -c 4 con3 # This will not work because two bridges isolated

Docker Host networking diriver

Now in the experiment of Host Network example

 docker run --rm -d --network host --name my_nginx nginx
 curl localhost

Now you know about how to run a container attached to a network, But now a container created already with default bridge network that can be attach to a custom bridge network is it possible ? is that have same IP address?

How to connect a user-defined network object with running Container?

We can move a docker container from default bridge network to a user defined bridge network it is possible. When this change happen the container IP address dynamically changes. 

docker  network connect command is used to connect a running container to an existing user-defined bridge. The syntax as follows:

 docker network connect [OPTIONS] NETWORK CONTAINER

Example my_nginx already running container.

docker run -dit --name ng2 \
  --publish 8080:80 \
  nginx:latest 

docker network connect mynetwork ng2

Observe after connecting to User defined bridge network check the IP Address of the container.

To disconnect use the following:

docker network disconnect mynetwork ng2

Once the disconnected check the IP Address of the container ng2 using 'docker container inspect ug2'. This will concludes that container can be migrated from one network to other without stopping it. There is no errors while performing this.

Overlay Network for Multi-host

Docker overlay network will be part of Docker swarm or Kubernetes where you have Multi-Host Docker ecosystem.

docker network create --driver overlay net-overlay
# above command fails with ERROR
docker network ls

The overlay networks will be visible in the network list once you activate the Swarm on your Docker Engine. and for network overlay driver plugins that support it you can create multiple subnetworks. #if swarm not initialized use following command with the IP Address of your Docker Host

docker swarm init --advertise-addr 10.0.2.15

Overlay Network will serve the containers association with Docker 'service' object instead of 'container' object. # scope of the overlay shows swarm

docker service create --help
# Create service of nginx 
docker service create --network=net-overlay --name=app1 --replicas=4 ngnix
docker service ls|grep app1 
docker service inspect app1 |more # look for the VirtualIPs in th output

Docker Overlay driver allows us not only multi-host communication, the overlay driver plugins that support it you can create multiple subnetworks.

docker network create -d overlay \
                --subnet=192.168.0.0/16 \
                --subnet=192.170.0.0/16 \
                --gateway=192.168.0.100 \
                --gateway=192.170.0.100 \
                --ip-range=192.168.1.0/24 \
                --aux-address="my-router=192.168.1.5" --aux-address="my-switch=192.168.1.6" \
                --aux-address="my-printer=192.170.1.5" --aux-address="my-nas=192.170.1.6" \
                my-multihost-network
docker network ls
docker network inspect my-multihost-network # Check the subnets listed out of this command

The output as follows:

 

Docker doc has an interesting Network summary same as it says --

• User-defined bridge networks are best when you need multiple containers to communicate on the same Docker host.
• Host networks are best when the network stack should not be isolated from the Docker host, but you want other aspects of the container to be isolated.
• Overlay networks are best when you need containers running on different Docker hosts to communicate, or when multiple applications work together using swarm services.
• Macvlan networks are best when you are migrating from a VM setup or need your containers to look like physical hosts on your network, each with a unique MAC address.

Reference:

1. Docker Networking: https://success.docker.com/article/networking 
2. User-define bridge network : https://docs.docker.com/network/bridge/ 
3. Docker Expose Ports externally: https://bobcares.com/blog/docker-port-expose/

Docker Image Management

In this post, we will be discussing docker image creation, management and before jumping into this article if you do not yet install Docker? then, I also recommend you to go through my previous post where I've discussed how to install Docker-CE or Docker EE. I would like to expose most of the things related to Docker Images.

Assuming that now you have everything ready! that means Docker engine up and running.

What is all about Docker Image?

According to docker docs --

An image is an executable package that includes everything needed to run an application -- the code, runtime, libraries, environment variables and configuration files.

The runtime of a docker image is called a Docker container.

In simple words, an Image is nothing but a stopped container! Let me put my understanding into a picture first and then we explore all these possible syntax and examples.

Docker Image Life cycle

Let us talk about the docker image that was built with multiple layers.

Docker Images are Layered structure

The docker images in the layered structure make simple, very flexible and easy to built. Docker images are made up of multiple read-only layers(images). New images will be created from the existing set of images. Hundreds or thousands of containers can be spin up as per the need, they are typically based on the same image. When an image is instantiated into a container, a top writable layer is created where an application will be going to run and which will be deleted when the container removed. Docker accomplishes this by using storage drivers. Each storage driver manages the writeable layers and handles the implementation differently, but all storage drivers use the stackable image layer and the copy-on-write(CoW) strategy.


The docker image build will start on top of bootfs filesystem. Layer 0 which we call it as a base image that contains the root filesystem (rootfs). On top of Layer-0 are the read-only layers (1 .. n-1) which may contain the desired new configuration changes on the base image. Perhaps you may have a layer to install the application. Upon some more changes related to the application may be required that could be another layer. This forms layer cake which docker union file system to create docker image. If these configuration changes overlap each other (conflicts) the change on the top layer overrides.

Each image layer will be associated with unique UUIDs.

Docker Image stacked layers in Image and a container

Docker images are read-only, But how do we change inside the Image?

• We don't really need to do changes in an image that is already existing, instead -
• We create a container from that image and then
• Do the required changes on top of it in the container layer and when we satisfied with those changes then, transform into a new layer in the image stack that is using 'docker save' container as image.

What are the differences between Docker image vs docker container?

Differences between Docker Image vs Container

Docker Images - CLI

Docker search command

All the docker search commands will be refer to the Docker public repository content only. Simple search you could do for Jenkins Docker image like :

docker search jenkins

To get top 5 jenkins images out of search list use --limit optoin

docker search jenkins --limit 5

To filter out only the Official images, use the flag value as 'true'. This images will be called "Official" because they were scanned for vulnerabilities check inside the Image done by Docker Inc. This could be helpful when you selecting the Image for your project ensuring they are safe by selecting this option.

docker search jenkins --limit 5  --filter "is-official=true"

Note: There could be multiple images as Official for the same software image.

Docker search for Jenkins Image with limit official options

docker search nginx --filter "is-official=true"

In a contrary we can also use the flag value as 'false'. when we prepare similar kind of image and searching for it.

  docker search nginx --filter "is-official=false"

The default limit is 25 lines default there will be hunders of Public Images but will show only top 25 lines which are sorted with the "stars" count high.

docker search nginx --filter "is-official=false" --limit 10

You should aware of all the help options that are associated with 'docker image'

Manage images

Commands:
  build       Build an image from a Dockerfile
  history     Show the history of an image
  import      Import the contents from a tarball to create a filesystem image
  inspect     Display detailed information on one or more images
  load        Load an image from a tar archive or STDIN
  ls          List images
  prune       Remove unused images
  pull        Pull an image or a repository from a registry
  push        Push an image or a repository to a registry
  rm          Remove one or more images
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE

Run 'docker image COMMAND --help' for more information on a command.

Remove Image

docker image rm --help
Usage:  docker image rm [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Aliases:
  rm, rmi, remove
Options:
  -f, --force      Force removal of the image
      --no-prune   Do not delete untagged parents

Examples:

docker rmi flask:1.0
docker image rm top-img -f
docker image remove sri-flask:1.0 -f

Image Inspect

To get multiple images information in one go you can use this.

$ docker image inspect --help
Usage:  docker image inspect [OPTIONS] IMAGE [IMAGE...]
Display detailed information on one or more images
Options:
  -f, --format string   Format the output using the given Go template

Example:

vagrant@dockerhost:~/samples$ docker image inspect python:3
[
    {
        "Id": "sha256:a6a0779c5fb25f7a075c83815a3803f9fbc5579beb488c86e27e91c48b679951",
        "RepoTags": [
            "python:3"
        ],
        "RepoDigests": [
            "python@sha256:f265c5096aa52bdd478d2a5ed097727f51721fda20686523ab1b3038cc7d6417"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2021-05-12T15:27:54.005567496Z",
        "Container": "0bf84fa1b959359a29c7fa92d2c9e5fc4159c2e3092efda39e9f070d8c3f0017",
        "ContainerConfig": {
            "Hostname": "0bf84fa1b959",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.9.5",
                "PYTHON_PIP_VERSION=21.1.1",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/1954f15b3f102ace496a34a013ea76b061535bd2/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=f499d76e0149a673fb8246d88e116db589afbd291739bd84f2cd9a7bca7b6993"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"python3\"]"
            ],

Image tagging

This is like versioning your docker image build that is used by a dockerfile or another docker image for rename.

$ docker tag --help
Usage:  docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]

Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
Example:

docker tag nginx localhost:5000/nginx

https://docs.docker.com/engine/reference/commandline/image_tag/

Image push

docker image push  --help
Usage:  docker image push [OPTIONS] NAME[:TAG]
Push an image or a repository to a registry
Options:
      --disable-content-trust   Skip image signing (default true)

List images

Usage:  docker image ls [OPTIONS] [REPOSITORY[:TAG]]
Aliases:
  ls, list
Options:
  -a, --all             Show all images (default hides intermediate images)
      --digests         Show digests
  -f, --filter filter   Filter output based on conditions provided
      --format string   Pretty-print images using a Go template
      --no-trunc        Don't truncate output
  -q, --quiet           Only show image IDs

Examples: 1. Filtering dangling images

vagrant@dockerhost:~/samples$ docker image list  --filter dangling=true
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
           fc54bebe79ee   17 hours ago   57.1MB
           2878761c8f4d   34 hours ago   57.1MB

2. List of imaage ids which are dangling using

--quite

option

vagrant@dockerhost:~/samples$ docker image list --quiet --filter dangling=true
fc54bebe79ee
2878761c8f4d

3. Find all latest images

vagrant@dockerhost:~/samples$ docker images --filter=reference='*:latest'
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
namaste_py   latest    44ef94c791ae   2 days ago    895MB
cassandra    latest    132406477368   11 days ago   402MB
busybox      latest    c55b0f125dc6   2 weeks ago  
  

Image History

Docker history command will gives you how this Docker image is build with what instructions. We can compare the the dockerfile content of any of the image with the docker image history command output.  

$ docker image history --help
Usage:  docker image history [OPTIONS] IMAGE
Show the history of an image
Options:
      --format string   Pretty-print images using a Go template
  -H, --human           Print sizes and dates in human readable format (default true)
      --no-trunc        Don't truncate output
  -q, --quiet           Only show numeric IDs

Example for the docker image history

$ docker image history hello-world
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
fce289e99eb9        11 months ago       /bin/sh -c #(nop)  CMD ["/hello"]               0B                  
<  missing >         11 months ago       /bin/sh -c #(nop) COPY file:f77490f70ce51da2…   1.84kB              

We can use the history subcommand with --format option to display only the "CREATED BY" column containing lines.

docker image history hello-world –format {{ .CreatedBy }}
  docker image history hello-world –format {{ .CreatedBy }}={{size}}
  

Example of Tomcat image format with CreatedBy

  docker image history tomcat –format {{ .CreatedBy }}
  

Tomcat docker image history 

Docker Tomcat image history format with "CreatedBy"

Check more Docker commands executions : 

1. Docker container concepts 
2. Docker custom images with Dockerfile

User Management on Universal Control Plane (UCP)

This is a quick tutorial on Docker UCP usage for User Management. Docker UCP provides us multiuser management and Role-based user control. which allows us to create and manage users and teams in an organization. Let's take a look over this user management in detail in this post.

First, we create Organization then we associate a couple of teams then after that add users to those teams.

Login to your UCP management console.

Create an Organization on UCP

Click on the 'user management' in the left side pane.

User Management on UCP

Now in the right pane work area, you can click on the 'Create Organization' top right button.

Enter your organization name a single word without any spaces. even though you enter the name in Capitals it will convert into lower case and store it.

Create Organization on UCP

To complete it click on the 'Create' button.
Once Organization is created it will be listed in the work area. Click on the newly created organization it will give us the option to create the teams.

Create a Team on UCP

Let's prepare the list of commonly required teams for any organization. Then, create them so the list as  following teams:

• dev - Development team
• QA - Quality Assurance team
• prod - Production team

Create Team on UCP

Create User

There will be 'admin' User already created by UCP. we can create new users with 'admin' roles or without it. We would like to create a user with 'admin' access and another without 'admin' access role.

Let's explore this user creation process now.

Create User on docker UCP

The same way we can create another user that having the 'Docker EE Admin' role.
After creating users the summary looks as:

Users created on UCP summary

Add Users to Team

Go to the organization that you have already created. select it. Choose the team to which you will add the user. Here I am adding user to 'qa' team.

Add user to organization/team in UCP

I hope you enjoyed this post about user management on UCP for Docker EE


Next, let us explore the Docker Trusted Registry (DTR).