DevOps Hunter: dockerfile

Showing posts with label dockerfile. Show all posts

Saturday, November 6, 2021

HEALTHCHECK Instructions in Dockerfile

Hello Guys in this post I wish to explore more intresting instructions HEALTHCHECK which can be used in Dockerfile.

The most common requirement for any real-time projects monitoring using a side-car container which could run in parallel and try to check the process or application URL check or container reachability using ping command etc.

Dockerfie instruction HEALTHCHECK

In Dockerfile we can have HEALTHCHECK instruction that allows us to know the condition of an application test runs as expected or not, when a container runs this will be returns a status as healthy, unhealthy based on the HEALTHCHECK command exit code. If the exit code 0 then returns healthy otherwise unhealthy.

HEALTHCHECK [options] CMD [health check command]

Example:

HEALTHCHECK --interval=3s CMD ping c1 172.17.0.2

here are the Healthcheck options

--interval=time in sec (duration 30s is default)
--timeout=time in sec (duration 30s is default)
--start-period=time in sec (duration 0s is default)
--retries=3 () default 3

Let's jump into experiment mode:

docker run -dt --name main-target busybox sh; docker ps 
docker inspect main-target

More specific

To get only IPaddress of a containter use the following format option:

alias dcip="docker inspect \
 -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "
ipdc main-target

Note: when you define the alias for ipdc important point don't miss space at the of the line.
Guys, here is glimpse of 'ping' command exit code:

ping -c1 google.com ; echo $?
ping -c1 shekhar.com ; echo $?

Observe that exit codes values, if a website exits returns 0 if not non-zero.

Get the main-target container IPAddress from the docker inspect command output. Now we will create a Dockerfile with the following code:

#File: Dockerfile
FROM busybox 
LABEL AUTHOR Pavan Deverakonda
HEALTHCHECK --interval=5s CMD ping -c 1 172.17.0.2

Note: You could find the IP from previous docker inspect command.

Let's call this image as monitor

docker build -t monitor .
docker run -dt --name monping monitor; docker ps

Observe the STATUS column for corresponding containers.

alias s="docker inspect -f '{{json .State}}' "
s monping |jq

docker inspect output filter with jq for HEALTHCHECK

USECASE 2: HEALTHCHECK for web applications

Let's see the 'curl' command usage in general

The following command returns HTML content which may be multiple lines

curl http://devopshunter.blog.com/test

Let's make this command usage in minimal way using -f or --fail options in curl command:

# Fail in silently single liner 
curl http://devopshunter.blog.com/test.txt -f

curl command with --fail or -f option

Run a container with a healthcheck command using. A Linux command that checks http uri using `curl` that returns HTML code or HTTP code as per the web applicaiton availability.

docker run -dt --name health-con \
 --health-cmd "curl --fail http://localhost" busybox

Here we have not used any HEALTHCHECK options, so it will try to check by running health-cmd 30sec interval 3 retries and timeout for each as 30sec. that means after 2 minutes you can get the health status as 'unhealthy'. Because busybox don't run any web server inside the container.
We can check the health status of self container or other container which is accessable. Otherwise if it shared a network with other container which is running a web server.

#File: Dockerfile
FROM busybox 
LABEL AUTHOR Pavan Deverakonda
HEALTHCHECK --interval=5s CMD curl --fail http://localhost

Build the monitoring image that contains HEALTHCHECK instruction.

  
docker build -t moncurl .
docker images

For now we will test the same busybox container - unhealthy status.

docker run -dt --name moncurl-con moncurl sh
# Check the container health 
watch docker ps
  

#cleanup
docker rm -v -f health-con

Now let's see the interval option how it will impact a container health:

docker run -dt --name health-con  --health-cmd "curl --fail http://localhost" --health-interval=3s busybox
watch docker ps

My observation - at 0s(when container started) healthcheck starts after 3s test it, retries 3 times that means 3times 3s = 9s you will get the health status changed.

USECASE 3: HEALTHCHECK with Interval and Retries options

We can run a container to check the health with options interval and retries together as:

UNHEALTHY

docker run -dt --name health-con3 --health-cmd "curl -f http://localhost" --health-interval=3s --health-retries=1 busybox 
watch docker ps

HEALTHY

docker run -dt --name health-con3 --health-cmd "curl -f http://localhost" --health-interval=3s --health-retries=1 nginx 
watch docker ps

healthy status

Let's build a Healthcheck image

  
#File: Dockerfile
FROM nginx
LABEL AUTHOR Pavan Deverakonda
HEALTHCHECK --interval=5s --timeout=3s CMD curl --fail http://localhost || exit 1
EXPOSE 80

Now build the image

docker build -t moncurl:2 .
docker images

create the container with that image:

docker run -dt --name health-con2 moncurl:2 sh

Please comment and share with your friends!

Saturday, May 22, 2021

Creating your Custom Image using Dockerfile

How do you create docker image?

There are multiple ways to create new docker images

docker commit: creates a new layer (and a new image) from a running container.
docker build: performs a repeatable build sequence using dockerfile.
docker import: loads a tarball into Docker engine, as a standalone base layer.

We don't prefer to use docker import, because the import option can be used for various hacks, but its main purpose is to bootstrap the creation of base images.

Working on Docker Custom Image

Docker images can be pulled from Docker registries. Docker owns Docker Hub and Docker cloud public repositories where you could find the free images. Docker store is a place where Docker images could be on sale. You can create an image and sell it who need it. Docker client is CLI which allows us to deal with the Docker objects. Docker objects are Docker images, Containers, Network, volumes and services. An image is a read-only template that allows us to create instances in runtime containers.

Dockerfile structure

We can create an image from a base image by adding some customizations like installing new libraries, software. For example, using oraclelinux as base image on top it installs httpd.
Here base image could be available in the public registry or custom image build for your project.

The easiest way to customize our docker image using Dockerfile, Where Dockerfile includes instructions to create layers in your docker image. Let's explore how to build our own image with dockerfile with syntax

How to create an efficient image via a Dockerfile?

Start with an appropriate base image
Do NOT Combine multiple applications into a single container
Avoid installing unnecessary packages
Use multi-stage builds

Dockerfile format

The dockerfile is a construct of a set of instructions to build the docker image. The dockerfile starts with comments where every scripting language allows us to use '#'(hash symbol) to comment a line. Next base image FROM where the image can be pulled. Here we have two options Docker Private registry for your organization otherwise Docker Hub public cloud which can be used for the initial learnings or for Proof of Concepts to built.
Create a Dockerfile in any editor, my preferrable editor is VisualStudio Code, which gives me nice syntax highlighting for Dockerfile instructions.

# This is first sample Dockerfile
FROM busybox
LABEL AUTHOR Pavan Devarakonda
RUN echo "Building sample docker image"
CMD echo "Hello Vybhava Container!"

To create the docker image following command:

docker build -t hello .

Here the important thing is that Dockerfile don’t follow any root directory you need to specify at the end of the build command in the above we have used dot(.) that is the current directory. You have other option too you can define other PATH as per your needs of project team collaboration.
Docker build command best options

docker build -f /path/to/a/Dockerfile . #absolute path
docker build -t vybhava/myapp . #relative path of Dockerfile
docker build -t vybhava/myapp:1.0 . # with tag label value

Usage of .dockerignorefile

To increase the build performance, exclude files and directories by adding a .dockerignore file to the context directory.

 *.md
 !README.md
 *.tmp

Escape Parser Directive

Exploring Docker instructions

Let's examine each command in detail how it works.

FROM Instruction

The FROM instruction initializes a new build stage and sets the Base Image for subsequent instructions. Let's see an example

# This is first sample Dockerfile
FROM ubuntu
CMD ["/usr/bin/wc","--help"]

Ref: https://docs.docker.com/engine/reference/builder/#from

COPY and ADD instructions

copies all files and folders from current directory to /app folder in container-Image

COPY . /app

copies everything from the web folder to container-Image /app/web

COPY ./web /app/web

copies specific file to /data/my-sample.txt file inside container-Image

COPY sample.txt /data/my-sample.txt

Compressed files such as tar, tar.gz, gz or zip files when copying to container-Image it will uncompresses it which can be any format.

ADD jdk-8u241-linux-x64.tar.gz /u01/app
ADD APEX_21.1.zip /u01/

You can add files from the url to a container Image

ADD http://github.com/sample.txt /data/

EXPOSE

The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. https://docs.docker.com/engine/reference/builder/#expose The LABEL instruction adds metadata to an image. A LABEL is a key-value pair. To include spaces within a LABEL value, use quotes and backslashes as you would in command-line parsing.

Reference : https://docs.docker.com/engine/reference/builder/#label

RUN instruction

RUN has 2 forms:

RUN  (shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows)
RUN ["executable", "param1", "param2"] (exec form)

The RUN instruction will execute any commands in a new layer on top of the current image and commit the results. The resulting committed image will be used for the next step in the Dockerfile.

Reference: https://docs.docker.com/engine/reference/builder/#run

ENV instruction

The ENV instruction sets the environment variable 'key' to the value "value". Example:

ENV JDK_DIR /tmp
ENV myvar=/home/tomcat
ENV myvar=/home/apache2 var2=$myvar
ENV var3=$myvar

The environment variables set using ENV will persist when a container is run from the resulting image.

WORKDIR instruction

Once the container comes to running state will switch to the directory which you mentioned as WORKDIR instruction. The value you pass it to WORKDIR is a PATH that should be existing in the container.

Example: WORKDIR /tmp

ENTRYPOINT and CMD instructions

The ENTRYPOINT and CMD both the commands are executed run-time that is Container startup time. We cannont have multiple ENTRYPOINT or CMD commands in a Dockerfile

An ENTRYPOINT allows configuring a container that will run as an executable.

ENTRYPOINT has two forms:

   ENTRYPOINT ["executable", "param1", "param2"] (exec form, preferred)
   ENTRYPOINT command param1 param2 (shell form)

Example1

ENTRYPOINT with CMD params

   
# This is Entrypoint sample dockerfile
FROM alpine
LABEL MAINTAINER BHAVANISHEKHAR@GMAIL.COM
ENTRYPOINT ["ping"]
CMD ["-c","4","www.docker.com"]

Build the image from the above instructions:

 docker build -t entrycp:1.0 -f entrycmdparm.Dockerfile .

#Run the container

docker run entrycp:1.0

Example 2:

ENTRYPOINT param with CMD param

   
# This is Entrypoint sample dockerfile
FROM ubuntu
LABEL USER Pavan
ENTRYPOINT ["top", "-b"]
CMD ["-c"]

dockerfile is a blue-print for docker image created with instructions

Python Web Framework Flask App on Docker Container

Namaste this post is an extension of the Docker image management. In this post, I would like to talk about how the Dockerfile content will work to build the Python web framework using the Flask app.

Let's consider a web platform we have multiple choices to use as a web application to run on docker container:

Python web app using Flask
Node.JS app
Web app run with Go

For all these program execution structures is similar you must have 'app' folder. Where you can write your web application code.

FROM python:3.9.4-alpine

COPY requirements.txt /
RUN pip3 install -r /requirements.txt

COPY . /app
WORKDIR /app

ENTRYPOINT ["./gunicorn.sh"]

Here the base image used as Python3 version with apline which is thinner image. As pip installer will use requirement.txt file to install the dependencies libraries and packages required for running Flask.
COPY instruciton used to copy the files from docker host machine to container image.
WORKDIR is like cd command.
ENTRYPOINT instruction will take param which can be a script or command that could be executed

Step 1: Get the Flask Docker app Dockerflle from GitHub. You can have the following repository by Fort on the GitHub then you can use the

git clone

on Docker daemon running VM.

    git clone https://github.com/BhavaniShekhar/Flask_Docker_App-1.git
    cd Flask_Docker_App-1/
	ls -lrt

Step 2: Update some files and commit your changes. Here I've modified the index.html present in the

 vi app/templates/index.html

. Now, all set we can save the changes.

  git add .
  git commit -m 'added text to index.html'

Remember this will works only when you set the

 git config --global user.email

and

 git config --global user.name

Step 3: Build the docker image with the following command:

 
  docker build -t flask:1.3 .

Step 4: Construct the container based on the flask image

 
  docker run -d -p 8081:80 flask:1.3

Step 5: Python Flask based web app ready to use open a browser and try to access it with host ip and host port http://192.168.33.250:8081/

You can see the output as follows:

Thursday, November 28, 2019

Docker Image Management

In this post, we will be discussing docker image creation, management and before jumping into this article if you do not yet install Docker? then, I also recommend you to go through my previous post where I've discussed how to install Docker-CE or Docker EE. I would like to expose most of the things related to Docker Images.

Assuming that now you have everything ready! that means Docker engine up and running.

What is all about Docker Image?

According to docker docs --

An image is an executable package that includes everything needed to run an application -- the code, runtime, libraries, environment variables and configuration files.

The runtime of a docker image is called a Docker container.

In simple words, an Image is nothing but a stopped container! Let me put my understanding into a picture first and then we explore all these possible syntax and examples.

Docker Image Life cycle

Let us talk about the docker image that was built with multiple layers.

Docker Images are Layered structure

The docker images in the layered structure make simple, very flexible and easy to built. Docker images are made up of multiple read-only layers(images). New images will be created from the existing set of images. Hundreds or thousands of containers can be spin up as per the need, they are typically based on the same image. When an image is instantiated into a container, a top writable layer is created where an application will be going to run and which will be deleted when the container removed. Docker accomplishes this by using storage drivers. Each storage driver manages the writeable layers and handles the implementation differently, but all storage drivers use the stackable image layer and the copy-on-write(CoW) strategy.

The docker image build will start on top of bootfs filesystem. Layer 0 which we call it as a base image that contains the root filesystem (rootfs). On top of Layer-0 are the read-only layers (1 .. n-1) which may contain the desired new configuration changes on the base image. Perhaps you may have a layer to install the application. Upon some more changes related to the application may be required that could be another layer. This forms layer cake which docker union file system to create docker image. If these configuration changes overlap each other (conflicts) the change on the top layer overrides.

Each image layer will be associated with unique UUIDs.

Docker Image stacked layers in Image and a container

Docker images are read-only, But how do we change inside the Image?

We don't really need to do changes in an image that is already existing, instead -
We create a container from that image and then
Do the required changes on top of it in the container layer and when we satisfied with those changes then, transform into a new layer in the image stack that is using 'docker save' container as image.

What are the differences between Docker image vs docker container?

Differences between Docker Image vs Container

Docker Images - CLI

Docker search command

All the docker search commands will be refer to the Docker public repository content only. Simple search you could do for Jenkins Docker image like :

docker search jenkins

To get top 5 jenkins images out of search list use --limit optoin

docker search jenkins --limit 5

To filter out only the Official images, use the flag value as 'true'. This images will be called "Official" because they were scanned for vulnerabilities check inside the Image done by Docker Inc. This could be helpful when you selecting the Image for your project ensuring they are safe by selecting this option.

docker search jenkins --limit 5  --filter "is-official=true"

Note: There could be multiple images as Official for the same software image.

Docker search for Jenkins Image with limit official options

docker search nginx --filter "is-official=true"

In a contrary we can also use the flag value as 'false'. when we prepare similar kind of image and searching for it.

  docker search nginx --filter "is-official=false"

The default limit is 25 lines default there will be hunders of Public Images but will show only top 25 lines which are sorted with the "stars" count high.

docker search nginx --filter "is-official=false" --limit 10

You should aware of all the help options that are associated with 'docker image'

Manage images

Commands:
  build       Build an image from a Dockerfile
  history     Show the history of an image
  import      Import the contents from a tarball to create a filesystem image
  inspect     Display detailed information on one or more images
  load        Load an image from a tar archive or STDIN
  ls          List images
  prune       Remove unused images
  pull        Pull an image or a repository from a registry
  push        Push an image or a repository to a registry
  rm          Remove one or more images
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE

Run 'docker image COMMAND --help' for more information on a command.

Remove Image

docker image rm --help
Usage:  docker image rm [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Aliases:
  rm, rmi, remove
Options:
  -f, --force      Force removal of the image
      --no-prune   Do not delete untagged parents

Examples:

docker rmi flask:1.0
docker image rm top-img -f
docker image remove sri-flask:1.0 -f

Image Inspect

To get multiple images information in one go you can use this.

$ docker image inspect --help
Usage:  docker image inspect [OPTIONS] IMAGE [IMAGE...]
Display detailed information on one or more images
Options:
  -f, --format string   Format the output using the given Go template

Example:

vagrant@dockerhost:~/samples$ docker image inspect python:3
[
    {
        "Id": "sha256:a6a0779c5fb25f7a075c83815a3803f9fbc5579beb488c86e27e91c48b679951",
        "RepoTags": [
            "python:3"
        ],
        "RepoDigests": [
            "python@sha256:f265c5096aa52bdd478d2a5ed097727f51721fda20686523ab1b3038cc7d6417"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2021-05-12T15:27:54.005567496Z",
        "Container": "0bf84fa1b959359a29c7fa92d2c9e5fc4159c2e3092efda39e9f070d8c3f0017",
        "ContainerConfig": {
            "Hostname": "0bf84fa1b959",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.9.5",
                "PYTHON_PIP_VERSION=21.1.1",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/1954f15b3f102ace496a34a013ea76b061535bd2/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=f499d76e0149a673fb8246d88e116db589afbd291739bd84f2cd9a7bca7b6993"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"python3\"]"
            ],

Image tagging

This is like versioning your docker image build that is used by a dockerfile or another docker image for rename.

$ docker tag --help
Usage:  docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]

Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
Example:

docker tag nginx localhost:5000/nginx

https://docs.docker.com/engine/reference/commandline/image_tag/

Image push

docker image push  --help
Usage:  docker image push [OPTIONS] NAME[:TAG]
Push an image or a repository to a registry
Options:
      --disable-content-trust   Skip image signing (default true)

List images

Usage:  docker image ls [OPTIONS] [REPOSITORY[:TAG]]
Aliases:
  ls, list
Options:
  -a, --all             Show all images (default hides intermediate images)
      --digests         Show digests
  -f, --filter filter   Filter output based on conditions provided
      --format string   Pretty-print images using a Go template
      --no-trunc        Don't truncate output
  -q, --quiet           Only show image IDs

Examples: 1. Filtering dangling images

vagrant@dockerhost:~/samples$ docker image list  --filter dangling=true
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
           fc54bebe79ee   17 hours ago   57.1MB
           2878761c8f4d   34 hours ago   57.1MB

2. List of imaage ids which are dangling using

--quite

option

vagrant@dockerhost:~/samples$ docker image list --quiet --filter dangling=true
fc54bebe79ee
2878761c8f4d

3. Find all latest images

vagrant@dockerhost:~/samples$ docker images --filter=reference='*:latest'
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
namaste_py   latest    44ef94c791ae   2 days ago    895MB
cassandra    latest    132406477368   11 days ago   402MB
busybox      latest    c55b0f125dc6   2 weeks ago

Image History

Docker history command will gives you how this Docker image is build with what instructions. We can compare the the dockerfile content of any of the image with the docker image history command output.

$ docker image history --help
Usage:  docker image history [OPTIONS] IMAGE
Show the history of an image
Options:
      --format string   Pretty-print images using a Go template
  -H, --human           Print sizes and dates in human readable format (default true)
      --no-trunc        Don't truncate output
  -q, --quiet           Only show numeric IDs

Example for the docker image history

$ docker image history hello-world
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
fce289e99eb9        11 months ago       /bin/sh -c #(nop)  CMD ["/hello"]               0B                  
<  missing >         11 months ago       /bin/sh -c #(nop) COPY file:f77490f70ce51da2…   1.84kB

We can use the history subcommand with --format option to display only the "CREATED BY" column containing lines.

docker image history hello-world –format {{ .CreatedBy }}
  docker image history hello-world –format {{ .CreatedBy }}={{size}}

Example of Tomcat image format with CreatedBy

  docker image history tomcat –format {{ .CreatedBy }}

Tomcat docker image history

Docker Tomcat image history format with "CreatedBy"

Check more Docker commands executions :