Kubernetes cluster configuration in a Virtualbox with vagrant

Thanks to Rajkumar who had developed the Vagrantfile and published in the github on Kubernetes cluster configuration in a Virtualbox with vagrant. For those who don't know about Vagrant it is a tool that will be used for virtualization into a different level and more powerful way of using your system resources to run multiple operating virtual boxes in your Laptop/Desktop systems.

You just need to follow the simple steps which I had done in my experiment:

Prerequisites for Kubernetes Cluster Creation

1. Download latest Vagrant
2. Download latest version of Oracle VirtualBox

System resources requirements on VirtualBox

• 2 GB for each node
• 2 cores CPUs for each node

Here I have don this expeiment on my Windows 7 laptop. You could do same on any Windows higher version as well. Total 3 VMs will be created under a group named as - "Kubernetes Cluster" as defined in Vagrantfile.

Infrastructure as a Code: Vagrantfile 

The Vagrantfile will be composed with the Ruby array that creates k8s-head and k8s-node1, k8s-node2 definitions. Once the Ubuntu Xenial boxes provisioned custom shell scripts are used for boot time execution.

• Both Master, Slave nodes common tasks are executed with the Shell provisioning inline options.
• Install Docker CE 17.03
• Added vagrant user to docker group to run docker commands as vagrant user (without using sudo for each not required)
• Install the kubelet kubeadm kubectlk
• kubelet requires swap off

You can do all the setups required to run the following in the sequence : 

• k8s-master node runs on 192.168.33.10
• k8s-slave1 node runs on 192.168.33.11
• k8s-slave2 node runs on 192.168.33.12

Bootstrap Setup

Master node will be required the following steps

Slave node will be running and after bootup only runs inline joining the kubernetes cluster with a script generated in the master. node.

Executing the setup

vagrant up

check the VM are created as expected

vagrant status

Vagrant status of kuberenetes cluster

Check that all are in running state, if not you need to check the log file that is generated in the same path where Vagrantfile exists.

Connect with your PuTTY to k8s-master that is running on 192.168.33.10 IP address.

Check the versions of kubeadm, kubectl, and kubelet

  kubectl version
  kubeadm version
  # Better format output
  kubectl version -o yaml
  kubeadm version -o yaml
  

Kubeadm, kubectl, kubelet versions

Check the nodes list

kubectl get nodes

kubectl get nodes output

Note: Make sure that your Windows firewall disabled to run the Vagrant on your Windows laptop.

You might be more interested to explore and know about the latest Docker 19 Community Edition learning experiments on Ubuntu 19.04

References:

1. Kubernetes Cluster on GitHub

Docker Container Concepts

Container Concepts: Steps for Automation

Hey! guys this post is about the containers specific commands examining each and the best outputs are collected. Hope this could help you in preparing automation in docker pipeline CI/CD scripts, and you feel enjoy this exploring post helpful!

Docker is a platform for developers and sysadmin to develop, deploy and run applications with containers. The use of Linux containers to deploy applications is called containerization.

Docker Container Concepts

Containerization is increasingly popular because containers are :

• Flexible: Even the most complex applications can be containerized.
• Lightweight: Containers leverage and share the host kernel.
• Interchangeable: You can deploy updates and upgrades on-the-fly.
• Portable: You can build locally, deploy to the cloud, and run anywhere.
• Scalable: You can increase and automatically distribute container replicas.
• Stackable: You can stack services vertically and on-the-fly.

How Images and containers are connected?

An image is an executable package that includes everything needed to run an application--the code, a runtime, libraries, environment variables, and configuration files.

A container is a runtime instance of an image--what the image becomes in memory when executed (that is, an image with state, or a user process). You can see a list of your running containers with the command, docker ps

Differences in Containers and virtual machines

A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.
By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtual access to host resources through a hypervisor. In general, VMs provide an environment with more resources than most applications need.

Docker Container Deep Dive

Now let's start exploring the commands about the container specific options from the creation to listing to controlling them to saving to removing from the docker host.

Docker Container related command Help

To learn any CLI tool best way is to check the manual or help option. So lets see more options realted to container.

docker container --help

docker container command help

How can I see the docker Container Size?

Get the Container Size with two options -s or --size sample following:

Docker Container Size

How to create containers?

We can create a container using 'container create' command, which internally calls 'docker pull' if the image is in not available on that docker engine.

docker container create [OPTIONS] IMAGE [COMMAND] [ARG...]

Here note that 'create' command will not run the any container, it's status would be 'Created' state.

docker container create --name test-nginx --interactive --tty nginx

docker create examples

Now let's check the Containers status:
Here we can select the columns which are required to fetch the container details.

docker ps --all --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Status}}"

docker container list with custom format

You can create the container and attach it to STDIN, STDOUT or STDERR in the following example showing that a tomcat container created and then attach to STDIN

Container Create command execution

How to start the Docker Container?

Any container which is just created state that could be started using container start command.To start one or more stopped containers we can use the following :

docker start [OPTIONS] CONTAINER [CONTAINER...]

docker start modest_shannon

How to Stop the Container?

Similar to the start subcommand  we have stop The following command to stop a running container

docker container stop test-nginx

If there is no issues, this is will returns the name of the container or container id whatever you pass as argument.

How to Restart the Docker container?

This will stop the running container and start the container then the status of the container will be in 'Up'.

docker container restart test-nginx

docker container create, start, stop, restart 

How to remove/Delete a docker Container? 

The docker container rm command will be used to remove the container from the docker engine. If the container is running status then it will refuse to remove it. So before you remove the container ensure that is stopped means Status should be Exited. We can delete one or more containers at a time as :

docker container rm my_container 

Kill Containers

As situation demands that if the docker container crashed due to internal resource problems such as insufficient memory or load for execution is in uncontrolled then container might fail to serve. In such cases we can use docker container  kill  command will cleanup those jambe container, we can also remove forcefully using kill subcommand all running containers.

docker container kill $(docker ps -q)  

One command to sweep/Delete all containers that are not running on the docker engine.

docker container rm $(docker ps -a -q)

How to Inspect a docker container?

To look for the storage driver, volume details or network ip address and lots of information.

docker container inspect C1_web

Docker container list 

List of containers can be output with the aliases: ls, ps, list That means all the three commands will give the same output.

docker container ls
docker container ps
docker container list

Container list command output

Docker attach and detach containers

Allows one terminal to attach a running container. It allows you to connect to the process' STDIO in another terminal.

From the Docker docs following topper Example: Let's run in detach mode OracleLinux container, where inside the container by running top command with -b that is infinite.

$ ID=$(sudo docker run -d oraclelinux /usr/bin/top -b)
Unable to find image 'oraclelinux:latest' locally
latest: Pulling from library/oraclelinux
04e172e76262: Pulling fs layer
04e172e76262: Verifying Checksum
04e172e76262: Download complete
04e172e76262: Pull complete
Digest: sha256:27c0c3e5ca8ce6e9ef8121c106de2d8ad28067074d7f5f5ce4e5cb78a0f07b3d
Status: Downloaded newer image for oraclelinux:latest
[node1] (local) root@192.168.0.23 ~
$ docker attach $ID


top - 05:31:27 up 15 days,  2:52,  0 users,  load average: 5.67, 6.12, 4.91
Tasks:   1 total,   1 running,   0 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.0 us, 22.3 sy,  0.0 ni, 64.6 id,  0.1 wa,  0.0 hi,  1.1 si,  0.0 st
KiB Mem : 32929708 total,  7670004 free,  9502344 used, 15757360 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 21260512 avail Mem 

[node1] (local) root@192.168.0.23 ~
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
e29921306526        oraclelinux         "/usr/bin/top -b"   7 seconds ago       Up 6 seconds                            eloquent_wescoff

Well, Ctrl+C (or Ctrl+\) should detach you from the container but it will kill the container because your main process is a bash or bach command.

docker run -ti -d --name test python:3.6 /bin/bash -c 'while [ 1 ]; do sleep 30; done;'
 docker container ls
 

Let's attach to the test container

  docker attach test
  

Type Ctrl+p, Ctrl+q will help you to turn interactive mode to daemon mode.
Typical detach keys sequence examples here.

docker attach --detach-keys="ctrl-a"  test 

Please note here that escape sequence ^P^Q does work, BUT only when -t and -i is used to launching the container.

For example:

applications have no system dependencies
updates can be pushed to any part of a distributed application
resource density can be optimized.

With Docker, scaling your application is a matter of spinning up new executables

further links you could refer

Docker - Post install Tasks

Docker - Post-install steps

Manage Docker as a non-root user

Docker daemon binds to a Linux socket instead of a TCP port. By default that Unix socket is owned by the user "root" and other users can only access it using sudo. The docker daemon always runs as the root user.

To avoid using sudo , create a Unix group called docker and add users to it. When the Docker daemon starts, it creates a Unix socket accessible by members of the docker group.

The docker group grants privileges equivalent to the root user.

To create the docker group and add your user:

1. Create the docker group

sudo groupadd docker

2. Add your user to the docker group

 
sudo usermod -aG docker $USER

3.Logout and log back

4.Verify that you can run docker commands without sudo:

docker run hello-world

If you initially ran Docker CLI commands using sudo before adding your user to the docker group, you may see the following error, which indicates that your ~/.docker/ directory was created with incorrect permissions due to the sudo commands.

WARNING: Error loading config file: /home/user/.docker/config.json -
stat /home/user/.docker/config.json: permission denied
To fix this problem, either remove the ~/.docker/ directory (it is recreated automatically, but any custom settings are lost), or change its ownership and permissions using the following commands:

 
sudo chown "$USER":"$USER" /home/"$USER"/.docker -R
sudo chmod g+rwx "$HOME/.docker" -R

Configure Docker to start on boot

Most current Linux distributions (RHEL, CentOS, Fedora, Ubuntu 16.04 and higher) use systemd to manage which services start when the system boots. Ubuntu 14.10 and below use upstart.

 
systemd

 sudo systemctl enable docker

To disable this behavior, use disable instead.

 sudo systemctl disable docker

upstart
Docker is automatically configured to start on boot using upstart. To disable this behavior, use the following command:

 
echo manual | sudo tee /etc/init/docker.override
chkconfig 
sudo chkconfig docker on

Docker CE Installation on CentOS

Docker CE Installation on CentOS:

For this minimum required version of CentOS 7 required for Docker and enabled with the centos-extras repository.

Uninstall old versions of docker if any 

Docker or docker-engine old versions along with associated dependencies should be uninstalled before proceeding for new installation:

$ sudo yum remove docker \
                  docker-client\
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-selinux \
                  docker-engine-selinux \
                  docker-engine


The contents of /var/lib/docker, including images, containers,volumes, and networks are preserved.

Installing Docker CE:

There are different ways of Docker CE installations:

1. Docker's repositories - recommended approach for easy upgrades
2. Download RPM package and instanll manually - if no internet access to the machine then it is recommended an approach
3. Use automated convenience scripts which docker provides.

To avoid disk space issues:

1. Login as root. On some hosted machines, sudo is "/usr/local/bin/sudo".
2. To avoid filling up, your / directory ( To avoid space issues)

• Remove /var/lib/docker directory.
• Create a new location for docker files, e.g. /u01/docker-env/docker
• sudo ln -s /u01/docker-env/docker /var/lib/docker

Alternatively,

• You can configure docker directory in /etc/sysconfig/docker using -g option

• # /etc/sysconfig/docker
OPTIONS='-g /u01/docker'



Install using repository:
=================

Setup the Docker repository in the machine before proceeding with the installation

Set up the repository

1. Install requires packages : yum-utils provides the yum-config-manager utility, and device-mapper-persistent-data and lvm2 are required by the devicemapper storage driver.

$sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2

2.For stable build installer use the below command:

$sudo yum-config-manager \
 --add-repo \
https://download.docker.com/linux/centos/docker-ce.repo

1. To install the latest version of docker CE

$sudo yum install docker-ce

If prompted to accept the GPE key verify that the fingerprint matches 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35, and if so, accept it.

Docker is installed but not started. The docker group is created, but no users are added to the group.

2. To install a specific version of Docker CE , list the available versions in the repo, then select and install:

a. List and sort the versions available in your repo. This example sorts results by version number , highest to lowest and is truncated:

$yum list docker-ce --showduplicates | sort -r

docker-ce.x86_64            18.03.0.ce-1.el7.centos             docker-ce-stable


$ sudo yum install docker-ce-<version string>

3. start Docker :

$sudo systemctl start docker

4. Verify that docker is installed correctly by running the hello-world image

$sudo docker run hello-world

The command downloads a test image and runs it in a container.

Docker Ce is intalled and running, you need to use sudo to run docker commands.

Install from a package:

1.Download .rpm file from the url : https://download.docker.com/linux/centos/7/x86_64/stable/Packages/

2.Install Docker CE

$sudo yum install /path/to/package.rpm

3.start Docker

$ sudo systemctl start docker

4.verify the docker is installer correctly by running the hello-world image

$sudo docker run hello-world

This command downloads a latest image and runs it in a container.

Upgrade docker CE:
Download newer package file and release installation procedure using yum -y upgrade instead of yum -y install.

Docker Basic commands

docker --version # to check docker version
docker version # detailed version
docker info # to see more details about docker installation

Docker Basic check for Version Installation

Uninstall Docker CE:
1. uninstall the Docker package:

$sudo yum remove docker-ce

2. Image, containers, volumes, or customized configuration files on your host are not automatically removed. To delete all images containers and volumes:

$ sudo rm -rf /var/lib/docker

Overview of Docker Editions

In this post, we would like to discuss on the different flavours currently available in docker.io site. When you start exploring things at the top level to design the infrastructure for a project that has microservice architecture and needs to be scaled in future, this would give some insights to make a decision on what to choose why.

docker editions in details

Docker is available in two editions:

• Community Edition (CE)
• Enterprise Edition (EE)

Docker Community Edition (CE) is ideal for individual developers and small teams looking to get started with Docker and experimenting with container-based apps.

Docker Enterprise Edition (EE) is designed for enterprise development and IT teams who build, ship and run business-critical applications in production at scale.

About Docker Community Edition (CE):

Docker Community Edition (CE) is ideal for developers and small teams looking to get started with Docker and experimenting with container-based apps. Docker CE has three types of update channels, stable, test, and nightly:

• Stable gives you the latest releases for general availability.
• Test gives pre-releases that are ready for testing before general availability.
• Nightly gives you latest builds of work in progress for the next major release.

Support

Docker CE releases of a year-month branch are supported with patches as needed for 7 months after the first year-month 

About Docker EE

Docker Enterprise Edition (Docker EE) is designed for enterprise development and IT teams who build, ship, and run business-critical applications in production and at scale. Docker EE is integrated, certified, and supported to provide enterprises with the most secure container platform in the industry. For more info about Docker EE.

There are currently two versions of Docker EE Engine available:

• 18.03 - Use this version if you’re only running Docker EE Engine.
• 17.06 - Use this version if you’re using Docker Enterprise Edition 2.0 (Docker Engine, UCP, and DTR).

In Enterprise edition we have three additional editions available:

1.Enterprise Edition Basic

2.Enterprise Edition Standard

3.Enterprise Edition Advanced

EE Basic:

With Docker EE Basic, you can deploy Docker Enterprise Engine to manage your container workloads in a flexible way. You can manage workloads on Windows, Linux, on-premise or on the cloud.

Docker EE Basic has enterprise-class support with defined SLAs, extended maintenance cycles for patches for up to 24 months.

EE Standard:

Docker EE Standard has everything the Basic edition has, and extends it with private image management, integrated image signing policies, and cluster management with support for Kubernetes and Swarm orchestrators.

EE Advanced:

Docker EE Advanced takes this one step further and allows you to implement node-based RBAC policies, image promotion policies, image mirroring, and scan your images for vulnerabilities.

Support :

Each Docker EE release is supported and maintained for 24 months, and receives security and critical bug fixes during this period.

Docker Certified Cloud Infrastructure

This is a critical section where you do the options to choose according to the facilities : 

• Infrastructure based cloud supported :
• VMware
• AWS 
• Microsoft Azure 
• IBM cloud (coming soon)
• Software as a service
• Oracle Cloud industry 

Reference docs : https://docs.docker.com/ee/supported-platforms/#supported-platforms

Docker Namespaces - The Containerization Platform

DOCKER the containerization Platform

Docker is open-source software that makes it easier to build, deploy and run the application using LXC (Linux Containers).

LXC 

LXC was developed based on cgroups Linux kernel feature(released in kernel version 2.6.24) which is an operating system-level virtualization. On a single Linux host, multiple isolated Linux containers can run by sharing the OS resources independently.

Cgroups

Cgroups (control groups) is a Linux kernel feature that limits, accounts and isolates the resource usage (CPU, memory, disk I/O, Network, and so on) of a collection of processes.

Docker Namespaces

Namespaces is a kernel feature that partitions the kernel so that one set of processes can see one set of resources including processes trees, networking, userid, and mount file system. Each process is associated with namespaces can only see or use the resources associated with that namespace only.

Docker Engine uses namespaces such as the following on Linux:

• The pid namespace: Process isolation (PID: Process ID).
• The net namespace: Managing network interfaces (NET: Networking).
• The ipc namespace: Managing access to IPC resources (IPC: InterProcess Communication).
• The mnt namespace: Managing filesystem mount points (MNT: Mount).
• The uts namespace: Isolating kernel and version identifiers. (UTS: Unix Timesharing System).

Docker uses the Linux namespaces in combination with cgroups to isolate their processes.

Union file systems

Union file systems, or UnionFS, are file systems that operate by creating layers, making them very lightweight and fast. Docker Engine uses UnionFS to provide the building blocks for containers. Docker Engine can use multiple UnionFS variants, including AUFS, btrfs, vfs, and DeviceMapper.

Container format

Docker Engine combines the namespaces, control groups, and UnionFS into a wrapper called a container format. The default container format is libcontainer.

Container

Containers are a method of operating system virtualization that allows running the application and its dependencies in resource isolated processes. Linux containers contain applications in a way that keep them isolated from the host system that they run on. Application packaged in one Linux container on a particular flavor of OS can run on a different flavor of OS along with its dependency software.
For example, the container image created on UBUNTU OS can run on CentOS.

Docker Image

Image is an instance of the container. Image is a set of layers of your application as you described If we start this image, we have a running container of this image. We can have many running containers of the same image.

OS = kernel + filesystem/libraries
Image = filesystem/libraries

Virtual machines vs Containers

Virtual machines are created on a bare metal machine with the help of a hypervisor - which is a software/firmware that sits between the hardware and VM is necessary to virtualize the servers.

Virtual machines runs on guest os, the different os can run on the same physical machine. Every VM has its own binaries, libraries, and applications that it services and may be in different disk sizes.

Server virtualization benefits in utilizing the machine effectively, it will provide the ability to consolidate application on to the single machine. where as it has a disadvantage that each VM includes a separate OS. The application developed or deployed in one VM may not work in other VMs without any modifications.

So portability of application is difficult with VMs which helps in popularize container.

The container provides a way to run applications on a single server or host OS. Containers resides on a physical server and its host OS for example, Linux or Windows.
Each container shares the host OS kernel and, usually, the binaries and libraries, too. Shared components are read-only. Containers are thus exceptionally “light”—they are only megabytes in size and take just seconds to start, versus gigabytes and minutes for a VM.


Virtual machines and containers differ in several ways, but the primary difference is that containers provide a way to virtualize an OS so that multiple workloads can run on a single OS instance. With VMs, the hardware is being virtualized to run multiple OS instances. Containers’ speed, agility, and portability make them yet another tool to help streamline software development.

Finally, What is Docker?

Docker is a software program that performs operating system virtualization also known as Containerization. Docker is developed by Docker, Inc.

Docker uses the resource isolation features of the Linux kernel such as cgroups and kernel namespaces, and a union-capable file system such as OverlayFS to allow independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.

Docker includes the libcontainer library as its own way to directly use virtualization facilities provided by the Linux kernel, in addition to using abstracted virtualization interfaces via libvirt, LXC, and systemd-nspawn.

Who developed Docker?

Solomon Hykes started Docker in France as an internal project within dotCloud, a platform-as-a-service company.
The software debuted to the public in Santa Clara at PyCon in 2013.

Docker was released as open-source in March 2013. On March 13, 2014, with the release of version 0.9, Docker dropped LXC as the default execution environment and replaced it with its own libcontainer library written in the Go programming language

Now its time to know how to install docker? Different installation types? What are the flavours of docker available? Where we get support? ..... to continue in the next article.


Don't forget to add your comments below....