DevOps Hunter: Docker Swarm - master workers : Docker Cluster Architecture

Friday, October 4, 2019

Docker Swarm - master workers : Docker Cluster Architecture

Docker Swarm

Docker swarm is one of the crucial components of the Docker ecosystem. Native Docker Clustering with Swarm gives us the ability to do scheduling, high availability, security, and platform scalability.
Kubernetes, fleet, Mesos work similar to achieve the same goal. They get layer abstraction for system resources and allows the interfaces to the cluster manager.

Docker swarm is NOT a plugin it is built-in into docker engine. A basic docker installation can run swarm cluster it does not require any plugin to install.

What is Docker Swarm Orchestration?

The docker swarm is a clustering and scheduling tool for Docker containers. With this docker swarm, DevOps operators and developers can establish and manage a cluster of Docker nodes as a single virtual system

A swarm is a group of nodes that are running on Docker daemon (doccker engine) with Swarm master and worker nodes which will be joined to form a containers cluster to provide HA in production environments.

Docker Swarm features

Cluster management: Use the Docker Engine CLI to create a swarm of Docker Engines where you can deploy application services

Declarative service model: Uses a declarative approach to let you define the desired state of the various services in your application stack

Scaling: When you scale up or down, the swarm manager automatically adapts by adding or removing tasks to maintain the desired state

Desired state reconciliation: Swarm manager node constantly monitors the cluster state and reconciles any differences between the actual state and your expressed desired state

Service discovery: Swarm manager nodes assign each service in the swarm a unique DNS name and load balances running containers

Rolling updates: At rollout time you can apply service updates to nodes incrementally. If anything goes wrong, you can roll-back a task to a previous version of the service

To work with Docker swarm, we need to enable it. To run proper docker swarm better run at least 3 docker clients VM with docker installed already. Make sure that all the VMs should be in the same timezone and also with the same date and time.

# IN MANAGER NODE docker swarm 'init' subcommand is used for creating a new docker swarm cluster. The node on which 'init' runs by default becomes a manager.

 docker swarm init --advertise-addr=192.168.33.110

docker swarm init command execution

# Confirm the manager node in the list

 docker node ls

# In the 192.168.33.111 box
# To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-5th0jl9mqq7fthcpabioyinoc1x109z3b6viahdilm2v841rwc-c8c17dj7stkld1zpwkxibt4zl 192.168.33.110:2377

docker swarm cluster joining commands execution

Note: If you did not have the manager's output. to get the token for the worker to join as follows:

docker swarm join-token worker

In the Manager node check the node list after joining the worker it should show 2 nodes information.

 docker node ls

docker Node list to check the swarm cluster

Note that we cannot run a node that made as 'Manager' cannot join the worker on the same node. Nodes, services, containers, and tasks

Service deploy in Swarm

Services are really just running “containers in production”
A service only runs one image, but it codifies the way that image runs—what ports it should use, how many replicas of the container should run so the service has the capacity it needs, and so on
Scaling a service changes the number of container instances running that piece of software

Service Creation

Create service across the docker swarm cluster

 docker service create --name webapp1 --replicas=4 --publish 8090:80 nginx

# List of services

 docker service ls

docker service with publish option command execution

# List of services running

 docker service ps webapp1

You can view the container running Nginx with a default page from any one of the node host IP address:

Running service in multiple replicas on Swarm cluster nodes

Scaling service

Scale up to 8 tasks

docker service scale web=8

Global mode service creation

docker service create --name test-redis --mode global redis

This will run a task of the service on every node in the Docker Swarm cluster check it with

docker service ls

Look at the changes in the network after swarm initiated

docker network ls

# Observed 2 new networks added docker_gwbridge, ingress

docker swarm - overlay network

Docker autolock is enabled at two commands: swarm init and swarm update The following is the docker command to enable autolock on an exists

root@dockerhost:~# docker swarm update --autolock=true
Swarm updated.
To unlock a swarm manager after it restarts, run the `docker swarm unlock`
command and provide the following key:

    SWMKEY-1-iWhW+8iF17n1C/2aPKcULTAwpi9pcmEsl2GKHtUwzhU

Please remember to store this key in a password manager, since without it you
will not be able to restart the manager.
root@dockerhost:~# vi swarmkey.txt

Default Overlay network

Remember this, 10.0.0.1/8 is the default address pool used by docker swarm for global scope overlay network

Previous Post for review:

GCP Cloud users note:

On Google Cloud if you are joining the swarm cluster please use the Private IP address of master in 'docker swarm join' command.
You can take out the node from the swarm cluster using 'docker swarm leave' command on the worker node.
When you try to access the new service port on the GCP open that on the firewall - tcp port for the same.

DevOps Hunter