Saturday, September 24, 2022

Scheduling Pods 1: Taint and Tolerance

 Node in the Kubernetes cluster are schedule the pods as per the Node level Taints will control the Pod creation on the Node. 


We can update the taints on one or more nodes with single command. The following are instructions from the kubectl label --help
  • A taint consists of a key, value, and effect. As an argument here, it is expressed as key=value:effect. 
  • The key must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to 253 characters. 
  • Optionally, the key can begin with a DNS subdomain prefix and a single '/', like example.com/my-app. 
  • The value is optional. If given, it must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to 63 characters.  
  • The effect must be NoSchedule, PreferNoSchedule or NoExecute. 
  • Currently taint can only apply to node.

How does Taint - Tolerance works in Kubernetes?

  • Taints allow a node to repel a set of pods.
  • Tolerations are applied to pods, and allow the pods to schedule onto nodes with matching taints.
  • Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes.
  • One or more taints are applied to a node; this marks that the node should not accept any pods that do not tolerate the taints.

How do you know how many worker nodes participating in the Kubernetes cluster?

kubectl get no
This list may includes the controlplane as well. 

How to check do any taints exist on a particular node say node01?

kubectl describe nodes/node01 |grep -i taint

Creating a taint on node01 with key of 'spray', value of 'mortein'and effect of NoSchedule.
kubectl taint nodes node01 spray=mortein:NoSchedule
# Verify the above
kubectl describe nodes/node01 |grep -i taint

Let's create a new pod with the nginx image and pod name as mosquito.
kubectl run mosquito --image=nginx
kubectl get po -w
Observe that pod name  mosquito status is in "Pending" continuously when pod trying to get inside, unable to enter into the node01 due to its spray taint. Here you can taint with any name that will stop pod to schedule it on a particular node.

What is the reason Pod is in pending state?

The main reason here is Pod mosquito connot tolerate taint 'Mortein' spray.

Let's create another pod named 'bamblebee' with the nginx image, which has a toleration set to the taint 'mortein'.

apiVersion: v1
kind: Pod
metadata:
  name: bamblebee
spec:
  containers:
  - image: nginx
    name: bee
  tolerations:
  - key: spray
    value: mortein
    effect: NoSchedule
    operator: Equal
  
Create with
kubectl apply -f toleration.yaml
kubectl get po -w
Noticed that the bee pod was scheduled on node node01 despite the taint. 

How do you check the ControlPlane is having any taints on it? 


It is simple, You can run the describe sub-command on the node/controlPlane
kubectl describe node controlplane
deafult following taint was used Taints: node-role.kubernetes.io/master:NoSchedule 

Remove from node 'controlplane' the taint with key 'node-role.kubernetes.io/master' and effect 'NoSchedule' if one of these key-value pair exists we can control the scheduling the pods.
kubectl taint nodes controlplane node-role.kubernetes.io/master:NoSchedule-

Most of the Pods in the real-time will be created under 'deployment' to have replication controllers the pods creation on the nodes. So most of the toleration declared in a section will be helpful.

Best practice is to take a back up of the original $HOME/.kube/config file then go for the overwrite the 

Kubernetes Labels - Selectors, Annotations

What this post covers?

In this post we will explore all possible options that can be used for Kubernetes Labels and selectors.
  • Adding labels
  • Show labels 
  • Replace labels
  • Deleting labels 

 Adding Labels to Pods

Adding labels while you creating bare pods imperatively using 'kubectl run' command, here I'm using three different images httpd, redis, rabbitmq.

 
kubectl run web --image=httpd:2.4.54-alpine \
  --labels="env=prod,author=pavandeverakonda,component=customer,tier=frontend"
kubectl run db --image=redis:alpine \
 --labels="env=prod,author=pavandeverakonda,component=customer,tier=backend"

kubectl run web2 --image=httpd:2.4.54-alpine \
 --labels="env=dev,author=pavandeverakonda,component=customer,tier=frontend"
kubectl run db2 --image=redis:alpine \
--labels="env=dev,author=pavandeverakonda,component=customer,tier=backend"

kubectl run msg-pod --image=rabbitmq \
 --labels="env=prod,author=ranjan,component=customer,tier=integrat"

#Validate the labels with the pods 
kubectl get po --show-labels
Create Pod along with Labels


Show labels for Kubernetes Objects

To see what all labels associated with a kubernetes object use --show-labels option
# for Pods
 kubectl get pods --show-labels 

# for nodes
kubectl get nodes --show-labels
To filter only the prod environment
 kubectl get pods -l env=prod --show-labels 
 
To get without header use --no-headers=true
 kubectl get pods -l env=dev --show-labels  --no-headers=true
 
To get the count of the dev environemnt pods we can use wc -l to count lines
 kubectl get pods -l env=dev --show-labels  --no-headers=true|wc -l
 

Using --selector option 


 To get how many pods present in the business unit(bu) as finance, here we can use --selector option
 kubectl get pods --selector bu=finance --no-headers=true --show-labels
 
To get the count of the pods in finance bu
 kubectl get pods --selector bu=finance --no-headers=true|wc -l
 
To find all objects are present in the prod environment including Pods, ReplicaSet and any other
kubectl get all --selector env=prod --show-labels  --no-headers=true
To get the count
 kubectl get all --selector env=prod  --no-headers=true|wc -l

To Identify the Pod which is part of the "prod" environment, the "finance" BU and of "frontend" tier?
kubectl get all --selector env=prod,bu=finance,tier=frontend  --no-headers=true
more specific to pods only display
kubectl get pod --selector env=prod,bu=finance,tier=frontend
# Only Pod names
kubectl get pod --selector env=prod,bu=finance,tier=frontend  --no-headers=true  -o NAME

Deleting labels

a. Edit the Kubernetes Object b. Using - operator to remove labels

Using - operator to remove labels

Update pod 'web' by removing a label named 'author' (if that label exists with that pod)
kubectl label pods web author-
Deleting a label from a Pod


You can also remove two or more labels at one go, removing env, tier labesl from the web pod:
kubectl label pods web env- tier- 
Deleting multiple labels from a Pod



Kubernetes Annotations Another intresting fact about Kubernetes Objects, they are like labels but they are just metadata not used in the selectors section.
kubectl annotate pod/web contact="admin@vybhavasolutions"
You can also add more number of annotationsto to your a Kubernetes objects anytime like this:
kubectl annotate pod/web whatsapp="9618715457" email="vybhavatechnologies@gmail.com"
viewing Annotations of kubernetes objects The first option is always using describe kubernetes object, where you could see a section for Annotations is available.
kubectl describe pods web -o yaml |grep -c 3 'annotations'
Keep writing to us for more intresting facts on Kubernetes related errors which you have encountered.

Friday, September 2, 2022

Understanding the Kubernetes Pods

Kubernetes pod 

The Kubernetes pod is going to represent a running process on the Kubernetes cluster.

It encapsulates an application container, storage resources, a unique network IP Address. Inside a pod, we can have one or more containers. The containers in a Pod are automatically co-located and co-scheduled on the same physical or virtual machine in the Kubernetes cluster.

A Multi container pod is going to be very tightly coupled inside. The containers share access to their memory space. They can be connected to each other using localhost. That is if a web application running on a container can be accessed in a busybox container running in the same pod. Containers in the pod can share storage(volumes) disk space as well.

What is a Pod in Kubernetes?

A pod is an atomic unit of scheduling in the Kubernetes ecosystem. Pods are ephemeral they live in a short time span.

Virtualization, Containerization, Pod



On the master node, the different Controllers are running for monitoring and self-healing of the pods. Containers in different pods have distinct IP addresses. And if the pod regenerated every time new IP address will be assigned to the Pod.


What does a pod do?

K8s Pods represent the processes running on a cluster. By limiting pods to a single process, Kubernetes can report on the health of each process running in the cluster. 
Pods have:
  • a unique IP address (which allows them to communicate with each other)
  • persistent storage volumes (as required)
  • configuration information that determines how a container should run.

Although most pods contain a single container, many will have a few containers that work closely together to execute the desired function

What are the benefits of a Pod?

=>When pods contain multiple containers, communications, and data sharing between them is simplified. Since all containers in a pod share the same network namespace, they can locate each other and communicate via localhost. Pods can communicate with each other by using another pods IP address or by referencing a resource that resides in another pod.


Pods can include containers that run when the pod is started, say to perform initiation required before the application containers run. Additionally, pods simplify scalability, enabling replica pods to be created and shut down automatically based on changes in demand

How does a pod work?

=>Pods are created by workload resources called controllers, which manage the rollout, replication, and health of pods in the cluster. For example, if a node in the cluster fails, a controller detects that the pods on that node are unresponsive and creates replacement pod(s) on other nodes.


The three most common types of controllers are:

  • Jobs for batch-type jobs that are ephemeral, and will run a task to completion
  • Deployments for applications that are stateless and persistent, such as web servers (HTTP servers)
  • StatefulSets for applications that are both stateful and persistent such as databases

If a pod has multiple containers, they are all scheduled together on the same server in the cluster, whether VM or physical server. All containers in the pod share their resources and dependencies and can coordinate their execution and termination. For example, pods can contain ‘init’ containers that run before the application containers run, setting up the environment for the applications that follow.


Pods are almost always created by controllers which then can automatically manage the pod lifecycle, including replacing failed pods, replicating pods when necessary, and evicting the pod from cluster nodes when they are complete or no longer needed.


Controllers use information in a pod template to create the pods, and controllers ensure that running pods match the deployment defined in the pod template, for example by creating replicas to match the number defined in the deployment.

How do pods communicate with each other?

=>When a pod is created it is assigned its own unique IP address. If there are multiple containers within the pod, they can communicate with each other simply by using localhost. Communications outside of the pod is achieved by exposing a port. Communications between pods in a cluster takes advantage of the fact that Kubernetes assigns a cluster-private IP address to every pid in a cluster, eliminating the need to either explicitly create links between pods or to map container ports to host ports. In this way, every pod in a cluster can ‘see’ each other without the need for NAT

6)what are the basic kubectl commands used for pods?
1) get
2) create pods
3) delete pods
4) get pods

What are Pod lifecycle states?

The pod life cycle states will be represented with the docker action as shown:

Kubernetes vs Docker
Kubernetes Pod Life Cycle


=>pending
=>running
=>succeeded
=>failed
=>unknown

downloading=>     blocked=>          pending=>      starting=>         ready=>       failed
   (docker image) (docker create)    (created)     (docker start)    (container up)    (exited)
--------------------------------------------------------------------------------------------------------------------



Kubernetes multi-container pods

=>Sidecar containers "help" the main container (may be web application or database). 
For example, log or data change watchers, monitoring adapters, and so on. 
A log watcher, for example, can be built once by a different team and reused across different applications. 

Another example of a sidecar container is a file or data loader that generates data for the main container.

Multi-container Pod



We can define a fresh pod-config in a YAML file sample file is given below:
#File: nginx-pod.yaml
apiVersion: v1
kind: pod
metadata:
  name: nginx-pod
  labels:
    app: nginx
    tier: dev
spec:
  container:
  -name: nginx-container
   image: nginx

Pod Create & Display commands 

1) To create a new pod we can use the kubectl create and then pass the pod definition YAML file. 
kubectl create -f nginx-pod.yaml 
  pod/nginx-pod created
  
Other options to Create a new pod with the nginx image using deployment or using run sub-command:
  kubectl create deployment mynginx -- image=nginx
  kubectl run nginx --image=nginx
  

2) How to check how many pods exists on the system?
To know how many pods running in the current default namespace. To check pod is RUNNING or failed  we can use the kubectl with  get sub-command on Kubernetes cluster running system to get pods status
kubectl get pods

* The READY column in the output indicates Running containers in Pod/Total containers in Pod. 3) To check IP address of the pod we can use the -o wide option
kubectl get pod -o wide 

4) To get the Pod definition in JSON or  YAML file we have to mention the output format as shown:
kubectl get pod nginx-pod -o yaml 

5) Display all details and events of a pod
kubectl describe pod nignx-pod 

6) Validating the pod network by pinging the container IP from the Kubernetes master node
ping 10.240.1.26 

7) To enter into the Pod/container we need to use exec sub-command like in docker we do. We can use sh or bash as per the image used for creation of container, following is the command to getting a shell to a RUNNING container
kubectl exec -it nginx-pod -- /bin/sh

8) To delete the pod use the delete sub-command
kubectl delete pod nginx-pod

9) We can use the get sub-command for many resources together only thing here we need to use comma to separate them. See the following command list one or more resources
kubectl get pods,deploy

10) Display detailed state of one single pod
kubectl describe pods [pod-name] 
11) To delete pod and services which have the label provided
kubectl delete pods,service -l name=[label-name]

When you delete the pod - wait till that termination completes. 12) To delete all pod resources from the default namespace
kubectl delete pods --all

13) Execute a command against a container in a pod.
kubectl exec [pod-name] date

14) If there are multiple container is running inside a Pod, we need to communicate or connect with single container the command is note (this command execute in master node)
kubectl exec [pod-name] -c [container-name] date 

15) The exec command with -i indicates interactive and -t indicates terminal for the container inside the container
kubectl1 exec -ti [pod-name] /bin/bash 

How to Look into the Pod logs

Kubernetes maintains a logger in the ecosystem. To accessing the Kubernetes pod logs  there are two log levels are available:

  • Node-level
  • Cluster-level



print the logs for a container in a pod
 
kubectl logs [pod-name] 
for example: using nginx pod logs.

Categories

Kubernetes (24) Docker (20) git (13) Jenkins (12) AWS (7) Jenkins CI (5) Vagrant (5) K8s (4) VirtualBox (4) CentOS7 (3) docker registry (3) docker-ee (3) ucp (3) Jenkins Automation (2) Jenkins Master Slave (2) Jenkins Project (2) containers (2) create deployment (2) docker EE (2) docker private registry (2) dockers (2) dtr (2) kubeadm (2) kubectl (2) kubelet (2) openssl (2) Alert Manager CLI (1) AlertManager (1) Apache Maven (1) Best DevOps interview questions (1) CentOS (1) Container as a Service (1) DevOps Interview Questions (1) Docker 19 CE on Ubuntu 19.04 (1) Docker Tutorial (1) Docker UCP (1) Docker installation on Ubunutu (1) Docker interview questions (1) Docker on PowerShell (1) Docker on Windows (1) Docker version (1) Docker-ee installation on CentOS (1) DockerHub (1) Features of DTR (1) Fedora (1) Freestyle Project (1) Git Install on CentOS (1) Git Install on Oracle Linux (1) Git Install on RHEL (1) Git Source based installation (1) Git line ending setup (1) Git migration (1) Grafana on Windows (1) Install DTR (1) Install Docker on Windows Server (1) Install Maven on CentOS (1) Issues (1) Jenkins CI server on AWS instance (1) Jenkins First Job (1) Jenkins Installation on CentOS7 (1) Jenkins Master (1) Jenkins automatic build (1) Jenkins installation on Ubuntu 18.04 (1) Jenkins integration with GitHub server (1) Jenkins on AWS Ubuntu (1) Kubernetes Cluster provisioning (1) Kubernetes interview questions (1) Kuberntes Installation (1) Maven (1) Maven installation on Unix (1) Operations interview Questions (1) Oracle Linux (1) Personal access tokens on GitHub (1) Problem in Docker (1) Prometheus (1) Prometheus CLI (1) RHEL (1) SCM (1) SCM Poll (1) SRE interview questions (1) Troubleshooting (1) Uninstall Git (1) Uninstall Git on CentOS7 (1) Universal Control Plane (1) Vagrantfile (1) amtool (1) aws IAM Role (1) aws policy (1) caas (1) chef installation (1) create organization on UCP (1) create team on UCP (1) docker CE (1) docker UCP console (1) docker command line (1) docker commands (1) docker community edition (1) docker container (1) docker editions (1) docker enterprise edition (1) docker enterprise edition deep dive (1) docker for windows (1) docker hub (1) docker installation (1) docker node (1) docker releases (1) docker secure registry (1) docker service (1) docker swarm init (1) docker swarm join (1) docker trusted registry (1) elasticBeanStalk (1) global configurations (1) helm installation issue (1) mvn (1) namespaces (1) promtool (1) service creation (1) slack (1)