Sunday, April 10, 2022

Job & CronJob - Batch Job

What is Job object in Kubernetes?


A Job object will be used to create one or more Pods and the Job ensures that a specified number of Pod instances will be created and terminates after completion of the Job. There could be finite jobs which will run within given certain timeout values. Job tracks for 'Successful' completion of the required task. Jobs can be run in two varients they can be parallel and also non-parallel.

Kubernetes Job types



There are 3 types of jobs non-parallel jobs [single pod jobs - unless it fails. creates replacement pod when pod goes down] parallel jobs with a fixed completion count parallel jobs with task queue 

##Example type 1: hundred-fibonaccis.yml
---
apiVersion: batch/v1
kind: Job
metadata:
    name: fibo-100
spec:
  template:
    spec: 
      containers:
      - name: fib-container 
        image: truek8s/hundred-fibonaccis:1.0
      restartPolicy: OnFailure
  backoffLimit: 3
Create the Job:
kubectl create -f hundred-fibonaccis.yml
Now let's describe the job:
kubectl describe job fibo-100
Describing a Job in Kubernetes



In the Job description you can observe the attributes such as parallelism, Pod Statuses.
 
On the other terminal which is running in parallel, Observe the pod status change from Running to Completed:
kubectl get po -w 
Pod still exists to get the logs
kubectl logs [podname] -- [container] 
We can see the Fibonacci number series printed out from the container logs.
Fibonacci series printed from kubctl logs command



##Example type 2
---
apiVersion: batch/v1
kind: Job
metadata:
  name: counter
spec:
  template:
    metadata:
      name: count-pod-template
    spec:
      containers:
      - name: count-container
        image: alpine
        command:
         - "/bin/sh"
         - "-c"
         - "for i in 100 50 10 5 1; do echo $i; done"
      restartPolicy: Never
To create the counter-job use the following command :
kubectl create -f counter-job.yaml
Check Job list:
kubectl get jobs
Check the Pod:
kubectl get po 
Check the log:
kubectl logs count-pod-xxx 
Counter Job execution output using kubectl logs



Now let's Describe the job counter :
kubectl describe jobs counter 
You can observer the Start Time and Pod Statuses from the above command ##cleanup delete job:
kubectl delete jobs counter-job 
no need to delete pod, When a Job deleted automatically removes the corresponding all its pods.

Controlling Job Completions

In some situations you need to run the same job multiple times, we need to define completions: number it under Job-> Specifications (spec section)
  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  completions: 2
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["echo", "hello Kubernetes job!!!"]
       restartPolicy: Never
  
Observe the number of the Completions on the
watch kubectl get all
  kubectl get pods
  kubectl delete job hello-job
  kubectl get pods 
Once Job is deleted all its relavant resource will be cleaned up automaticall.

Parallelism

In some project there will be need to run multiple pods running in parallel
  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  parallelism: 2
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["echo", "hello Kubernetes job!!!"]
       restartPolicy: Never
  
Observe the number of the Completions on the How backoffLimit works on Job? Let's do simple exeriment and understand this 'backoffLimit' attribute.
  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  parallelism: 2
  backoffLiit: 4
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["ech0o", "hello Kubernetes job!!!"]
       restartPolicy: Never
  
Observe command mistyped purposefully, which will fail to create new Pod.

Working with CronJob


Job that works like a crontab in Linux systems. Any task that needs to be executed based on the scheduled time then we can use this Kubernetes Object.

##Example CronJob
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello-cron
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: busybox-container
            image: busybox
            command: ["echo", "Namste Kubernetes Cronjob!!!"]
            startPolicy: OnFailure
Creating
kubectl create -f cronjob.yaml
Open in new terminal and run the following command to watch continuously :
watch kubectl get all
Check the pods section in the above output

No comments:

Categories

Kubernetes (24) Docker (20) git (13) Jenkins (12) AWS (7) Jenkins CI (5) Vagrant (5) K8s (4) VirtualBox (4) CentOS7 (3) docker registry (3) docker-ee (3) ucp (3) Jenkins Automation (2) Jenkins Master Slave (2) Jenkins Project (2) containers (2) create deployment (2) docker EE (2) docker private registry (2) dockers (2) dtr (2) kubeadm (2) kubectl (2) kubelet (2) openssl (2) Alert Manager CLI (1) AlertManager (1) Apache Maven (1) Best DevOps interview questions (1) CentOS (1) Container as a Service (1) DevOps Interview Questions (1) Docker 19 CE on Ubuntu 19.04 (1) Docker Tutorial (1) Docker UCP (1) Docker installation on Ubunutu (1) Docker interview questions (1) Docker on PowerShell (1) Docker on Windows (1) Docker version (1) Docker-ee installation on CentOS (1) DockerHub (1) Features of DTR (1) Fedora (1) Freestyle Project (1) Git Install on CentOS (1) Git Install on Oracle Linux (1) Git Install on RHEL (1) Git Source based installation (1) Git line ending setup (1) Git migration (1) Grafana on Windows (1) Install DTR (1) Install Docker on Windows Server (1) Install Maven on CentOS (1) Issues (1) Jenkins CI server on AWS instance (1) Jenkins First Job (1) Jenkins Installation on CentOS7 (1) Jenkins Master (1) Jenkins automatic build (1) Jenkins installation on Ubuntu 18.04 (1) Jenkins integration with GitHub server (1) Jenkins on AWS Ubuntu (1) Kubernetes Cluster provisioning (1) Kubernetes interview questions (1) Kuberntes Installation (1) Maven (1) Maven installation on Unix (1) Operations interview Questions (1) Oracle Linux (1) Personal access tokens on GitHub (1) Problem in Docker (1) Prometheus (1) Prometheus CLI (1) RHEL (1) SCM (1) SCM Poll (1) SRE interview questions (1) Troubleshooting (1) Uninstall Git (1) Uninstall Git on CentOS7 (1) Universal Control Plane (1) Vagrantfile (1) amtool (1) aws IAM Role (1) aws policy (1) caas (1) chef installation (1) create organization on UCP (1) create team on UCP (1) docker CE (1) docker UCP console (1) docker command line (1) docker commands (1) docker community edition (1) docker container (1) docker editions (1) docker enterprise edition (1) docker enterprise edition deep dive (1) docker for windows (1) docker hub (1) docker installation (1) docker node (1) docker releases (1) docker secure registry (1) docker service (1) docker swarm init (1) docker swarm join (1) docker trusted registry (1) elasticBeanStalk (1) global configurations (1) helm installation issue (1) mvn (1) namespaces (1) promtool (1) service creation (1) slack (1)