Job & CronJob - Batch Job

What is Job object in Kubernetes?

A Job object will be used to create one or more Pods and the Job ensures that a specified number of Pod instances will be created and terminates after completion of the Job. There could be finite jobs which will run within given certain timeout values. Job tracks for 'Successful' completion of the required task. Jobs can be run in two varients they can be parallel and also non-parallel.

Kubernetes Job types

There are 3 types of jobs non-parallel jobs [single pod jobs - unless it fails. creates replacement pod when pod goes down] parallel jobs with a fixed completion count parallel jobs with task queue 

##Example type 1: hundred-fibonaccis.yml

---
apiVersion: batch/v1
kind: Job
metadata:
    name: fibo-100
spec:
  template:
    spec: 
      containers:
      - name: fib-container 
        image: truek8s/hundred-fibonaccis:1.0
      restartPolicy: OnFailure
  backoffLimit: 3

Create the Job:

kubectl create -f hundred-fibonaccis.yml

Now let's describe the job:

kubectl describe job fibo-100

Describing a Job in Kubernetes

In the Job description you can observe the attributes such as parallelism, Pod Statuses.

 
On the other terminal which is running in parallel, Observe the pod status change from Running to Completed:

kubectl get po -w 

Pod still exists to get the logs

kubectl logs [podname] -- [container] 

We can see the Fibonacci number series printed out from the container logs.

Fibonacci series printed from kubctl logs command

##Example type 2

---
apiVersion: batch/v1
kind: Job
metadata:
  name: counter
spec:
  template:
    metadata:
      name: count-pod-template
    spec:
      containers:
      - name: count-container
        image: alpine
        command:
         - "/bin/sh"
         - "-c"
         - "for i in 100 50 10 5 1; do echo $i; done"
      restartPolicy: Never

To create the counter-job use the following command :

kubectl create -f counter-job.yaml

Check Job list:

kubectl get jobs

Check the Pod:

kubectl get po 

Check the log:

kubectl logs count-pod-xxx 

Counter Job execution output using kubectl logs

Now let's Describe the job counter :

kubectl describe jobs counter 

You can observer the Start Time and Pod Statuses from the above command ##cleanup delete job:

kubectl delete jobs counter-job 

no need to delete pod, When a Job deleted automatically removes the corresponding all its pods.

Controlling Job Completions

In some situations you need to run the same job multiple times, we need to define completions: number it under Job-> Specifications (spec section)

  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  completions: 2
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["echo", "hello Kubernetes job!!!"]
       restartPolicy: Never
  

Observe the number of the Completions on the

watch kubectl get all
  kubectl get pods
  kubectl delete job hello-job
  kubectl get pods 

Once Job is deleted all its relavant resource will be cleaned up automaticall.

Parallelism

In some project there will be need to run multiple pods running in parallel

  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  parallelism: 2
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["echo", "hello Kubernetes job!!!"]
       restartPolicy: Never
  

Observe the number of the Completions on the How backoffLimit works on Job? Let's do simple exeriment and understand this 'backoffLimit' attribute.

  ---
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  parallelism: 2
  backoffLiit: 4
  template:
    spec:
      containers:
        - name: busybox-container
          image: busybox
          command: ["ech0o", "hello Kubernetes job!!!"]
       restartPolicy: Never
  

Observe command mistyped purposefully, which will fail to create new Pod.

Working with CronJob

Job that works like a crontab in Linux systems. Any task that needs to be executed based on the scheduled time then we can use this Kubernetes Object.

##Example CronJob

---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello-cron
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: busybox-container
            image: busybox
            command: ["echo", "Namste Kubernetes Cronjob!!!"]
            startPolicy: OnFailure

Creating

kubectl create -f cronjob.yaml

Open in new terminal and run the following command to watch continuously :

watch kubectl get all

Check the pods section in the above output

ConfigMaps in Kubernetes

Hello guys, welcome back to my DevSecOps learning posts, Today I have a new Kubernetes object  Config Maps, This is like any other object we can create, get, delete and describe ConfigMaps. Just same as our Perl, Python, and Java languages support 'Hashmaps' to store the collection of data items. Here in t Kubernetes, we can use a map during the deployment of Pod/Container.

A ConfigMap object can be used to store the configuration data items. This could be a set of properties required inside the application that runs inside a Container. it could be an environment variable which can be a ref to a key that has a value defined in the ConfigMap.

What is ConfigMap in Kubernetes?

ConfigMaps hold configuration in key-value pairs which are accessed by Pod containers Similar to the Linux /etc configuratons 

ConfigMaps can hold Properties files of an application – For Example Java applications hibernate.properties log4j.properties, logging.conf

The entire configuration file can be used from ConfigMap

We can also add JSON files as ConfigMap

Once ConfirMap is created, Kubernetes objects are ready for injecting in running containers to get the new configuration changes.

ConfigMap used in Kubernetes Pods

Creating ConfigMaps

The following is the syntax for the creating the ConfigMaps

  kubectl create configmap [mapname] [data-source]
  kubectl create configmap [configmapName] [--from-file file_path]|[--from-literal key=value]
  

Here the data-source ConfigMaps can be created in three ways:

• directories
• Files
• Literal Values

Let's start exploring about ConfigMaps, To check ConfigMap have a short name and support for the namespaces using :

kubectl api-resources
(or)
kubectl api-resources |grep -i configmap

How do you check is there any ConfigMap exist in your  Kubernetes cluster?

We can check the configmaps exists on our Kubernetes system

kubectl get cm
  or 
kubectl get configmaps

Creating ConfigMaps using Literals

We can use key-value pairs in the command line and Create a simple configmap object using the literal by using --from-literal option:

  
kubectl create configmap devdb-cm --from-literal=dev.database_ip="192.168.30.2"

To check details about the newly created cm devdb-cm

  
kubectl describe cm devdb-cm

Note that  you can add multiple key-value pairs as data.

kubectl create configmap devapp-cm \
 --from-literal=app.envname="dev" \
 --from-literal=app.url="vtdev.com" \
 --from-literal=app.mem="1024m"
 
 kubectl create configmap prod-cm \
 --from-literal=app.envname="prod" \
 --from-literal=app.url="vtprod.com" \
 --from-literal=app.mem="8096m"
 
kubectl get cm 
kubectl describe cm devapp-cm
kubectl describe cm  prod-cm

2: Creating Using Directory

create a directory and have configMap files in it.

mkdir  cm-dir/; cd cm-dir
     echo "admin"> a.properties
	 echo "dev-env">b.properties
  
  
  kubectl create configmap app-config --from-file=cm-dir 
  
  kubectl get configmaps -o wide   
  

Describe the cm

  kubectl describe cm app-config 
  kubectl get configmaps app-config -o yaml 

observe the difference using -o yaml and the describe there you can find DATA section.

 

3: Creating Using a File

Create another ConfigMap using Files redis-file

  
  echo "app.name=web-state
  app.env=dev" > redis-file

creating...

    kubectl create configmap redis-config --from-file=redis-config

4: Declarative way using YAML

In the declarative approch, your YML you must create ConfigMap before refering it in Pod spec or template spec.

  
apiVersion: v1
kind: Pod 
metadata: 
  name: redis 
 spec: 
   containers: 
   - name: redis
     image: kubernetes/redis:v1
     volumeMounts:
	 - mountPath: /redis-master
	   name: config
volumes: 
  - name: config
    configMap: 
	  name: myredis-config
	  items:
	  - key: redis-config
	    path: redis.conf 

You can inject the configMap to a pod, here redis pod included as 'VolumeMounts', volume connects

  
kubectl apply -f redis-config.yml
kubectl exec redis cat /redis-master/redis.conf 		
kubectl exec -it redis redis-cli 

Pod associating with ConfigMaps

Using VolumeMounts pointing to Volume

Using env variable map to ConfigMap key reference