Kubernetes Secrets

Hello DevOps | DevSecOps teams, we are running into the new generation of microservices inside Pods where we need to focus on how we can protect them. And here this post is going with the Security rules imposing on the Kubernetes Cluster with Secret Objects which are specially designed to store the sensitive data in them to refer inside the Pod Containers. But they have limitation that they can hold up to 1MB size of data only.  

Why Secret objects?

• We can store Password, keys, tokens, certificates etc
• Secrets will reduce the risk of exposing sensitive data
• Access secrets using volumes and environment variables
• Secrets object will be created outside pod/containers 
• When it is created there is NO clues where it will be injected
• All secrets resides in ETCD database on the K8s master

This Kubernetes Secret Objects are similar to ConfigMaps Objects 

Kubernetes Secret objects Using Volume, ENVIRONMENT variables

Pre-check first we will check the Kubernetes Cluster is up and running.

kubectl get nodes

All the Kubernetes master and slave nodes are in Ready status.

There are two ways to access these Secrets inside the Pod.

1. Using Environment variables 
2. Assign to Path inside Pod 

Creating Kubernetes Generic Secrets

We are talking about Secrets, Let's create a text that can be converted to encrypted format, if we want we can decode that into plain text. In Linux CLI we have base64 command that will be used to do this encode or decode text from file or from standard input to display on standard output that is on our terminal. 

Secrets can be created in Kubernetes following ways: 

1. from file or directory
2. from literals
3. Using YAML Declarative approace

echo "admin" | base64 > username.txt 
cat username.txt 
cat username.txt | base64 -d

# Now password
echo "abhiteja" |base64 > password.txt
cat password.txt | base64 -d

# Create secret from file
kubectl create secret generic db-user-pass --from-file=./username.txt --from-file=./password.txt
kubectl get secrets
kubectl describe secrets db-user-pass

Validate

Kubernetes secret creation with username, password

The Pod  with Redis  image will be using the secrets as environment variables

FileName: secretenv-pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: secret-env-pod
spec:
  containers:
  - name: mycontainer
    image: redis
    env:
      - name: SECRET_USERNAME
        valueFrom:
          secretKeyRef:
            name: db-user-pass
            key: username.txt
      - name: SECRET_PASSWORD
        valueFrom:
          secretKeyRef:
            name: db-user-pass
            key: password.txt
  restartPolicy: Never  

You can create the Pod using the following command:

kubectl create -f secretenv-pod.yml
kubectl get pods

Secret environment variables in Redis Pod

Get inside the Redis Pod into the container and check with the 'env' command which will show the SECRET_USERNAME and SECRET_PASSWORD 

kubectl exec -it secret-env-pod -- bash 
env | grep SECRET

Secret as environment variables inside Pod Container

From the literal

Now let's see the simple option that is using --from-literal we can have as many as you wish to store as secret here I'm using three variables stored int the 'mysqldb-secret' object.

  
k create secret generic mysqldb-secret \
 --from-literal=DB_Host=mysql01.vybhava.com \
 --from-literal=DB_User=root \
 --from-literal=DB_Password=Welcome123
 
  k describe secret mysqldb-secret
  k get secret -o yaml
  

Execution output as follows:

Kubernetes Secret creation from literal example

 
Pod implementation the secret object you can see in the first option we have already discussed.

Creating the Secret declarative way

We can create a secret using the YAML where we can set the data fields as encrypted values

Filename : mysecret.yaml

apiVersion: v1
data:
  username: a3ViZWFkbQo=
  password: a3ViZXBhc3MK
kind: Secret
metadata:
  name: mysecret

Let's create the secret object with kubectl command

kubectl create -f mysecret.yaml
kubectl get secrets
kubectl describe secrets mysecret

Creating secret object in Kubernetes using kubectl

Note that secret never shows the data present in the secret section, instead it will be showing the size of the data. It is like masking the sensitive data. And this secrete can be used inside the Pod defination as follows: Filename: mypod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: mypod
    image: redis
    volumeMounts:
    - name: redis-db
      mountPath: "/etc/redis-db"
      readOnly: true
  volumes:
  - name: redis-db
    secret:
      secretName: mysecret

Creating the mypod defined as Pod and that using the volume defined with the secret section with mysecret which will referred to the above secret that we have created earlier.

Create Pod that uses secrets as volume

Encryption at rest configuration

The secrets which are created in the Kubernetes are not really not secrets! So we don't share these declarative YAML files in source code repositories. 

How this secret is stored in the ETCD DB 

To know this we must have etcd-client installed on your machine, on my system it is Ubuntu so let me install it.

apt-get install etcd-client
# Validate installation 
etcdctl 
  

Kubernetes ETCD DB Client installation on Ubuntu

Hope you enjoyed this post!!

Please share with your friends and comment if you find any issues here.

External References:

1. Official Documentation
2. Storage Volumes dynamic provisioning
3. PV - HostPath