Showing posts with label kubectl. Show all posts
Showing posts with label kubectl. Show all posts

Tuesday, March 1, 2022

Kubernetes installation on CentOS7 Vagrant boxes Manually

This page is updated in 2022 with new instructions which works for Rocky LInux as well!
As we have seen and had a setup that Docker EE installation on CentOS7. A year ago when I have worked on Kubernetes setup on the Ubuntu Linux that virtualization included all the steps involved in Docker installation to Kubernetes cluster configuration everything automated within Vagrantfile.


Kubernetes Cluster on your Desktop or Laptop or Mac book


In this post, I would like to share the manual steps that work to build a Kubernetes Cluster on CentOS7. We will be using the Docker EE installed nodes to install Kubernetes. So bringing up vagrant boxes the same thing that we had discussed earlier post proceed further.

Step 1: Check the System requirements

We have three nodes: master, node1, node2.

On ALL Nodes:
CPU Cores  2,
RAM size- 2GB Minimum 4GB good
Otherwise, Master node make it 3GB, Slave nodes with 1.5GB also a wise plan if you have limited resources.
Preparing the host mappings for master and worker nodes, Here I'm using sample names you can change as per your project needs.
   
hostnamectl set-hostname master-node
cat << EOF >> /etc/hosts
	10.128.0.27 master-node
	10.128.0.29 node-1 worker-node-1
	10.128.0.30 node-2 worker-node-2
EOF
Setup the firewall rules
 
  # master box run this
  sudo firewall-cmd --zone=public --permanent --add-port={6443,2379,2380,10250,10251,10252}/tcp
  
  # worker box firewall settings 
  sudo firewall-cmd --zone=public --permanent --add-port={10250,10251}/tcp
  
  #for both boxes
  firewall-cmd –reload
  modprobe br_netfilter
  echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
  

Step 2: Why do we need to do swap disable?

All Kubernetes masters and nodes are expected to have swap disabled. This is recommended by Kubernetes community for deployments. If swap is not disabled, kubelet service will not start on the masters and nodes,

 
# check swap available 
free -m
# if exists then run the following commands
swapoff -a # must for gcloud and aws instances
# permanent swap off from fstab
vi /etc/fstab --> comment swap entry
(OR) 
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

 
yum update -y
systemctl disable firewalld
systemctl stop firewalld
vi /etc/selinux/config ---> disabled

Restart all of the boxes
 
init 6

Now install Docker if you have not installed yet! this following will installs Docker-CE.
 
yum install docker -y
systemctl status docker #if it is inactive do the following
systemctl enable docker
systemctl start docker
systemctl start docker # make sure it is active state

Step 4: Add Kubernetes Repo 

This repo setting for CentOS boxes on ANY cloud env will works and same will work on vagrant box as well.
 
vi /etc/yum.repos.d/kubernetes.repo

Enter the following content into the file
 
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

Step 5: Install kubeadm, kubelet, kubectl and start

Now run the following yum installation commands on every node.
 
yum install kubeadm -y #This will includes kubectl, kubelet part of kubeadmin installation

systemctl enable kubelet
systemctl start kubelet
systemctl status kubelet # ensure kubelet is in active state

After starting kubeadm you will get the following:
Kubernetes Installation


output

Let's configure bridge network for Kubernetes
 
vi /etc/sysctl.d/k8s.conf
Enter the following lines
 
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

After file saving run the following command in the command shell.
 
sysctl --system

In the Master node execute the following command for Kubernetes Cluster initialization:

 
# NOTE: Please use your host IP address here
# This will do
kubeadm init

#alternatively try
kubeadm init --pod-network-cidr=192.148.0.0/16 --apiserver-advertise-address=192.168.33.100

(OR)

#To ignore preflight checks
kubeadm init --pod-network-cidr=192.148.0.0/16 --apiserver-advertise-address=192.168.33.100 --ignore-preflight-errors=Hostname,SystemVerification,NumCPU


On the Worker / Slave nodes:
 
kubeadm join 192.168.33.100:6443 --token h1ufen.hvs0nr49ua0my7u8 \
    --discovery-token-ca-cert-hash sha256:0bc179854b5c759333360737ff53ca2c4246b61823b033ecbac50593a9c334f6

Kubernetes Worker joining

On the master node do the following:
 
vi /etc/profile
export KUBECONFIG=/etc/kubernetes/admin.conf
Run the following: source /etc/profile
(OR)
 
 mkdir -p $HOME/.kube
 cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 chown $(id -u):$(id -g) $HOME/.kube/config

Now
flannel network
 
kubectl get nodes # all nodes NotReady state
kubectl get pods --all-namespaces
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl get pods --all-namespaces
kubectl get nodes

Get the status of the node in the Kubernetes cluster, after all pods Running

Validate with Deployment 

Let us validate the Kubernetes Cluster Ready for deploy web application

Step 1 Let's take nginx image for deployment creation on the Kubernetes cluster
 
 kubectl create deployment mynginx --image=nginx

First Kubernetes deployment : create deployment

Now let's see the description of the above 'mynginx' deployment.

Describe Kubernetes deployment

Scale the 'mynginx' application deployment upto 3
 
 kubectl scale --replicas=3 deployment/mynginx

Scale deployment on Kubernetes Cluster

List of all pods in the Kubernetes cluster
 
 kubectl get po
 kubectl get po -o wide

get the list of pods in Kubernetes

Next step is Create service using 'mynginx' deployment.
 
kubectl create service nodeport mynginx --tcp=8080:80
kubectl get services

Service creation in Kubernetes Cluster
all set to go for checking in the browser
http://192.168.33.110:32286/


As our slave node running on 192.168.33.110 and the node port exposed as 32286.

Here I conclude our Kubernetes cluster working as expected! Please post your comments or suggestions to improve our learnings more useful to many other starters.

Monday, October 22, 2018

Kubernetes cluster configuration in a Virtualbox with vagrant

Thanks to Rajkumar who had developed the Vagrantfile and published in the github on Kubernetes cluster configuration in a Virtualbox with vagrant. For those who don't know about Vagrant it is a tool that will be used for virtualization into a different level and more powerful way of using your system resources to run multiple operating virtual boxes in your Laptop/Desktop systems.

You just need to follow the simple steps which I had done in my experiment:

Prerequisites for Kubernetes Cluster Creation

  1. Download latest Vagrant
  2. Download latest version of Oracle VirtualBox
System resources requirements on VirtualBox

  • 2 GB for each node
  • 2 cores CPUs for each node
Here I have don this expeiment on my Windows 7 laptop. You could do same on any Windows higher version as well. Total 3 VMs will be created under a group named as - "Kubernetes Cluster" as defined in Vagrantfile.



Infrastructure as a Code: Vagrantfile 
# -*- mode: ruby -*-
# vi: set ft=ruby :
#Vagrant::DEFAULT_SERVER_URL.replace('https://vagrantcloud.com')
servers = [
{
:name => "k8s-master",
:type => "master",
:box => "ubuntu/xenial64",
:box_version => "20180831.0.0",
:enp0s8 => "192.168.33.10",
:mem => "2048",
:cpu => "2"
},
{
:name => "k8s-slave-1",
:type => "node",
:box => "ubuntu/xenial64",
:box_version => "20180831.0.0",
:enp0s8 => "192.168.33.11",
:mem => "2048",
:cpu => "2"
},
{
:name => "k8s-slave-2",
:type => "node",
:box => "ubuntu/xenial64",
:box_version => "20180831.0.0",
:enp0s8 => "192.168.33.12",
:mem => "2048",
:cpu => "2"
}
]
# This script to install k8s using kubeadm will get executed after a box is provisioned
$configureBox = <<-SCRIPT
# install docker v17.03
# reason for not using docker provision is that it always installs latest version of the docker, but kubeadm requires 17.03 or older
apt-get update
apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable"
apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print $3}')
# run docker commands as vagrant user (sudo not required)
usermod -aG docker vagrant
# install kubeadm
apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
# kubelet requires swap off
swapoff -a
# keep swap off after reboot
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# ip of this box
IP_ADDR=`ifconfig enp0s8 | grep Mask | awk '{print $2}'| cut -f2 -d:`
# set node-ip
sudo sed -i "/^[^#]*KUBELET_EXTRA_ARGS=/c\KUBELET_EXTRA_ARGS=--node-ip=$IP_ADDR" /etc/default/kubelet
sudo systemctl restart kubelet
SCRIPT
$configureMaster = <<-SCRIPT
echo "This is master"
# ip of this box
IP_ADDR=`ifconfig enp0s8 | grep Mask | awk '{print $2}'| cut -f2 -d:`
# install k8s master
HOST_NAME=$(hostname -s)
kubeadm init --apiserver-advertise-address=$IP_ADDR --apiserver-cert-extra-sans=$IP_ADDR --node-name $HOST_NAME --pod-network-cidr=172.16.0.0/16
#copying credentials to regular user - vagrant
sudo --user=vagrant mkdir -p /home/vagrant/.kube
cp -i /etc/kubernetes/admin.conf /home/vagrant/.kube/config
chown $(id -u vagrant):$(id -g vagrant) /home/vagrant/.kube/config
# install Calico pod network addon
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://raw.githubusercontent.com/ecomm-integration-ballerina/kubernetes-cluster/master/calico/rbac-kdd.yaml
kubectl apply -f https://raw.githubusercontent.com/ecomm-integration-ballerina/kubernetes-cluster/master/calico/calico.yaml
kubeadm token create --print-join-command >> /etc/kubeadm_join_cmd.sh
chmod +x /etc/kubeadm_join_cmd.sh
# required for setting up password less ssh between guest VMs
sudo sed -i "/^[^#]*PasswordAuthentication[[:space:]]no/c\PasswordAuthentication yes" /etc/ssh/sshd_config
sudo service sshd restart
SCRIPT
$configureNode = <<-SCRIPT
echo "This is worker"
apt-get install -y sshpass
sshpass -p "vagrant" scp -o StrictHostKeyChecking=no vagrant@192.168.33.10:/etc/kubeadm_join_cmd.sh .
sh ./kubeadm_join_cmd.sh
SCRIPT
Vagrant.configure("2") do |config|
servers.each do |opts|
config.vm.define opts[:name] do |config|
config.vm.box = opts[:box]
config.vm.box_version = opts[:box_version]
config.vm.hostname = opts[:name]
config.vm.network :private_network, ip: opts[:enp0s8]
config.vm.provider "virtualbox" do |v|
v.name = opts[:name]
v.customize ["modifyvm", :id, "--groups", "/Kubernetes Cluster"]
v.customize ["modifyvm", :id, "--memory", opts[:mem]]
v.customize ["modifyvm", :id, "--cpus", opts[:cpu]]
end
# we cannot use this because we can't install the docker version we want - https://github.com/hashicorp/vagrant/issues/4871
#config.vm.provision "docker"
config.vm.provision "shell", inline: $configureBox
if opts[:type] == "master"
config.vm.provision "shell", inline: $configureMaster
else
config.vm.provision "shell", inline: $configureNode
end
end
end
end

The Vagrantfile will be composed with the Ruby array that creates k8s-head and k8s-node1, k8s-node2 definitions. Once the Ubuntu Xenial boxes provisioned custom shell scripts are used for boot time execution.


  • Both Master, Slave nodes common tasks are executed with the Shell provisioning inline options.
  • Install Docker CE 17.03
  • Added vagrant user to docker group to run docker commands as vagrant user (without using sudo for each not required)
  • Install the kubelet kubeadm kubectlk
  • kubelet requires swap off


You can do all the setups required to run the following in the sequence : 
  • k8s-master node runs on 192.168.33.10
  • k8s-slave1 node runs on 192.168.33.11
  • k8s-slave2 node runs on 192.168.33.12
Bootstrap Setup



Master node will be required the following steps
Slave node will be running and after bootup only runs inline joining the kubernetes cluster with a script generated in the master. node.

Executing the setup
vagrant up

check the VM are created as expected
vagrant status

Vagrant status of kuberenetes cluster
Check that all are in running state, if not you need to check the log file that is generated in the same path where Vagrantfile exists.

Connect with your PuTTY to k8s-master that is running on 192.168.33.10 IP address.

Check the versions of kubeadm, kubectl, and kubelet
  kubectl version
  kubeadm version
  # Better format output
  kubectl version -o yaml
  kubeadm version -o yaml
  

Kubeadm, kubectl, kubelet versions
Check the nodes list

kubectl get nodes

kubectl get nodes output

Note: Make sure that your Windows firewall disabled to run the Vagrant on your Windows laptop.

You might be more interested to explore and know about the latest Docker 19 Community Edition learning experiments on Ubuntu 19.04

References:

Categories

Kubernetes (25) Docker (20) git (15) Jenkins (12) AWS (7) Jenkins CI (5) Vagrant (5) K8s (4) VirtualBox (4) CentOS7 (3) docker registry (3) docker-ee (3) ucp (3) Jenkins Automation (2) Jenkins Master Slave (2) Jenkins Project (2) containers (2) create deployment (2) docker EE (2) docker private registry (2) dockers (2) dtr (2) kubeadm (2) kubectl (2) kubelet (2) openssl (2) Alert Manager CLI (1) AlertManager (1) Apache Maven (1) Best DevOps interview questions (1) CentOS (1) Container as a Service (1) DevOps Interview Questions (1) Docker 19 CE on Ubuntu 19.04 (1) Docker Tutorial (1) Docker UCP (1) Docker installation on Ubunutu (1) Docker interview questions (1) Docker on PowerShell (1) Docker on Windows (1) Docker version (1) Docker-ee installation on CentOS (1) DockerHub (1) Features of DTR (1) Fedora (1) Freestyle Project (1) Git Install on CentOS (1) Git Install on Oracle Linux (1) Git Install on RHEL (1) Git Source based installation (1) Git line ending setup (1) Git migration (1) Grafana on Windows (1) Install DTR (1) Install Docker on Windows Server (1) Install Maven on CentOS (1) Issues (1) Jenkins CI server on AWS instance (1) Jenkins First Job (1) Jenkins Installation on CentOS7 (1) Jenkins Master (1) Jenkins automatic build (1) Jenkins installation on Ubuntu 18.04 (1) Jenkins integration with GitHub server (1) Jenkins on AWS Ubuntu (1) Kubernetes Cluster provisioning (1) Kubernetes interview questions (1) Kuberntes Installation (1) Maven (1) Maven installation on Unix (1) Operations interview Questions (1) Oracle Linux (1) Personal access tokens on GitHub (1) Problem in Docker (1) Prometheus (1) Prometheus CLI (1) RHEL (1) SCM (1) SCM Poll (1) SRE interview questions (1) Troubleshooting (1) Uninstall Git (1) Uninstall Git on CentOS7 (1) Universal Control Plane (1) Vagrantfile (1) amtool (1) aws IAM Role (1) aws policy (1) caas (1) chef installation (1) create organization on UCP (1) create team on UCP (1) docker CE (1) docker UCP console (1) docker command line (1) docker commands (1) docker community edition (1) docker container (1) docker editions (1) docker enterprise edition (1) docker enterprise edition deep dive (1) docker for windows (1) docker hub (1) docker installation (1) docker node (1) docker releases (1) docker secure registry (1) docker service (1) docker swarm init (1) docker swarm join (1) docker trusted registry (1) elasticBeanStalk (1) global configurations (1) helm installation issue (1) mvn (1) namespaces (1) promtool (1) service creation (1) slack (1)