DevOps Hunter Security Automation

 Hello DevSecOps experts, In this post we will discuss about how we can ensue container security scan.  Snyk  is one of the container scan tool option, we can sign up with GitHub or google account. Snyk will operates in three different ways: Snyk Web UI Snyk CLI Snyk Broker Heer we will experiment with Snyk CLI we can download from GitHub, as per your platform you can choose it. I'm going to work Preferrable option is to install the latest stable version.  sudo apt update sudo apt install curl -y curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash - sudo apt install -y nodejs npm install -g snyk When you run the above commands you will see the following output: snyk installation using npm package manager Ignore the npm warn lines which are expected as per the version incompatibilities. Prerequisites for snyk installation Docker engine you be installed already as per your operating system To have a docker image that can have vuln...

  Docker container command Tips & Tricks Here my idea is to use the Unix/Linux 'alias' command for most those common docker container,   network, volume sub- commands to form as shorten to give you more productivity while working on developing the docker images and playing around the newly constructing containers. This trick work on bash, zsh shells. Improve Productivity with smart work alias for Docker commands   First examine the docker container listing with the powerful option '--format' docker container ps -s \ --format "table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Size}}" docker ps command To get the logs of any applications that runs in containers we can use the following: alias dkrlogs='docker logs' alias dkrlogsf='docker logs -f ' docker logs with alias trick List of the images alias dkri='docker image ls' docker image list alias trick The container list alias dkrcs='docker container ls' d...

Hey DevOps, DevSecOps Engineers, SRE new bees here I am going to share the Learnings which I've executed every day found that this the best and new tip to improve the performance of ansible playbook executions sorted out and listed here. Planning and designing automation with Ansible Most common DevOps tool used for Planning and Designing is Confluence page Design document must contain a  clear " Objective " - where you will be describe why you wish to do automation on what area Tracking purpose always use a ticketing tool entry preferred tool Jira The design can be breakdown into two levels High level design where we will detail about what each task need to be covered Low level design where we discuss in-depth ideology on each task along with the possible constraints   Usage of global variables (AWX UI use extra vars, host_vars, group_vars etc) discuss their necessity AWX/Tower Job template construct possible options as input to handle overall objective, if...

What is GitGaurdian and ggsheild? The ggsheild is a security CLI tool developed by GitGuardian that helps developers and organizations prevent the exposure of sensitive information, such as API keys, credentials, and secrets, in their Git repositories. What are key features of ggsheild? Pre-Commit and Pre-Push Scanning: Scans code before it is committed or pushed to detect secrets. Prevents accidental leaks of sensitive data in version control. CI/CD Pipeline Integration: Works with GitHub Actions, GitLab CI/CD, Jenkins, and other CI tools. Ensures security checks are part of automated workflows. Real-Time Monitoring and Alerts: Detects exposed secrets in public or private repositories. Sends alerts and suggests remediation steps. Custom Rules & Policies: Allows defining custom regex patterns to detect organization-specific secrets. Supports allowlists to prevent false positives. How to install ggsheild on Ubuntu 24.04? apt update apt in...

Hey, hello, dear DevOps, DevSecOps, and SRE team heroes!! Here I came across a new challenge to solve the common problem on Git. You may be using GitHub or GitLab or even Bitbucket for source code management. Now, a few projects, websites, or mobile apps require storing images, audio files, or video files that are larger in size. During the transfer to the client systems, they are facing the following issues: Slowness in git clone and fetch operations: files taking too long to upload or download, leading to delays in deployment and user experience Sluggish commits and status checks: some clients are encountering errors related to file size limitations, causing frustration and hindering workflow efficiency Repository size bloat Complexity in managing multiple branches It's crucial for us to explore solutions that can streamline this process and ensure smooth handling of large sized media files. Git LFS installation on Ubuntu Using the package manager command we can ...

Git installation on Ubuntu is pretty simple. If you are looking for git instatlling on the RHEL or Rocky or Oracle Linux you can use this link . Now most of the software projects are using Git. So let's do installation on Ubuntu wit the following steps: Check for Git exists Install Git Confirm Git Installation Pre-requisites: Pick an instance on Cloud or online terminal of Ubuntu 20+ version to this experiment. Here I'm using the KillerCoda provided Ubuntu. Check for Git exists This is a common requirement when you join a new project and on the Linux machine you would like to know git installed or not. We have couple of options to check it. Let's do it here: dpkg -l git #or dpkg --list git In the output first 'ii' in the list means (if there are packages installed, you should see this mark) that the package is correctly installed and available. alternatively you can also try other option to check git installation on Ubuntu. apt list g...

Hey Guys!! I'm back with a new learning this week, I worked and experimented on Apache Cassandra Distributed database. It's special feature is it's quering capability with NoSQL - Not only SQL. Let's jump to our last set blog post where we have learnt about the Cassandra installation on a VM. Hope you are ready with Cassandra DB node. 1: Optimizing Data Modeling Objective: Understand partitioning and primary key design to optimize performance. Create an inefficient table within company_db keyspace: CREATE KEYSPACE company_db WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}; USE company_db; CREATE TABLE company_db.employees_bad ( id UUID PRIMARY KEY, name TEXT, department TEXT, age INT, city TEXT ); Now let's Insert some sample records into the table and try to query it. INSERT INTO employees_bad (id, name, department, age, city) VALUES (uuid(), 'Reemi', 'Engineering', 30, 'New York...

To monitor an Apache Cassandra cluster from the command line interface (CLI), you can use the nodetool utility, which is a powerful command-line tool specifically designed for managing and monitoring Cassandra clusters. Here are some key commands and their functionalities: Key nodetool Commands Check Cluster Status : nodetool status This command displays the status of all nodes in the cluster, including whether they are up or down, their load, and other important metrics. Column Family Statistics : nodetool cfstats [keyspace_name] . [table_name] This command provides detailed statistics for a specific table (column family), including read/write counts, disk space used, and more. Thread Pool Statistics : nodetool tpstats This command shows statistics about thread pools used for read, write, and mutation operations, helping to identify potential bottlenecks. Network Statistics : nodetool netstats This command displays information about netwo...

In this post we will have step-by-step process of installation of the Latestt version of Apache Cassandra 5.0.3 (as of Feb 2025 available as latest) on Ubuntu 20. What problem I'm solving with this? There is no direct documentation on the Cassandra to help on installation of latest version that is 5.0.3 on Ubuntu. So I've experimented it on the online Ubuntu terminal(kllercoda) and posting all the steps here. Pre-requisite to install Cassandra 1. Ubuntu Terminal either killercoda or codespace on github works good for this experiment. 2. Cassandra has specific compatibility requirements with different Java versions, which are crucial for ensuring optimal performance and stability. you must have super(root) user access to install cassandra. Step 1: Install Java/JRE Ensure Java installed by checking with command `java -version` and if not existing then install as following: #install jdk17-jre apt install openjdk-17-jre-headless java -version Step 2: Add Ca...

  Setting up the Kubernetes namespace for kafka apiVersion: v1 kind: Namespace metadata: name: "kafka" labels: name: "kafka" k apply -f kafka-ns.yml Now let's create the ZooKeeper container inside the kafka namespace apiVersion: v1 kind: Service metadata: labels: app: zookeeper-service name: zookeeper-service namespace: kafka spec: type: NodePort ports: - name: zookeeper-port port: 2181 nodePort: 30181 targetPort: 2181 selector: app: zookeeper --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: zookeeper name: zookeeper namespace: kafka spec: replicas: 1 selector: matchLabels: app: zookeeper template: metadata: labels: app: zookeeper spec: containers: - image: wurstmeister/zookeeper imagePullPolicy: IfNotPresent name: zookeeper ports: - containerPort: 2181 image1 - kube-kafka1 From th...

There are many monitoring trools available for Kafka brokers. Ive collectiect Various monitoring options: Factor House Kpow Datadog Logit.io Kafka Lag Exporter Confluent CMAK (Cluster Manager for Apache Kafka) Kafdrop Offset Explorer  Let's explore the kafdrop monitoring Prerequisites: To run the Kafdrop monitoring tool Ensure Java installed by checking java -version If you don't find java on your Ubuntu run the following : sudo apt update sudo apt install openjdk-21-jdk -y for other Linux distributions you need to use right package manger to install JDK. Download Kafdrop jar file from github : sudo mkdir kafka-monitor cd kafka-monitor curl -L -o kafdrop.jar https://github.com/obsidiandynamics/kafdrop/releases/download/4.1.0/kafdrop-4.1.0.jar The curl command ensures that Kafdrop may have any release version but the jar file renamed as `kafdrop.jar`. Now all set to go and run the Kafdrop jar file with the --kafka.brokerConnect option where you can give sing...

Kafka message broker is developed by Linkedin. To support their social media platform across the world. Message systems are defined as two Queues, Topics. If a messge send from a producer and received by single consumer then the communication will be point-to-point this is will be allowed in Queues. If Producer produce the message and there coudl be multiple consumers then it is One-to-many or pubf-sub of Publisher/Subscriber model it is implemented by Topic. Kafka supports both message models. Initial Setup for Kafka Kafka is build on scale and it runs on Java run time. So prerequisite to run Kafka we need Java as prerequisite. Optional other tools to help for troubleshoot and identify Kafka ports in use with 'netstat' command, topic content to view 'jq' and to have tree view 'tree' need to installed. apt update apt install -y net-tools jq tree On Ubuntu run the following commands to install compatible Java, here I'm using OpenJDK 21 version apt ...

What is Job object in Kubernetes? A Job object will be used to create one or more Pods and the Job ensures that a specified number of Pod instances will be created and terminates after completion of the Job. There could be finite jobs which will run within given certain timeout values. Job tracks for 'Successful' completion of the required task. Jobs can be run in two variants they can be parallel and also non-parallel. Kubernetes Job types There are 3 types of jobs non-parallel jobs [single pod jobs - unless it fails. creates replacement pod when pod goes down] parallel jobs with a fixed completion count parallel jobs with task queue  ##Example type 1: hundred-fibonaccis.yml --- apiVersion: batch/v1 kind: Job metadata: name: fibo-100 spec: template: spec: containers: - name: fib-container image: truek8s/hundred-fibonaccis:1.0 restartPolicy: OnFailure backoffLimit: 3 Create the Job: kubectl create -f hundred-fibonaccis.yml Now let's...