Apache Kafka has become the backbone of modern event-driven architectures, powering real-time data pipelines, microservices communication, log aggregation, financial transactions, and IoT workloads.

As Kafka clusters grow, monitoring becomes critical to ensure:

Topics are healthy
Consumer groups are processing messages
Lag is under control
Partitions are balanced
Brokers are available
Messages are flowing as expected

There are several Kafka monitoring solutions available in the market:

There are many several Kafka monitoring tools available, and here I've collected interesting facts about those monitoring tools:

Tool	Primary Focus
Factor House Kpow	Enterprise Kafka Operations
Datadog	Full-stack Observability
Logit.io	Managed Logging & Monitoring
Kafka Lag Exporter	Consumer Lag Metrics
Confluent Control Center	Confluent Platform Monitoring
CMAK (Cluster Manager for Apache Kafka)	Kafka Administration
Kafdrop	Lightweight Kafka Web UI
Offset Explorer	Desktop Kafka Client

I this article, we will focus on Kafdrop monitoring tool, which is one of the most popular open-source tool for Kafka monitoring and exploration tool

What is Kafdrop?

Kafdrop is a lightweight web application that provides a graphical interface for Kafka clusters.

Instead of using Kafka command-line utilities, Kafdrop allows engineers to inspect topics, partitions, consumer groups, offsets, and messages directly from a browser.

Kafdrop monitoring Kafka broker

Kafdrop does not store any data itself, it connects directly to the Kafka brokers and display the meta-data in user friendly format.

Key Benefits of Kafdrop

Easy Deployment: Single kafdrop JAR file deployment.
Lightweight: Minimal resource consumption compared to enterprise monitoring platforms.
Browser-Based UI: No need to install any extra desktop clients, we can access on web browser.
Message Inspection: View messages directly from Kafka topics.
Consumer Group Visibility: Monitor consumer lag and offsets.
Multi-Broker Support: Works with standalone brokers and clustered deployments.
Production Troubleshooting: Quickly identify consumer bottlenecks and topic issues.

Who can use this Kafdrop?

DevOps Teams

Validate Kafka topics after deployments.

Platform Engineers

Monitor consumer group health.

Developers

Inspect messages during troubleshooting.

SRE Teams

Investigate lag and partition issues.

Support Teams

Verify message delivery without command-line access.

Prerequisites:

Before installing Kafdrop make sure that we must have:

Component	Version
Java	17+ (21 recommended)
Kafka	2.x or higher
RAM	1 GB minimum
Browser	Chrome / Firefox
Network Access	Kafka Brokers Reachable

Step 1: Verify Java Installation

To run the Kafdrop monitoring tool Ensure Java installed by checking

java -version

If you don't find java on your Ubuntu/Debian OS run the following :

sudo apt update
sudo apt install openjdk-21-jdk -y

for Rocky/ RHELLinux

sudo dnf install java-21-openjdk -y

other Linux distributions you need to use right package manger to install JDK. After installation check again

java -version

Step 2: Create a dedicated directory for store Kafdrop jar

Let's create a dedicated directory, where we can download Kafdrop jar file from github and store Kafdrop jar file:

sudo mkdir /opt/kafdrop
cd  /opt/kafdrop
curl -L -o kafdrop.jar \
https://github.com/obsidiandynamics/kafdrop/releases/download/4.1.0/kafdrop-4.1.0.jar

The curl command ensures that Kafdrop may have any release version but the jar file renamed as `kafdrop.jar`.

Step 3: Connect Kafka cluster

Now all set to go and run the Kafdrop jar file with the --kafka.brokerConnect option where you can give single Kafka broker details or Kafka cluster details as mentioned here:

java -jar kafdrop.jar \
--kafka.brokerConnect=kafka-broker1-host:port,kafka-broker2-host:port

If you don't specified anything, that means standard single broker usage the option kafka.brokerConnect default points to localhost:9092

Step 4: Accessing Kafdrop Web UI

As Kafdrop uses 9000 as default port, So we can open a fresh browser and access the Kafdrop with localhost:9000 or from online terminals there will be Ports opened by providing custom port as '9000'.

On the Kafdrop web UI we can see:

Cluster Summary

Topics

Consumer Groups

Brokers

Partitions

Production Configuration for Kafdrop

Instead of hardcoding parameters, create a dedicated configuration file as 'application.properties'. Use your favorite text editor to create 'application.properties' file and the content should be like this:

  server.port=9090
  kafka.brokerConnect=kafka-broker1-host:port,kafka-broker2-host:port

when you have application.properties file in the same path where kafdrop.jar presents then while starting the kafdrop it will picked automatically.

You can have start-kafdrop.sh with the following content:

java -jar kafdrop.jar

Running Kafdrop as a Linux Service

Production deployments should run Kafdrop as a systemd service. Create a dedicated user as kafdrop:

# Adding user
sudo useradd kafdrop

# Add the ownership:
sudo chown -R kafdrop:kafdrop /opt/kafdrop

Create service file:

sudo vi /etc/systemd/system/kafdrop.service

Service file Content as:

[Unit]
Description=Kafdrop Kafka Monitoring
After=network.target

[Service]
User=kafdrop
WorkingDirectory=/opt/kafdrop

ExecStart=/usr/bin/java \
-jar /opt/kafdrop/kafdrop.jar \
--kafka.brokerConnect=broker1:9092,broker2:9092,broker3:9092

Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

To use service definitions to reflect Reload the daemon:

sudo systemctl daemon-reload

Enable reboot should not effect this service (auto start the service):

sudo systemctl enable kafdrop

You an Start kafdrop service:

sudo systemctl start kafdrop

Verify that kafdrop service status (hope it is running):

sudo systemctl status kafdrop

Setting Reverse Proxy with NGINX for Kafdrop

Production environments should avoid exposing port 9000 directly. Install NGINX

# For RHEL flavors
sudo dnf install nginx -y

# For Ubuntu/Debian
sudo apt install nginx -y

Configuration for avoiding the port exposing 9000:

server {

    listen 80;

    server_name kafdrop.company.com;

    location / {

        proxy_pass http://localhost:9000;

        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Benefits: * TLS support * Better Access control * Centralized routing

Security Best Practices

1. Restrict Access

Only Kafka administrators, DevOps Engineer and SRE support teams should access Kafdrop.

2. Enable HTTPS

Use any option to have HTTPS:

NGINX
Apache
Load Balancer

with TLS certificates.

3. Firewall Rules

Allow only Ports:

80
443

Internally. Avoid exposing the default port:

to the internet when you are running on Cloud platforms

4. Use Kafka Authentication

Production Kafka clusters often use secure connection with:

SASL
SCRAM
Kerberos
SSL/TLS

Configure Kafdrop accordingly.

Conclusion

Kafdrop is one of the easiest and most effective tools for monitoring Kafka clusters. It provides instant visibility into topics, partitions, brokers, consumer groups, offsets, and messages without requiring complex setup.

For production environments, deploy Kafdrop as a systemd service behind an NGINX reverse proxy with HTTPS enabled. Combined with Kafka metrics, Prometheus, Grafana, and centralized logging, Kafdrop becomes an excellent operational tool for DevOps, Platform Engineering, and Site Reliability teams.

Reader Challenge

Try extending your setup by:

Deploying Kafdrop using Docker
Integrating with Kafka SASL authentication
Running behind NGINX with SSL
Monitoring a multi-broker Kafka cluster
Comparing Kafdrop with Kpow and CMAK

How do you currently monitor Kafka in your organization —Kafdrop, Grafana, Datadog, Confluent Control Center, or another platform? Share your experience in the comments.

Search This Blog

DevOps Hunter Security Automation

Production-Ready Kafka Monitoring with Kafdrop – Complete Setup Guide