Apache Kafka Cheat Sheet

Here’s Apache Kafka cheat sheet:

Kafka Basics

Start Kafka Server:

bin/kafka-server-start.sh config/server.properties

Create a Topic:

bin/kafka-topics.sh --create --topic myTopic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

List Topics:

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Producer

Send Messages:

bin/kafka-console-producer.sh --topic myTopic --bootstrap-server localhost:9092

Consumer

Consume Messages:

bin/kafka-console-consumer.sh --topic myTopic --bootstrap-server localhost:9092 --from-beginning

Consumer in a Consumer Group:

bin/kafka-console-consumer.sh --topic myTopic --bootstrap-server localhost:9092 --group myGroup

Kafka Connect

Start Kafka Connect:

bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-source.properties

Create a Connector:

bin/kafka-topics.sh --create --topic connect-test --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

Kafka Streams

Start a Kafka Streams Application:

bin/kafka-run-class.sh org.apache.kafka.streams.examples.WordCountDemo

Kafka Administration

Check Consumer Lag:

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group myGroup

Alter Topic Configuration:

bin/kafka-configs.sh --zookeeper localhost:2181 --entity-type topics --entity-name myTopic --alter --add-config max.message.bytes=2097152

Kafka ACL (Access Control Lists)

Add ACL for a User:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:alice --operation Read --topic myTopic

List ACLs:

bin/kafka-acls.sh --list --authorizer-properties zookeeper.connect=localhost:2181

Kafka Security

Enable SSL:

bin/kafka-server-start.sh config/server-ssl.properties

Create Key and Truststores:

keytool -keystore server.keystore.jks -alias localhost -validity 365 -genkey

This Apache Kafka cheat sheet covers some basic commands and operations. For detailed documentation and more advanced configurations, refer to the Apache Kafka Documentation.