Home

Awesome

KAFKA CLUSTER KRAFT MODE - DOCKER COMPOSE

How to run

docker-compose up -d

Before connecting to cluster from outside docker ( ex from your docker host - your PC ), we also need to config host file:

Example for windows:

# C:\Windows\System32\drivers\etc\hosts

127.0.0.1 kafka01
127.0.0.1 kafka02
127.0.0.1 kafka03

Done !

Connection string (public)

kafka01:29192,kafka02:29292,kafka03:29392

.NET Demo (run outside docker)

Check here

Or run demo directly inside docker using bellow commands:

docker run -it --rm --network kafka-kraft-cluster-docker-compose_default confluentinc/cp-kafka /bin/kafka-console-producer --bootstrap-server kafka01:9092,kafka02:9092,kafka03:9092 --topic test_topic

then enter some text to produce message

Performance test

inside docker (container => container)

 docker run -it --rm --network kafka-kraft-cluster-docker-compose_default confluentinc/cp-kafka /bin/kafka-producer-perf-test --topic test_topic --num-records 1000000 --throughput -1 --producer-props bootstrap.servers=kafka01:9092,kafka02:9092,kafka03:9092 batch.size=16384 acks=1 linger.ms=50 --record-size 1000
<img src="assets/internal-docker-exec-demo.png" style="width: 100%;" />

outside docker (from host => container)

 $ .\kafka-producer-perf-test.bat --topic test_topic --num-records 1000000 --throughput -1 --producer-props bootstrap.servers=kafka01:29192,kafka02:29292,kafka03:29392 acks=1 linger.ms=50 --record-size 1000
<img src="assets/external-docker-exec-demo.png" style="width: 100%;" />
<img src="assets/dotnet-app.png" style="width: 100%;" /> 
$ .\kafka-consumer-groups.bat --bootstrap-server kafka01:29192,kafka02:29292,kafka03:29392 --group my-group1 --describe