As all know when is need to print the kafka broker id’s we can use the following cli this cli print the following Its means that we have kafka with id’s But our kafka names are So how to know which kafka broker id ( 1018 , 1017 , 1016 ) is belong to the real host ( Kafka_confluent01 /
Tag: apache-kafka
Kafka – Broker: Group coordinator not available
I have the following structure: Created topic with replication factor 3 and partitions 3 by kafka-topics shell script. And use group localConsumers. it works fine when leader is ok. Consumers’ log But if leader is down – I get the error in consumer (systemctl stop kafka): Node 3 is unavailable. ok Consumers’ log Consumer unable to connect until leader is
Changing Kafka Host name entry in zookeeper and persisting it across storm topology restart
Background 6 node Kafka Cluster 3 node Zookeeper Cluster 3 node Nimbus Cluster Apache Storm Worker hosts dynamically adjusted using amazon spot fleet Scenario For a particular topology for a given partition it subscribes to, the Zookeeper entry looks as follows {“topology”:{“id”:”Topology_Name-25-1520374231″,”name”:”Topology_Name”},”offset”:217233,”partition”:0,”broker”:{“host”:”Zk_host_name”,”port”:9092},”topic”:”topic1″} Now for worker hosts to access Zk_host_name, a mapping is added on each worker host in /etc/hosts file
Kafka broker crash every day – OOM killer
I have a cluster of 3 kafka brokers Version 0.10.2.1. Each broker has it’s own host 2 cpu / 16G RAM, In addition we are using docker to wrap the broker process. The problems is as follows: Almost every day at the same time we see all of our kafka clients failed for 10 minutes. At the beginning I thought