Skip to content
Advertisement

Tag: apache-kafka

Kafka – Broker: Group coordinator not available

I have the following structure: Created topic with replication factor 3 and partitions 3 by kafka-topics shell script. And use group localConsumers. it works fine when leader is ok. Consumers’ log But if leader is down – I get the error in consumer (systemctl stop kafka): Node 3 is unavailable. ok Consumers’ log Consumer unable to connect until leader is

Changing Kafka Host name entry in zookeeper and persisting it across storm topology restart

Background 6 node Kafka Cluster 3 node Zookeeper Cluster 3 node Nimbus Cluster Apache Storm Worker hosts dynamically adjusted using amazon spot fleet Scenario For a particular topology for a given partition it subscribes to, the Zookeeper entry looks as follows {“topology”:{“id”:”Topology_Name-25-1520374231″,”name”:”Topology_Name”},”offset”:217233,”partition”:0,”broker”:{“host”:”Zk_host_name”,”port”:9092},”topic”:”topic1″} Now for worker hosts to access Zk_host_name, a mapping is added on each worker host in /etc/hosts file

Kafka broker crash every day – OOM killer

I have a cluster of 3 kafka brokers Version 0.10.2.1. Each broker has it’s own host 2 cpu / 16G RAM, In addition we are using docker to wrap the broker process. The problems is as follows: Almost every day at the same time we see all of our kafka clients failed for 10 minutes. At the beginning I thought

Advertisement