kafka集群安装_inconsistentbrokeridexception: configured broker.i-优快云博客

本文链接：https://blog.youkuaiyun.com/shandadadada/article/details/50615866

本文详述了如何在两台服务器上安装Kafka集群，包括Zookeeper的下载、Kafka配置（如broker.id和host.name）、启动步骤以及解决启动时遇到的内存不足和broker.id不匹配等问题。此外，还提及了客户端连接Kafka需要配置advertised.host.name的重要性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1.zookeeper下载

 wget -c http://apache.fayea.com/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgz

2.分别在两台台服务器上安装kafka：

tar zxvf kafka_2.11-0.9.0.0.tgz

修改每台服务器的config/server.properties

broker.id：唯一，填数字，本文中分别为0/1

host.name：唯一，填服务器IP

zookeeper.connect=192.168.40.134:2181,192.168.40.132:2181

先启动zookeeper服务:（zookeeper安装详情见http://blog.youkuaiyun.com/shandadadada/article/details/48303019）

再在每台机器上执行：

bin/kafka-server-start.sh config/server.properties  &

注释：& 是指退出命令行仍运行

启动完毕，可用jps查看进程

root@iZ94uoxhu56Z:/home/kafka/kafka_2.11-0.9.0.0# jps
24548 QuorumPeerMain
20532 Bootstrap
3878 Kafka
3955 Jps
27832 Bootstrap

3.创建topic

root@iZ94uoxhu56Z:/home/kafka/kafka_2.11-0.9.0.0# bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 1 --topic mykafka 
Created topic "mykafka".

查看topic

root@iZ94uoxhu56Z:/home/kafka/kafka_2.11-0.9.0.0# bin/kafka-topics.sh --list --zookeeper localhost:2181
mykafka
test-topic

查看详细信息：

root@iZ94uoxhu56Z:/home/kafka/kafka_2.11-0.9.0.0# bin/kafka-topics.sh --describe --zookeeper 192.168.40.134:2181
Topic:mykafka   PartitionCount:1        ReplicationFactor:2     Configs:
        Topic: mykafka  Partition: 0    Leader: 0       Replicas: 0,1   Isr: 0
Topic:test-topic        PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: test-topic       Partition: 0    Leader: 0       Replicas: 0     Isr: 0

发送消息：

bin/kafka-console-producer.sh --broker-list 192.168.40.134:9092 --topic mykafka

接受消息：

bin/kafka-console-consumer.sh --zookeeper 192.168.40.133:2181 --topic mykafka --from-beginning

sfsa

asdg

sfa

asfda

sadf

1

3

去zk上看kafka集群

[zk: localhost:2181(CONNECTED) 5] ls /  
[admin, zookeeper, consumers, config, controller, zk-fifo, storm, brokers, controller_epoch]  
[zk: localhost:2181(CONNECTED) 6] ls /brokers   ----> 查看注册在zk内的kafka  
[topics, ids]  
[zk: localhost:2181(CONNECTED) 7] ls /brokers/ids  
[112, 110, 111]  
[zk: localhost:2181(CONNECTED) 8] ls /brokers/ids/112  
[]  
[zk: localhost:2181(CONNECTED) 9] ls /brokers/topics   
[test]  
[zk: localhost:2181(CONNECTED) 10] ls /brokers/topics/test   
[partitions]  
[zk: localhost:2181(CONNECTED) 11] ls /brokers/topics/test/partitions  
[2, 1, 0]  
[zk: localhost:2181(CONNECTED) 12]

遇到的问题：
1.启动kafka的时候报内存不足错误

你可以通过编辑调整JVM堆大小kafka-server-start.sh，zookeeper-server-start.sh等：

export KAFKA_HEAP_OPTS=”-Xmx1G -Xms1G”

的-Xms参数指定的最小堆大小。让你的服务器至少开始，试着改变它使用较少的内存。因为你只有512M，你应该改变最大堆大小（-Xmx）太：

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

我不知道是什么的最小内存要求卡夫卡在默认的配置文件是你可能需要调整的消息的大小在卡夫卡得到它运行。

2.brokerid不匹配问题

kafka.common.InconsistentBrokerIdException: Configured brokerId 1 doesn't match stored brokerId 0 in meta.properties
        at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:630)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:175)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
        at kafka.Kafka$.main(Kafka.scala:67)
        at kafka.Kafka.main(Kafka.scala)
[2015-12-11 09:49:20,175] INFO shutting down (kafka.server.KafkaServer)

doesn’t match stored brokerId 0 in meta.properties

错误的原因是log.dirs目录下的meta.properties中配置的broker.id和配置目录下的server.properties中的broker.id不一致了，解决问题的方法是将两者修改一致后再重启。

3.客户端连接kafka报错：

[2015-06-02 16:23:04,375] WARN Fetching topic metadata with correlation id 0 for topics [Set(eventbustopic)] from broker [id:1,host:SOME_HOST,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException                                       
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)           
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)        
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:113)                
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)        
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)        
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)         
[2015-06-02 16:23:04,515] WARN Fetching topic metadata with correlation id 0 for topics [Set(eventbustopic)] from broker [id:0,host:SOME_HOST,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException                                       
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)           
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)        
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:113)                
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)        
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)        
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)

是由于Kafka server.properties中的advertised.host.name 属性没有配置。这个属性代表客户端访问Kafka的地址。如果没配置默认是主机名。

bin/kafka-console-producer.sh --broker-list 114.215.173.49:9092,112.74.88.53:9092 --topic mykafka

bin/kafka-console-consumer.sh --zookeeper 112.74.88.53:2181,115.29.175.58:2181 --topic mykafka --from-beginning