zookeeper和kafaka服务0.8版本以后默认是不需要配置的,但是本文遇到了一种要修改配置的情形。
终端A开启zookeeper和kafaka服务后
发送消息
bin/kafka-console-producer.sh --broker-list localhost:9092--topic tpc
终端B消费信息
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic tpc --from-beginning
但是终端A 输入一条信息 摁回车输入第二条信息的时候报错
[2016-04-29 11:30:08,296] WARN Property topic is not valid(kafka.utils.VerifiableProperties)
message
[2016-04-29 11:30:15,574] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,589] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,605] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,605] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,607] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau ltEventHandler)
[2016-04-29 11:30:15,721] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,722] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,730] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,732] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,734] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau ltEventHandler)
[2016-04-29 11:30:15,845] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,847] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,856] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,857] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,858] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau ltEventHandler)
[2016-04-29 11:30:15,969] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,971] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,981] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:15,985] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:15,985] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau ltEventHandler)
[2016-04-29 11:30:16,097] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)
[2016-04-29 11:30:16,099] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->
No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr oducer.BrokerPartitionInfo)
[2016-04-29 11:30:16,103] ERROR Failed to send requests fortopics tpc with correlation ids in [0,8](kafka.producer.async.DefaultEventHandler)
[2016-04-29 11:30:16,107] ERROR Error in handling batch of 1events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to sendmessages after 3 tries.
atkafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
atkafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
atkafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
atkafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
atscala.collection.immutable.Stream.foreach(Stream.scala:594)
atkafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
atkafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
报错很多,主要意思就是说不能建立socket ,因为要配置的这台服务器是有固定的ip地址的,所以推测host文件或者 kafaka的config文件没有配置。
解决方案
(1)修改host文件
vi /etc/hosts
xx.xxx.xxx.xxx localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhostlocalhost.localdomain localhost6 localhost6.localdomain6
xx.xx.xx.xx hub
xx.xx.xx.xx dev1
要修改的部分是:红色部分的xx.xx.xx.xx 这里要变成自己服务器的ip
(2)修改kafka服务器的配置文件
vi config/server.properties
SocketServer Settings 里面的host.name 写成自己的ip地址
Zookeeper里面 的zookeeper.connect=xx.xxx.xxx.xxx :2181