全网最详细的启动Kafka服务时出现kafka.common.InconsistentBrokerIdException: Configured brokerId 3 doesn't match st...

 

 

  不多说,直接上干货!

 

 

 

 问题详情

    执行bin/kafka-server-start.sh config/server.properties 时,

[kfk@bigdata-pro03 kafka_2.11-0.9.0.0]$ bin/kafka-server-start.sh config/server.properties 
[2018-06-17 16:05:38,983] INFO KafkaConfig values: 
    request.timeout.ms = 30000
    log.roll.hours = 168
    inter.broker.protocol.version = 0.9.0.X

 

 

 

 

 

[2018-06-17 16:05:39,269] INFO Socket connection established to bigdata-pro01.kfk.com/192.168.80.151:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:05:39,280] INFO Session establishment complete on server bigdata-pro01.kfk.com/192.168.80.151:2181, sessionid = 0x1640c718dfa0016, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:05:39,282] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2018-06-17 16:05:39,345] INFO Loading logs. (kafka.log.LogManager)
[2018-06-17 16:05:39,351] INFO Logs loading complete. (kafka.log.LogManager)
[2018-06-17 16:05:39,352] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2018-06-17 16:05:39,354] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2018-06-17 16:05:39,365] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentBrokerIdException: Configured brokerId 3 doesn't match stored brokerId 1 in meta.properties
    at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:630)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:175)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2018-06-17 16:05:39,370] INFO shutting down (kafka.server.KafkaServer)
[2018-06-17 16:05:39,381] INFO Shutting down. (kafka.log.LogManager)
[2018-06-17 16:05:39,391] INFO Shutdown complete. (kafka.log.LogManager)
[2018-06-17 16:05:39,392] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2018-06-17 16:05:39,398] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:05:39,398] INFO Session: 0x1640c718dfa0016 closed (org.apache.zookeeper.ZooKeeper)
[2018-06-17 16:05:39,401] INFO shut down completed (kafka.server.KafkaServer)
[2018-06-17 16:05:39,402] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
kafka.common.InconsistentBrokerIdException: Configured brokerId 3 doesn't match stored brokerId 1 in meta.properties
    at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:630)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:175)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2018-06-17 16:05:39,403] INFO shutting down (kafka.server.KafkaServer)

 

 

 

 

 

 

  问题分析

    一般都是,你自己修改了kafka版本之间切换造成的。你之前,是用kafka0.8.X系列,现在你升级使用0.9.X。没有完全更改过来。

 

 

 

 

 

    解决办法:

  找到配置文件

  cat server.properties

 

  查看log.dirs所在位置

  log.dirs=/opt/modules/kafka_2.11-0.9.0.0/kafka-logs

 

  按着这个路径找到

  meta.properties

  修改brokerid

[kfk@bigdata-pro03 kafka-logs]$ cat meta.properties 
#
#Sun Jun 17 15:59:31 CST 2018
version=0
broker.id=1
[kfk@bigdata-pro03 kafka-logs]$ pwd
/opt/modules/kafka_2.11-0.9.0.0/kafka-logs
[kfk@bigdata-pro03 kafka-logs]$ vim meta.properties 

 

 

 

 

 

 

 

 

 

[kfk@bigdata-pro03 kafka-logs]$ cat meta.properties 
#
#Sun Jun 17 15:59:31 CST 2018
version=0
broker.id=3
[kfk@bigdata-pro03 kafka-logs]$ pwd
/opt/modules/kafka_2.11-0.9.0.0/kafka-logs
[kfk@bigdata-pro03 kafka-logs]$ 

 

 

 

 

 

 

 

 

ent@592c17ce (org.apache.zookeeper.ZooKeeper)
[2018-06-17 16:12:47,785] INFO Opening socket connection to server bigdata-pro03.kfk.com/192.168.80.153:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:12:47,785] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
[2018-06-17 16:12:47,790] INFO Socket connection established to bigdata-pro03.kfk.com/192.168.80.153:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:12:47,798] INFO Session establishment complete on server bigdata-pro03.kfk.com/192.168.80.153:2181, sessionid = 0x3640c720971000f, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2018-06-17 16:12:47,801] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2018-06-17 16:12:47,851] INFO Loading logs. (kafka.log.LogManager)
[2018-06-17 16:12:47,858] INFO Logs loading complete. (kafka.log.LogManager)
[2018-06-17 16:12:47,858] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2018-06-17 16:12:47,860] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2018-06-17 16:12:47,917] INFO Awaiting socket connections on bigdata-pro03.kfk.com:9092. (kafka.network.Acceptor)
[2018-06-17 16:12:47,919] INFO [Socket Server on Broker 3], Started 1 acceptor threads (kafka.network.SocketServer)
[2018-06-17 16:12:47,940] INFO [ExpirationReaper-3], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-06-17 16:12:47,944] INFO [ExpirationReaper-3], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-06-17 16:12:48,010] INFO Creating /controller (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2018-06-17 16:12:48,020] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2018-06-17 16:12:48,020] INFO 3 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2018-06-17 16:12:48,104] INFO [GroupCoordinator 3]: Starting up. (kafka.coordinator.GroupCoordinator)
[2018-06-17 16:12:48,115] INFO [ExpirationReaper-3], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-06-17 16:12:48,116] INFO [GroupCoordinator 3]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2018-06-17 16:12:48,119] INFO [ExpirationReaper-3], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-06-17 16:12:48,141] INFO [Group Metadata Manager on Broker 3]: Removed 0 expired offsets in 35 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-06-17 16:12:48,194] INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-06-17 16:12:48,200] INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-06-17 16:12:48,249] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2018-06-17 16:12:48,267] INFO Creating /brokers/ids/3 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2018-06-17 16:12:48,275] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2018-06-17 16:12:48,276] INFO Registered broker 3 at path /brokers/ids/3 with addresses: PLAINTEXT -> EndPoint(bigdata-pro03.kfk.com,9092,PLAINTEXT) (kafka.utils.ZkUtils)
[2018-06-17 16:12:48,316] INFO Kafka version : 0.9.0.0 (org.apache.kafka.common.utils.AppInfoParser)
[2018-06-17 16:12:48,316] INFO Kafka commitId : fc7243c2af4b2b4a (org.apache.kafka.common.utils.AppInfoParser)
[2018-06-17 16:12:48,321] INFO [Kafka Server 3], started (kafka.server.KafkaServer)
[2018-06-17 16:12:48,335] INFO New leader is 3 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)

 

 

 

 

 

 

 

 

 

 

欢迎大家,加入我的微信公众号:大数据躺过的坑        人工智能躺过的坑
 
 
 

同时,大家可以关注我的个人博客

   http://www.cnblogs.com/zlslch/   和     http://www.cnblogs.com/lchzls/      http://www.cnblogs.com/sunnyDream/   

   详情请见:http://www.cnblogs.com/zlslch/p/7473861.html

 

  人生苦短,我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神,汇聚于互联网和个人学习工作的精华干货知识,一切来于互联网,反馈回互联网。
  目前研究领域:大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及:Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里,每天必须有收获

 

      对应本平台的讨论和答疑QQ群:大数据和人工智能躺过的坑(总群)(161156071) 

 

 

 

 

 

 

 

 

 

 

 

 

 

转载于:https://www.cnblogs.com/zlslch/p/9193101.html

<think>首先,用户的问题是关于KafkaServer启动发生的错误:java.lang.NumberFormatException:Forinputstring:"2181ÂÂÂÂÂÂ"。错误信息看起来端口号2181后面跟了一些非打印字符(如空格或其他不可见字符),导致解析失败。回顾引用内容:-引用[1]:关于Scala启动的NumberFormatException错误,输入字符串是"0x100",解决方法是环境变量配置问题。-引用[2]:Kafka启动错误,ClusterID不匹配,建议检查zookeeper.connect配置。-引用[3]:Kafka启动成功示例。-引用[4]:Scala报错参考链接,但未直接关联。用户具体错误是:`java.lang.NumberFormatException:Forinputstring:"2181ÂÂÂÂÂÂ"`。这表明在解析端口号,字符串包含额外字符,可能来自配置文件。常见原因:1.**配置文件问题**:Kafka的配置文件(如server.properties)中,端口号设置可能被污染,比如有额外的空格、制表符或不可见字符。2.**环境变量或脚本问题**:启动脚本可能错误地引入了这些字符。3.**编码问题**:文件保存使用了错误的编码,导致特殊字符。解决步骤:-检查Kafka配置文件(通常是server.properties),特别是与端口相关的设置,如`listeners`,`advertised.listeners`,或Zookeeper连接字符串(如`zookeeper.connect=localhost:2181`)。-确保配置值没有多余空格或特殊字符。使用文本编辑器(如vim,nano)查看并清理。-如果涉及Zookeeper,确认其端口设置正确。用户错误中端口是2181,这通常是Zookeeper的默认端口,所以可能是在Zookeeper连接字符串中出错。参考引用[2],提到ClusterID不匹配,但用户错误是NumberFormatException,所以可能不同问题。但配置错误是共同点。用户提到:"inputstring:"2181ÂÂÂÂÂÂ""–"Â"看起来像UTF-8编码问题,可能是在Windows或其他系统上编辑配置文件引入的。解决方案:1.**清理配置文件**:打开server.properties,找到包含端口2181的行(如zookeeper.connect),删除任何非数字字符。2.**使用正确编码**:确保文件以ASCII或UTF-8withoutBOM保存。3.**验证启动命令**:检查启动脚本是否有硬编码值添加了额外字符。在回答中,使用中文,结构清晰:-解释错误原因。-提供解决步骤。-生成相关问题。回答结构:1.**错误分析**:解释NumberFormatException原因。2.**解决步骤**:逐步指导如何修复。3.**预防措施**:如何避免类似问题。4.**相关问题**:生成几个相关问题。后,添加引用标识:[^1],[^2],等,但基于提供的引用,引用[2]相关,引用[3]是成功示例。用户错误是"2181ÂÂÂÂÂÂ",而引用[1]是"0x100",所以不同,但原理类似:字符串解析失败。</think>### KafkaServer启动java.lang.NumberFormatException错误的解决方案 您的KafkaServer启动失败,报错信息为`java.lang.NumberFormatException: For input string: "2181 Â Â Â Â Â Â"`。这表示Kafka在尝试解析端口号遇到了非法字符(如多余空格或不可见字符),导致数字格式转换失败。常见原因是配置文件(如`server.properties`)中存在污染数据或编码问题。以下是系统化的解决步骤,基于常见Kafka配置实践和类似错误案例[^2][^3]。 --- ### 错误原因分析 - **核心问题**:Kafka启动脚本在读取端口号(如`2181`),字符串包含额外字符(如`Â`或空格),这通常由以下原因引起: - 配置文件(如`server.properties`)中端口号后有多余空格、制表符或不可见字符。 - 文件编码问题(例如,配置文件以UTF-8 with BOM保存,但脚本预期ASCII)。 - Zookeeper连接字符串(如`zookeeper.connect`)配置错误[^2]。 - **影响**:Kafka无法正确解析端口号,导致启动失败。您的错误中`2181`后出现`Â`,这可能是复制粘贴或编辑器保存引入的乱码。 --- ### 解决步骤(逐步操作) 按照以下步骤检查和修复配置,确保操作在Kafka安装目录下进行(如`/usr/local/kafka_`)。 #### 步骤1: 检查并清理配置文件 1. **打开`server.properties`文件**: ```bash nano /usr/local/kafka/config/server.properties # 使用nano编辑器,或替换为vim ``` 2. **查找端口相关配置**: - 搜索`zookeeper.connect`行(Zookeeper连接设置)。 - 搜索`listeners`或`advertised.listeners`行(Kafka监听端口)。 - 示例正确格式应为: ``` zookeeper.connect=localhost:2181 # 确保2181后无空格或特殊字符 listeners=PLAINTEXT://:9092 # 端口号后直接换行 ``` 3. **删除多余字符**: - 手动删除端口号后的所有空格、制表符或乱码(如`Â`)。如果整行污染严重,可整行删除后重新输入。 - 保存文件(在nano中按`Ctrl+O`,然后`Ctrl+X`退出)。 #### 步骤2: 验证文件编码 - **检查编码**:运行`file -i config/server.properties`,确保输出为`text/plain; charset=us-ascii`或`charset=utf-8`。如果显示`charset=utf-16`或`charset=utf-8 with BOM`,需转换编码: ```bash iconv -f UTF-8 -t ASCII//TRANSLIT config/server.properties > temp.properties mv temp.properties config/server.properties ``` - **预防措施**:在编辑配置文件,使用纯文本编辑器(如vim、nano),避免使用Windows记事本(易引入BOM)。 #### 步骤3: 清理临文件和日志 - 删除Kafka数据(避免旧配置缓存): ```bash rm -rf /tmp/kafka-logs # 默认日志目录,根据配置调整 ``` - 检查日志确认错误源: ```bash tail -n 50 logs/server.log # 查看新日志,定位具体错误行 ``` #### 步骤4: 重启Kafka服务 - 启动Zookeeper(如果独立运行): ```bash bin/zookeeper-server-start.sh config/zookeeper.properties & ``` - 启动Kafka: ```bash bin/kafka-server-start.sh config/server.properties 1>/dev/null 2>&1 & ``` - 验证运行状态: ```bash jps # 应显示Kafka和QuorumPeerMain(Zookeeper)进程 ``` #### 预期结果 - 如果配置修复正确,Kafka将正常启动,不再报`NumberFormatException`[^3]。 - 如果仍失败,检查是否端口冲突(如2181或9092被占用):使用`netstat -tuln | grep 2181`。 --- ### 预防措施 - **配置规范**:编辑文件避免复制粘贴,手动输入关键值。 - **环境隔离**:在Linux环境下操作(Windows易引入编码问题),参考类似环境配置[^3]。 - **备份配置**:修改前备份文件:`cp server.properties server.properties.bak`。 此解决方案基于Kafka常见配置错误修复实践[^2][^3],如果问题持续,请提供完整日志以进一步分析。 --- ### 相关问题 1. **如何诊断Kafka启动的其他常见错误(如Cluster ID不匹配)?** (参考引用[2]中的Cluster ID问题) 2. **Kafka配置文件(server.properties)中关键参数如何优化以提高性能?** (涉及监听器、日志保留等设置) 3. **在Windows系统上运行Kafka,如何避免编码和路径问题?** (对比Linux环境差异,如文件路径和脚本兼容性) [^1]: Deepin Linux 安装启动scala报错 java.lang.NumberFormatException: For input string: "0x100" 解决。如图是我scala解压的位置,以及环境变量写法 scala就这样安装好了,试着命令行下输入scala写两行代码 启动scala报错 java.lang.NumberFormatException: For input string: “0x100” 解决 [^2]: kafka: ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)。window启动kafka报错: ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.InconsistentClusterIdException: The Cluster ID fESar1C0RH6tml3NB3N2Mw doesn&#39;t match stored clusterId Some(7DWuwcd0SgmM4PoafW2LSQ) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong. at kafka.server.KafkaServer.startup(KafkaServer.scala:218) at kafka.Kafka$.main(Kafka.scala:109) at kafka.Kafka.main(Kafka.scala) [^3]: Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)。再次启动kafka,nohup /usr/local/kafka_2.11-2.3.0/bin/kafka-server-start.sh /usr/local/kafka_2.11-2.3.0/config/server.properties 1>/dev/null 2>&1 & ,终端窗口输入命令jps查看进程,发现此kafka已经启起来了。 [^4]: Linux下Scala报错:java.lang.NumberFormatException: For input string: "0x100。参考:https://stackoverflow.com/questions/44317384/sbt-error-failed-to-construct-terminal-falling-back-to-unsupported
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值