Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)

优快云

📕我是廖志伟,一名Java开发工程师、Java领域优质创作者、优快云博客专家、51CTO专家博主、阿里云专家博主、清华大学出版社签约作者、产品软文创造者、技术文章评审老师、问卷调查设计师、个人社区创始人、开源项目贡献者。跑过十五公里、徒步爬过衡山、有过三个月减肥20斤的经历、是个喜欢躺平的狠人。
📘拥有多年一线研发和团队管理经验,研究过主流框架的底层源码(Spring、SpringBoot、Spring MVC、SpringCould、Mybatis、Dubbo、Zookeeper),消息中间件底层架构原理(RabbitMQ、RockerMQ、Kafka)、Redis缓存、MySQL关系型数据库、 ElasticSearch全文搜索、MongoDB非关系型数据库、Apache ShardingSphere分库分表读写分离、设计模式、领域驱动DDD、Kubernetes容器编排等。有从0到1的高并发项目经验,利用弹性伸缩、负载均衡、报警任务、自启动脚本,最高压测过200台机器,有着丰富的项目调优经验。
📙在优快云创作了上千篇文章后,和清华大学出版社签约四本书籍,于明年陆续出版⚡《Java项目实战—深入理解大型互联网企业通用技术》⚡基础篇、进阶篇、架构篇、📚《解密程序员的思维密码–沟通、演讲、思考的实践》📚,具体出版计划根据实际情况调整,希望各位读者大大到时多多支持。

💂博客主页: 我是廖志伟 | 👉开源项目:java_wxid | 🌥哔哩哔哩:我是廖志伟 | 🔖个人微信号: SeniorRD

🤟 希望各位读者大大多多支持用心写文章的博主,现在时代变了,信息爆炸,酒香也怕巷子深,博主真的需要大家的帮助才能在这片海洋中继续发光发热,所以,赶紧动动你的小手,点波关注❤️,点波赞👍,点波收藏⭐,甚至点波评论✍️,都是对博主最好的支持和鼓励!如需转载或搬运文章,请私信我哈。
🍋今天是2023年10月14日,愿你在新的一周里能够保持专业素养,成为技术领域的佼佼者!

优快云


本文的大概内容:

🍊 一、业务场景:报错kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING

我通过docker命令启动kafka,进入kafka容器之后,过一二秒没做操作,就出现退出容器的情况,通过zookeeper客户端连接kafka配置的zookeeper发现可以连接的上,但是查看kafka日志却报错kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING


🍊 二、错误日志:Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)


在这里插入图片描述

[2021-11-16 13:39:02,215] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-16 13:39:02,215] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-16 13:39:02,219] INFO Initiating client connection, connectString=106.14.132.94:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@4466af20 (org.apache.zookeeper.ZooKeeper)
[2021-11-16 13:39:02,241] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2021-11-16 13:39:02,248] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2021-11-16 13:39:02,256] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2021-11-16 13:39:12,261] INFO Opening socket connection to server 106.14.132.94/106.14.132.94:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2021-11-16 13:39:12,268] INFO Socket error occurred: 106.14.132.94/106.14.132.94:2181: Host is unreachable (org.apache.zookeeper.ClientCnxn)
[2021-11-16 13:39:20,258] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2021-11-16 13:39:23,377] INFO Opening socket connection to server 106.14.132.94/106.14.132.94:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2021-11-16 13:39:23,482] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2021-11-16 13:39:23,483] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2021-11-16 13:39:23,485] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2021-11-16 13:39:23,492] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
	at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:262)
	at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:119)
	at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1881)
	at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:441)
	at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:466)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:233)
	at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
	at kafka.Kafka$.main(Kafka.scala:82)
	at kafka.Kafka.main(Kafka.scala)
[2021-11-16 13:39:23,496] INFO shutting down (kafka.server.KafkaServer)
[2021-11-16 13:39:23,502] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2021-11-16 13:39:23,503] INFO shut down completed (kafka.server.KafkaServer)
[2021-11-16 13:39:23,503] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2021-11-16 13:39:23,506] INFO shutting down (kafka.server.KafkaServer)

由于kafka运行是通过docker镜像的方式运行的,而且zookeeper外部的客户端也可以连接上zookeeper就排除了,zookeeper的异常情况,那么剩下的只可能是kafka运行的命令出问题了,我之前的运行命令如下:

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name kafka \
-p 9092:9092 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=106.14.132.94:2181 \
-e KAFKA_ADVERTISED_HOST_NAME=106.14.132.94 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://106.14.132.94:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/kafka

把监听的0.0.0.0改成服务器ip地址

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name kafka \
-p 9092:9092 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=106.14.132.94:2181 \
-e KAFKA_ADVERTISED_HOST_NAME=106.14.132.94 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://106.14.132.94:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://106.14.132.94:9092 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/kafka

但是还是有出错的情况:
Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
Host is unreachable (org.apache.zookeeper.ClientCnxn)
在这里插入图片描述

[2021-11-16 15:20:11,821] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.version=5.10.60-9.al8.x86_64 (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.memory.free=973MB (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,821] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,838] INFO Initiating client connection, connectString=106.14.132.94:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@4466af20 (org.apache.zookeeper.ZooKeeper)
[2021-11-16 15:20:11,857] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2021-11-16 15:20:11,864] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2021-11-16 15:20:11,871] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2021-11-16 15:20:21,879] INFO Opening socket connection to server 106.14.132.94/106.14.132.94:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2021-11-16 15:20:21,885] INFO Socket error occurred: 106.14.132.94/106.14.132.94:2181: Host is unreachable (org.apache.zookeeper.ClientCnxn)

奇怪的是zookeeper的节点有kafka相关的数据了
在这里插入图片描述SASL全称Simple Authentication and Security Layer,是一种用来扩充C/S模式验证能力的机制。

🍊 三、解决方案

网上百度了一堆没用的,我一一列举出来
第一个说是通过官网发现,在zookeeper 3.4.5之前,sasl认证是没有办法规避的,在3.4.6版本后修复了这个bug,因为不停的去检测认证虽然对功能没有什么影响,但是比较耗费服务器性能,比较占用容器资源。zk升级到3.4.6+的版本,可我的版本是zookeeper-3.4.13,肯定是符合的,无效。
第二个说是让zookeeper绕过sasl安全机制,禁用sasl认证,直接向系统获取资源。在conf目录下的zoo.cfg文件添加配置,zookeeper.sasl.client=false,然后重启zookeeper,亲身试过,无效。
第三个说是端口开放的问题,防火墙啊,入口和出口规则限制啊,这些符合,因为我可以通过客户端工具连接zookeeper,所以不存在端口问题,无效。
第四个说是给zookeeper做安全配置,但是我的初衷是对Kafka客户端使用SASL身份验证,但不要求kafka对Zookeeper进行SASL身份验证,我不关心kafka < - > zookeeper之间的安全校验问题,我就想单纯的通过docker进行运行kafka,可以让我进到容器内部进行测试发送消息。所以我还在寻找绕开sasl认证的办法,通过在apache bug管理官网:https://issues.apache.org/jira/browse/ZOOKEEPER-1657,我大致看了看还是和上面类似的解决方案。
在这里插入图片描述大致的意思就跟第一个说的差不多,不停的去检测认证虽然对功能没有什么影响,但是比较耗费服务器性能,比较占用容器资源。
下面的解决方案也如出一辙。
在这里插入图片描述除此之外,还发现了一篇帖子的解决方案也大体相同
在这里插入图片描述只不过我都尝试过,发现都不起效果。
然后我去查看有关SASL配置项

X509AuthenticationProvider.superUser:Java系统属性:zookeeper.X509AuthenticationProvider.superUser)支持SSL的方法,使ZooKeeper集合管理员能够以“超级”用户身份访问znode层次结构。当此参数设置为X500主体名称时,只有具有该主体的经过身份验证的客户端才能绕过ACL检查并拥有所有znode的完全权限。
zookeeper.superUser:Java系统属性:zookeeper.superUser)与zookeeper.X509AuthenticationProvider.superUser类似,但对于基于SASL的登录是通用的。它存储可以作为“超级”用户访问znode层次结构的用户的名称。

看了这些配置,发现并没什么作用。到这里我基本放弃了绕开sasl认证的办法了,网上也找不到更多有用的消息,还不如老老实实的配置安全认证,这里我参考了二篇博文,第一篇是集群模式的:https://blog.youkuaiyun.com/sdksdk0/article/details/95336382,第二篇是https://www.orchome.com/553。又熬到快十二点半了,熬不动了,睡觉睡觉,希望有大神可以帮忙解决这个绕开sasl认证的问题,至于权限认证的问题,等也时间再慢慢实现。这里我提供kafka和zookeeper的启动命令给大神作参考。
具体实现可参考:https://liaozhiwei.blog.youkuaiyun.com/article/details/120716913

早上起床接着排查
第五个说的是zookeeper客户端 和服务器连接时版本问题,不过我并没有用程序连接,无效
在这里插入图片描述
第六个说的是zookeeper服务器主动断开与客户端的连接问题,修改配置,无效
在这里插入图片描述这个时候我去看了看zookeeper的日志信息,发现也报错了
在这里插入图片描述这个时候,我将之前改过的配置都还原,把原有的镜像都删除,重新启动docker运行镜像
kafka

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name kafka \
-p 9092:9092 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=106.14.132.94:2181 \
-e KAFKA_ADVERTISED_HOST_NAME=106.14.132.94 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://106.14.132.94:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://106.14.132.94:9092 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/kafka

zookeeper

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2  --name zookeeper \
-p 2181:2181 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/zookeeper

然后发现zookeeper的错误信息日志
在这里插入图片描述

2021-11-18 00:11:05,537 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57390 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:05,537 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57390 (no session established for client)
2021-11-18 00:11:07,162 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57843
2021-11-18 00:11:07,162 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57843 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:07,163 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57843 (no session established for client)
2021-11-18 00:11:09,310 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57534
2021-11-18 00:11:09,318 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57534 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:09,318 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57534 (no session established for client)
2021-11-18 00:11:10,942 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57699
2021-11-18 00:11:10,943 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57699 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:10,943 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57699 (no session established for client)
2021-11-18 00:11:12,684 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57568
2021-11-18 00:11:12,695 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57568 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:12,696 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57568 (no session established for client)
2021-11-18 00:11:14,703 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57601
2021-11-18 00:11:14,743 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@922] - Refusing session request for client /114.87.234.228:57601 as it has seen zxid 0x7 our last zxid is 0x0 client must try another server
2021-11-18 00:11:14,743 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57601 (no session established for client)
2021-11-18 00:11:15,412 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57800
2021-11-18 00:11:15,414 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@949] - Client attempting to establish new session at /114.87.234.228:57800
2021-11-18 00:11:15,416 [myid:] - INFO  [SyncThread:0:FileTxnLog@213] - Creating new log file: log.1
2021-11-18 00:11:15,427 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@694] - Established session 0x100137139d30000 with negotiated timeout 5000 for client /114.87.234.228:57800
2021-11-18 00:11:16,843 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@487] - Processed session termination for sessionid: 0x100137139d30000
2021-11-18 00:11:16,845 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57800 which had sessionid 0x100137139d30000
2021-11-18 00:11:17,354 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57488
2021-11-18 00:11:17,354 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@949] - Client attempting to establish new session at /114.87.234.228:57488
2021-11-18 00:11:17,356 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@694] - Established session 0x100137139d30001 with negotiated timeout 5000 for client /114.87.234.228:57488
2021-11-18 00:11:18,817 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@487] - Processed session termination for sessionid: 0x100137139d30001
2021-11-18 00:11:18,819 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57488 which had sessionid 0x100137139d30001
2021-11-18 00:11:19,344 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57417
2021-11-18 00:11:19,355 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@949] - Client attempting to establish new session at /114.87.234.228:57417
2021-11-18 00:11:19,357 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@694] - Established session 0x100137139d30002 with negotiated timeout 5000 for client /114.87.234.228:57417
2021-11-18 00:14:25,024 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@487] - Processed session termination for sessionid: 0x100137139d30002
2021-11-18 00:14:25,026 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1056] - Closed socket connection for client /114.87.234.228:57417 which had sessionid 0x100137139d30002
2021-11-18 00:14:25,730 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@215] - Accepted socket connection from /114.87.234.228:57403
2021-11-18 00:14:25,731 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@949] - Client attempting to establish new session at /114.87.234.228:57403
2021-11-18 00:14:25,733 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@694] - Established session 0x100137139d30003 with negotiated timeout 5000 for client /114.87.234.228:57403

zookeeper客户端连接的截图
在这里插入图片描述

kafka的日志
在这里插入图片描述

[2021-11-18 00:18:27,177] INFO Client environment:java.library.path=/usr/lib/jvm/zulu8-ca/jre/lib/amd64/server:/usr/lib/jvm/zulu8-ca/jre/lib/amd64:/usr/lib/jvm/zulu8-ca/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,177] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.version=5.10.60-9.al8.x86_64 (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.memory.free=973MB (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,178] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,182] INFO Initiating client connection, connectString=106.14.132.94:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@4466af20 (org.apache.zookeeper.ZooKeeper)
[2021-11-18 00:18:27,215] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2021-11-18 00:18:27,223] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2021-11-18 00:18:27,226] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)

进入容器内部发送消息
在这里插入图片描述

`bash-5.1# ./kafka-console-producer.sh --broker-list 106.14.132.94:9092 --topic sun

[2021-11-18 00:17:06,318] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,319] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,413] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,414] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,514] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,514] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,715] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:06,716] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:08,276] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:08,276] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:09,031] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:09,031] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:11,092] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:11,092] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:13,012] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:13,012] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:14,017] WARN [Producer clientId=console-producer] Connection to node -1 (/106.14.132.94:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-18 00:17:14,017] WARN [Producer clientId=console-producer] Bootstrap broker 106.14.132.94:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
`

又找了找,还是没什么用
在这里插入图片描述用netstat查看网络连接状态

netstat -antp | grep 2181

用tcpdump抓包和wireshark分析

tcpdump -vv host 106.14.132.94 and port 2181 -w 2181.cap
tcpdump -r 2181.cap

在这里插入图片描述
在这里插入图片描述没有发现什么有用信息,这个时候原来的sasl认证的错误信息没有了,出现了新的问题。
zookeeper还是老样子,可以连接上
在这里插入图片描述
后续发现将命令绑定的ip替换成内网ip就好了,因为不用内网会出现以下的这种情况
kafka这个时候删除重来,命令如下:

docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka

firewall-cmd --add-port=8010/tcp --permanent
firewall-cmd --add-port=8011/tcp --permanent
firewall-cmd --reload

systemctl restart docker

firewall-cmd --query-port=2181/tcp
firewall-cmd --query-port=9092/tcp

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2  --name zookeeper \
-p 2181:2181 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/zookeeper


docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name kafka \
-p 9092:9092 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=172.21.17.47:2181 \
-e KAFKA_ADVERTISED_HOST_NAME=172.21.17.47 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.21.17.47:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://172.21.17.47:9092 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/kafka

然后又出现下面的这种情况了:Opening socket connection to server 172.21.17.47/172.21.17.47:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
Opening socket connection to server 172.21.17.47/172.21.17.47:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)又回到一开始的问题了,Will not attempt to authenticate using SASL 但是发现截图好像是多了一个ip地址:
Will not attempt to authenticate using SASL后面发现我的启动kafka的命令中配置错了,kafka的启动命令多加了二行配置

-e KAFKA_ADVERTISED_HOST_NAME=172.21.17.47 \
-e KAFKA_ADVERTISED_PORT=9092 \

调整后的整套启动命令

docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka

firewall-cmd --add-port=2181/tcp --permanent
firewall-cmd --add-port=9092/tcp --permanent
firewall-cmd --reload

systemctl restart docker
firewall-cmd --query-port=2181/tcp
firewall-cmd --query-port=9092/tcp

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2  --name zookeeper \
-p 2181:2181 \
-v /etc/localtime:/etc/localtime \
-t wurstmeister/zookeeper

docker run -dit --restart=always --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name kafka \
-p 9092:9092 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=172.21.17.47:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.21.17.47:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 wurstmeister/kafka

然后进入kafka容器进行校验

cd /opt/kafka_2.13-2.7.1
bin/kafka-topics.sh --create --zookeeper 106.14.132.94:2181 --replication-factor 1 --partitions 1 --topic mymytest
bin/kafka-topics.sh --list --zookeeper 106.14.132.94:2181
bin/kafka-console-producer.sh --broker-list 106.14.132.94:9092 --topic mytest
bin/kafka-console-consumer.sh --bootstrap-server 106.14.132.94:9092--topic mytest
bin/kafka-console-consumer.sh --bootstrap-server 106.14.132.94:9092 --from-beginning --topic mytest

kafka到此,这个问题彻底解决了,这个错误提示真的很误导人,同时也没有出现需要绕开zookeeper的sasl认证机制的错误了,一切都是细节。


📥博主的话

🚀🎉希望各位读者大大多多支持用心写文章的博主,现在时代变了,🚀🎉 信息爆炸,酒香也怕巷子深🔥,博主真的需要大家的帮助才能在这片海洋中继续发光发热🎨,所以,🏃💨赶紧动动你的小手,点波关注❤️,点波赞👍,点波收藏⭐,甚至点波评论✍️,都是对博主最好的支持和鼓励!

关注我

我是廖志伟
我是廖志伟
我是廖志伟

📥博主出版的书

🎥经过多年在优快云创作上千篇文章的经验积累,我已经拥有了不错的写作技巧,与清华大学出版社合作,签下了四本书籍的合约,并将陆续出版。

这些书籍包括了

具体出版计划会根据实际情况进行调整,希望各位读者朋友能够多多支持!

📚《Java项目实战——深入理解大型互联网企业通用技术(基础篇)》

封面如下图所示:
Java项目实战——深入理解大型互联网企业通用技术(基础篇)
书评如下图所示:
Java项目实战——深入理解大型互联网企业通用技术(基础篇)

📚《Java项目实战——深入理解大型互联网企业通用技术(进阶篇)》

封面如下图所示:
Java项目实战——深入理解大型互联网企业通用技术(进阶篇)书评如下图所示:
Java项目实战——深入理解大型互联网企业通用技术(进阶篇)

🌟 Java开发学习路线

名称链接
JavaSEhttps://blog.youkuaiyun.com/java_wxid/category_8600390.html
MySQL专栏https://blog.youkuaiyun.com/java_wxid/category_8600406.html
JDBC专栏https://blog.youkuaiyun.com/java_wxid/category_8600515.html
MyBatis专栏https://blog.youkuaiyun.com/java_wxid/category_8600566.html
Web专栏https://blog.youkuaiyun.com/java_wxid/category_8600518.html
Spring专栏https://blog.youkuaiyun.com/java_wxid/category_8600582.html
SpringMvc专栏https://blog.youkuaiyun.com/java_wxid/category_8600599.html
SpringBoot专栏https://blog.youkuaiyun.com/wangb_java/category_9268684.html
SpringCould专栏https://blog.youkuaiyun.com/forezp/category_9268575.html
Redis专栏https://blog.youkuaiyun.com/java_wxid/category_8623958.html
Linux专栏https://blog.youkuaiyun.com/java_wxid/category_8600719.html
Maven3专栏https://blog.youkuaiyun.com/wangb_java/category_9268285.html
Spring Security5专栏https://blog.youkuaiyun.com/wangb_java/category_9284112.html
更多专栏更多专栏,请到主页查看
<think>好的,我现在需要解决用户关于ZooKeeper请求超时配置的问题,特别是涉及initRequestTimeout参数和如何处理相关警告的问题。首先,我得回忆一下ZooKeeper的相关配置参数和常见问题处理。 用户提到的是initRequestTimeout参数以及enabled=false的解决方案。我记得ZooKeeper的配置文件中确实有一些超时相关的设置,比如initLimit、syncLimit,但不确定是否有一个叫initRequestTimeout的参数。可能需要查阅官方文档或者社区资料确认这个参数是否存在,或者用户是否可能记错了参数名。 另外,用户提到了处理initRequestTimeout enabled=false的警告,这让我想到可能是日志中出现的警告信息,比如在启动时或者运行过程中ZooKeeper报出某个参数未正确配置或者被忽略的情况。例如,引用中的例子是关于HTTP日志中的警告,可能类似的情况发生在ZooKeeper的配置中。 接下来,我需要验证initRequestTimeout是否真实存在。通过搜索ZooKeeper的官方配置文档,发现常见的超时参数包括tickTime、initLimit、syncLimit、maxSessionTimeout等,并没有直接叫做initRequestTimeout的配置项。这可能意味着用户可能混淆了参数名称,或者该参数是某个特定版本或自定义构建中的参数。 如果参数确实不存在,那么用户遇到的警告可能是由于配置文件中错误地使用了initRequestTimeout这个参数,导致ZooKeeper无法识别,从而忽略该配置项,并输出类似“Ignored unknown option”的警告。此时解决方案应该是检查配置文件中的拼写错误,确认参数的正确性。 另外,用户提到enabled=false,这可能涉及到某个功能的开关,比如是否启用某种超时机制。如果initRequestTimeout是一个子参数,可能属于某个更大的配置块,比如服务器或客户端的配置部分。需要确认参数的结构是否正确,例如是否在正确的配置块内设置,或者是否需要启用父配置项才能生效。 例如,可能正确的参数是initLimit,该参数用于配置追随者与领导者初始连接时的超时时间,以tickTime的倍数表示。如果用户错误地写成了initRequestTimeout,那么将其更正为initLimit并设置合适的值可能解决问题。 此外,处理请求超时的通用方法包括检查网络延迟、调整会话超时时间(maxSessionTimeout和minSessionTimeout)、优化服务器性能等。可能用户的问题不仅仅是配置参数的问题,还涉及到整体环境的调优。 总结可能的解决步骤: 1. 检查配置文件中的参数名是否正确,是否存在拼写错误,将initRequestTimeout更正为正确的参数,如initLimit。 2. 如果initRequestTimeout是某个特定插件或扩展的配置项,确认相关组件是否正确安装和配置。 3. 查看日志文件,确认警告信息的具体内容,根据日志提示调整配置。 4. 调整其他相关超时参数,确保ZooKeeper集群的稳定运行。 5. 如果问题仍然存在,考虑升级ZooKeeper版本,查看是否在后续版本中修复了相关的问题。 同时,参考用户提供的引用内容,如ZOOKEEPER-2795和ZOOKEEPER-3421,这些可能是JIRA工单,可能涉及日志级别调整或连接处理的改进,可能与用户的警告相关,需要检查是否在特定版本中这些问题已被解决,建议用户升级到包含这些修复的版本。 最后,生成相关问题需要考虑用户可能进一步想了解的内容,比如如何调整其他超时参数、日志分析的方法、集群性能优化等。</think>关于ZooKeeper请求超时配置及`initRequestTimeout`警告的处理方案如下: --- ### 一、请求超时核心参数 ZooKeeper的请求超时行为主要由以下参数控制(需在`zoo.cfg`配置文件中设置): 1. **tickTime** 基础时间单位(默认2000ms),用于计算其他超时参数 示例:`tickTime=2000` 2. **initLimit** 允许追随者(follower)连接领导者的最大初始化时间,计算公式: $$超时时间 = tickTime \times initLimit$$ 默认值通常为5(即10秒) 示例:`initLimit=10`(对应20秒) 3. **syncLimit** 心跳包最大延迟容忍时间,计算公式: $$超时时间 = tickTime \times syncLimit$$ 默认值通常为2(即4秒) 示例:`syncLimit=5`(对应10秒) --- ### 二、`initRequestTimeout`相关警告分析 1. **参数来源** `initRequestTimeout`并非ZooKeeper官方配置参数,可能出现在以下场景: - 第三方客户端库的自定义配置 - 旧版本兼容参数(如3.4.x之前) - 配置文件中存在拼写错误 2. **警告解决方案** 若日志出现`initRequestTimeout enabled=false`类警告,建议按以下步骤处理: ```properties # 检查zoo.cfg配置文件 grep -i "initRequestTimeout" conf/zoo.cfg # 正确做法:移除无效参数 # 若需控制初始化连接,改用initLimit: initLimit=10 ``` --- ### 三、请求超时优化实践 | 场景 | 调整方向 | 配置示例 | |------|----------|----------| | 高延迟网络 | 增大`initLimit`和`syncLimit` | `initLimit=15`<br>`syncLimit=8` | | 频繁会话超时 | 调整`maxSessionTimeout` | `maxSessionTimeout=40000` | | 客户端连接不稳定 | 客户端设置重试策略 | `curator-retry-policy: 5次重试` | --- ### 四、关键日志排查 检查ZooKeeper日志(默认路径`zookeeper.out`): ```log # 典型连接超时日志 2023-01-01 12:00:00,123 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@584] - Cannot open channel to 2 at election address zk-node2:3888 java.net.ConnectException: Connection timed out ``` **处理建议**: 1. 使用`telnet zk-node2 3888`验证网络连通性 2. 通过`mntr`命令检查集群状态: ```bash echo mntr | nc localhost 2181 ``` ---
评论 8
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

Java程序员廖志伟

赏我包辣条呗

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值