filebeat+kafka失败

最新推荐文章于 2025-01-05 15:40:18 发布

转载最新推荐文章于 2025-01-05 15:40:18 发布 · 716 阅读

1 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/zhzhang/p/6789711.html

文章标签：

#大数据 #网络

本文记录了Filebeat配置Kafka输出时遇到的问题及解决办法。主要问题是由于Filebeat尝试解析Kafka broker的主机名失败导致无法建立连接。解决方法是在Filebeat所在机器的hosts文件中绑定正确的IP地址和主机名。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

filebeat端配置

#----------------------------- Kafka output --------------------------------
output.kafka:
  # initial brokers for reading cluster metadata
  hosts: ["10.88.128.88:9092"]
  version: "0.8.2"

  topic: 'abc'
  #partition.round_robin:
  #  reachable_only: false

  required_acks: 1
  compression: none 
  #max_message_bytes: 1000000

报错如下：

2017/04/30 08:14:24.060269 log.go:12: WARN producer/broker/0 state change to [closing] because dial tcp: lookup test43 on 172.16.2.1:53: no such host
2017/04/30 08:14:24.060361 log.go:12: WARN producer/leader/rokid/0 state change to [retrying-48]
2017/04/30 08:14:24.060405 log.go:12: WARN producer/leader/rokid/0 abandoning broker 0
2017/04/30 08:14:24.060450 log.go:12: WARN producer/broker/0 shut down
x2017/04/30 08:14:24.160582 log.go:12: WARN client/metadata fetching metadata for [rokid] from broker 10.88.128.88:9092
2017/04/30 08:14:24.164350 log.go:12: WARN producer/broker/0 starting up
2017/04/30 08:14:24.164388 log.go:12: WARN producer/broker/0 state change to [open] on rokid/0
2017/04/30 08:14:24.164405 log.go:12: WARN producer/leader/rokid/0 selected broker 0
2017/04/30 08:14:24.164505 log.go:12: WARN producer/leader/rokid/0 state change to [flushing-48]
2017/04/30 08:14:24.164516 log.go:12: WARN producer/leader/rokid/0 state change to [normal]
i2017/04/30 08:14:24.185439 log.go:12: WARN Failed to connect to broker test43:9092: dial tcp: lookup test43 on 172.16.2.1:53: no such host
2017/04/30 08:14:24.185488 log.go:12: WARN producer/broker/0 state change to [closing] because dial tcp: lookup test43 on 172.16.2.1:53: no such host
2017/04/30 08:14:24.185534 log.go:12: WARN producer/leader/rokid/0 state change to [retrying-49]
2017/04/30 08:14:24.185552 log.go:12: WARN producer/leader/rokid/0 abandoning broker 0
2017/04/30 08:14:24.185592 log.go:12: WARN producer/broker/0 shut down
2017/04/30 08:14:24.285740 log.go:12: WARN client/metadata fetching metadata for [rokid] from broker 10.88.128.88:9092
2017/04/30 08:14:24.287170 log.go:12: WARN producer/broker/0 starting up
2017/04/30 08:14:24.287191 log.go:12: WARN producer/broker/0 state change to [open] on rokid/0
2017/04/30 08:14:24.287205 log.go:12: WARN producer/leader/rokid/0 selected broker 0
2017/04/30 08:14:24.287300 log.go:12: WARN producer/leader/rokid/0 state change to [flushing-49]
2017/04/30 08:14:24.287311 log.go:12: WARN producer/leader/rokid/0 state change to [normal]
2017/04/30 08:14:24.304521 log.go:12: WARN Failed to connect to broker test43:9092: dial tcp: lookup test43 on 172.16.2.1:53: no such host