Kafka2.2.*CDH命令行命令_bootstrap-servers: hadoop01:9092,hadoop02:9092,had-优快云博客

本文链接：https://blog.youkuaiyun.com/weixin_44675844/article/details/109539669

本文介绍了Kafka 2.2.*在CDH环境中的命令行操作，包括查看所有topic、使用生产者和消费者命令、从头开始消费数据以及详细查询消费者组和topic信息。由于新版本不依赖zookeeper存储offset，需使用--bootstrap-server指定broker。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Kafka安装完成启动后，测试连不起来，发现版本比以前高，他们的语法有点变化，zookeeper这个语法被淘汰掉了，因为新版本不存放offset在zk上了。使用–bootstrap-server来指定broker的位置

查看所有topic

kafka-topics --list --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092

生产者

kafka-console-producer --broker-list hadoop1:9092,hadoop2:9092,hadoop3:9092 --topic TP_LABEL

消费者

kafka-console-consumer --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --topic TP_LABEL

消费者-从头消费数据

kafka-console-consumer --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --from-beginning --topic TP_LABEL

查看topic主题详细信息

kafka-topics  --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --topic TP_LABEL --describe

查看消费者组

kafka-consumer-groups --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --list

查看指定消费者组详细信息(CONSUMER-ID HOST CLIENT-ID PARTITIONS ASSIGNMENT)

kafka-consumer-groups --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --group cloudera_mirrormaker --describe --members --verbose


CONSUMER-ID                                                 HOST            CLIENT-ID              #PARTITIONS     ASSIGNMENT
cloudera_mirrormaker-0-1d1a8103-3e9d-43c4-9a34-09f677fd049a /10.130.210.246 cloudera_mirrormaker-0 0               -
cloudera_mirrormaker-0-a6168ff9-e579-4536-b92a-fc6aaf82b8c1 /10.130.210.245 cloudera_mirrormaker-0 0               -
cloudera_mirrormaker-0-a110eabd-5191-49a0-acf4-dbdc7e9a602f /10.130.210.247 cloudera_mirrormaker-0 0               -

查看指定消费者组详细信息(CONSUMER-ID HOST CLIENT-ID PARTITIONS)

kafka-consumer-groups --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --group cloudera_mirrormaker --describe --members


//CONSUMER-ID                                                 HOST            CLIENT-ID              #PARTITIONS     
//cloudera_mirrormaker-0-1d1a8103-3e9d-43c4-9a34-09f677fd049a /10.130.210.246 cloudera_mirrormaker-0 0               
//cloudera_mirrormaker-0-a6168ff9-e579-4536-b92a-fc6aaf82b8c1 /10.130.210.245 cloudera_mirrormaker-0 0               
//cloudera_mirrormaker-0-a110eabd-5191-49a0-acf4-dbdc7e9a602f /10.130.210.247 cloudera_mirrormaker-0 0

查看指定消费者组详细信息(TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID)

kafka-consumer-groups --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --group cloudera_mirrormaker --describe


//TOPIC               PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                     HOST            CLIENT-ID
//xxx_trajectory 0          143693805       143693905       100             consumer-1-02f47385-5496-4094-9513-09c0b66b47bc /59.202.28.243  consumer-1
//xxx_trajectory 1          137186261       137186348       87              consumer-1-02f47385-5496-4094-9513-09c0b66b47bc /59.202.28.243  consumer-1

查看指定消费者组详细信息(COORDINATOR(ID) ASSIGNMENT-STRATEGY STATE MEMBERS)

kafka-consumer-groups --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --group cloudera_mirrormaker --describe --state


//COORDINATOR (ID)          ASSIGNMENT-STRATEGY       STATE                #MEMBERS
//hadoop3:9092 (63)         range                     Stable               3