Kafka基本命令

本文介绍了Kafka的基本命令操作,包括启动Kafka和Zookeeper,查看及创建topic,使用生产者和消费者,以及删除topic。在创建topic时,需要注意复制因子不能超过可用broker数量。通过kafka-console-producer.sh和kafka-console-consumer.sh命令进行消息生产和消费。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

启动

  1. zk.sh start
  2. kafka.sh start

查看topic列表

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --list

__consumer_offsets

创建topic

[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 3 --partitions 2 --topic mytopic1

Created topic "mytopic1".

//查看主题

[hadoop@hadoop1 ~]$ cd /opt/module/kafka/logs

[hadoop@hadoop1 logs]$ cd mytopic1-0 

[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 2 --partitions 3 --topic mytopic2

Created topic "mytopic2".

[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 4 --partitions 2 --topic mytopic3

Error while executing topic command : replication factor: 4 larger than available brokers: 3

[2020-10-16 22:27:48,002] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 4 larger than available brokers: 3

 (kafka.admin.TopicCommand$)

查看topic

[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --describe --topic mytopic2

Topic:mytopic2 PartitionCount:3 ReplicationFactor:2 Configs:

Topic: mytopic2 Partition: 0 Leader: 2 Replicas: 2,0 Isr: 2,0

Topic: mytopic2 Partition: 1 Leader: 0 Replicas: 0,1 Isr: 0,1

Topic: mytopic2 Partition: 2 Leader: 1 Replicas: 1,2 Isr: 1,2

生产者

[hadoop@hadoop1 logs]$ kafka-console-producer.sh --broker-list hadoop1:9092 --topic mytopic2

>hello

>world

消费者

[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --zookeeper hadoop1:2181 --topic mytopic2

Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].

[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic mytopic2 --from-beginning

world

hello

设定组名

[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest --consumer-property group.id=g1

删除topic

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --delete --topic mytopic1

Topic mytopic1 is marked for deletion.

Note: This will have no impact if delete.topic.enable is not set to true.

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --delete --topic mytopic2

Topic mytopic2 is marked for deletion.

Note: This will have no impact if delete.topic.enable is not set to true.

[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic __consumer_offsets --from-beginning --formatter "kafka.coordinator.group.GroupMetadataManager\$OffsetsMessageFormatter"

 

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值