kafka工具kafka-topic.sh

本文介绍如何使用kafka-topic.sh工具进行Kafka Topic的基本管理,包括创建、列出、描述及删除等操作,并解释了相关参数的意义。

kafka 工具 kafka-topic.sh

操作

kafka-topic.sh有6种操作:

创建topic – create

[lch@linux129 bin]$ ./kafka-topics.sh --create --zookeeper ip:2181,ip:2181 --replication-factor 2 --partitions 20 --topic TEST
参数说明:
--replication-factor    : 表明 副本的个数, 这里写成2,表明会有两个副本
--partitions              :  表明会有20个partition, 表现在log日志中,就是有20个文件夹,名称以“topic-编号” ,编号从0~19

执行完成后去kafka数据目录下查看,会看到创建了两个文件夹
    test-0
    test-1
    test-2

注: 在leader节点上面才是20个文件夹,如果是从节点上面,可能只有一个也可能没有

列出全部的topic –list命令

[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181  --list
__consumer_offsets
crashMessage
mobileActions
mobileActionsWrappedMessages
mobileErrors
mobileErrorsWrappedMessages
mobileInteractionTraces
mobileInteractionTracesWrappedMessages
mobileReportUsers
mobileReportUsersWrappedMessages

列出全部topic的信息 –describe

包括每个分区的leader,副本,isr信息

[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181  --describe
Topic:__consumer_offsets    PartitionCount:50   ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
    Topic: __consumer_offsets   Partition: 0    Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 1    Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 2    Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 3    Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 4    Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 5    Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 6    Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 7    Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 8    Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 9    Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
    Topic: __consumer_offsets   Partition: 10   Leader: 4   Replicas: 4,2,3 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 11   Leader: 0   Replicas: 0,3,4 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 12   Leader: 0   Replicas: 1,4,0 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 13   Leader: 2   Replicas: 2,0,1 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 14   Leader: 3   Replicas: 3,1,2 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 15   Leader: 4   Replicas: 4,3,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 16   Leader: 0   Replicas: 0,4,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 17   Leader: 0   Replicas: 1,0,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 18   Leader: 2   Replicas: 2,1,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 19   Leader: 3   Replicas: 3,2,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 20   Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 21   Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 22   Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 23   Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 24   Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 25   Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 26   Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 27   Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 28   Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 29   Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
    Topic: __consumer_offsets   Partition: 30   Leader: 4   Replicas: 4,2,3 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 31   Leader: 0   Replicas: 0,3,4 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 32   Leader: 0   Replicas: 1,4,0 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 33   Leader: 2   Replicas: 2,0,1 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 34   Leader: 3   Replicas: 3,1,2 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 35   Leader: 4   Replicas: 4,3,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 36   Leader: 0   Replicas: 0,4,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 37   Leader: 0   Replicas: 1,0,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 38   Leader: 2   Replicas: 2,1,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 39   Leader: 3   Replicas: 3,2,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 40   Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 41   Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 42   Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 43   Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 44   Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 45   Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 46   Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 47   Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 48   Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 49   Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
Topic:TEST  PartitionCount:20   ReplicationFactor:1 Configs:
    Topic: TEST Partition: 0    Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 1    Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 2    Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 3    Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 4    Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 5    Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 6    Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 7    Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 8    Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 9    Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 10   Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 11   Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 12   Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 13   Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 14   Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 15   Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 16   Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 17   Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 18   Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 19   Leader: 3   Replicas: 3 Isr: 3

删除topic –delete

 [root@hadoop221 bin]# ./kafka-topics.sh  --zookeeper ip:port  --topic TEST  --delete

topic无法删掉
Topic TEST is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

需要增加配置参数:delete.topic.enable=true

修改topic时配置参数 –alter –config

覆盖已经有topic参数

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port  --topic TEST --alter --config flush.messages=1

删除topic级别配置参数

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --delete-config flush.messages=1

修改topic分区数量

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --partitions 6
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值