kafka 工具 kafka-topic.sh
操作
kafka-topic.sh有6种操作:
创建topic – create
[lch@linux129 bin]$ ./kafka-topics.sh --create --zookeeper ip:2181,ip:2181 --replication-factor 2 --partitions 20 --topic TEST
参数说明:
--replication-factor : 表明 副本的个数, 这里写成2,表明会有两个副本
--partitions : 表明会有20个partition, 表现在log日志中,就是有20个文件夹,名称以“topic-编号” ,编号从0~19
执行完成后去kafka数据目录下查看,会看到创建了两个文件夹
test-0
test-1
test-2
注: 在leader节点上面才是20个文件夹,如果是从节点上面,可能只有一个也可能没有
列出全部的topic –list命令
[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181 --list
__consumer_offsets
crashMessage
mobileActions
mobileActionsWrappedMessages
mobileErrors
mobileErrorsWrappedMessages
mobileInteractionTraces
mobileInteractionTracesWrappedMessages
mobileReportUsers
mobileReportUsersWrappedMessages
列出全部topic的信息 –describe
包括每个分区的leader,副本,isr信息
[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181 --describe
Topic:__consumer_offsets PartitionCount:50 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
Topic: __consumer_offsets Partition: 0 Leader: 4 Replicas: 4,0,1 Isr: 0,1,4
Topic: __consumer_offsets Partition: 1 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
Topic: __consumer_offsets Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 1,3,2
Topic: __consumer_offsets Partition: 3 Leader: 2 Replicas: 2,3,4 Isr: 3,4,2
Topic: __consumer_offsets Partition: 4 Leader: 3 Replicas: 3,4,0 Isr: 0,3,4
Topic: __consumer_offsets Partition: 5 Leader: 4 Replicas: 4,1,2 Isr: 1,4,2
Topic: __consumer_offsets Partition: 6 Leader: 0 Replicas: 0,2,3 Isr: 0,3,2
Topic: __consumer_offsets Partition: 7 Leader: 1 Replicas: 1,3,4 Isr: 1,3,4
Topic: __consumer_offsets Partition: 8 Leader: 2 Replicas: 2,4,0 Isr: 0,4,2
Topic: __consumer_offsets Partition: 9 Leader: 3 Replicas: 3,0,1 Isr: 0,1,3
Topic: __consumer_offsets Partition: 10 Leader: 4 Replicas: 4,2,3 Isr: 3,4,2
Topic: __consumer_offsets Partition: 11 Leader: 0 Replicas: 0,3,4 Isr: 0,3,4
Topic: __consumer_offsets Partition: 12 Leader: 0 Replicas: 1,4,0 Isr: 0,1,4
Topic: __consumer_offsets Partition: 13 Leader: 2 Replicas: 2,0,1 Isr: 0,1,2
Topic: __consumer_offsets Partition: 14 Leader: 3 Replicas: 3,1,2 Isr: 1,3,2
Topic: __consumer_offsets Partition: 15 Leader: 4 Replicas: 4,3,0 Isr: 0,3,4
Topic: __consumer_offsets Partition: 16 Leader: 0 Replicas: 0,4,1 Isr: 0,1,4
Topic: __consumer_offsets Partition: 17 Leader: 0 Replicas: 1,0,2 Isr: 0,1,2
Topic: __consumer_offsets Partition: 18 Leader: 2 Replicas: 2,1,3 Isr: 1,3,2
Topic: __consumer_offsets Partition: 19 Leader: 3 Replicas: 3,2,4 Isr: 3,4,2
Topic: __consumer_offsets Partition: 20 Leader: 4 Replicas: 4,0,1 Isr: 0,1,4
Topic: __consumer_offsets Partition: 21 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
Topic: __consumer_offsets Partition: 22 Leader: 1 Replicas: 1,2,3 Isr: 1,3,2
Topic: __consumer_offsets Partition: 23 Leader: 2 Replicas: 2,3,4 Isr: 3,4,2
Topic: __consumer_offsets Partition: 24 Leader: 3 Replicas: 3,4,0 Isr: 0,3,4
Topic: __consumer_offsets Partition: 25 Leader: 4 Replicas: 4,1,2 Isr: 1,4,2
Topic: __consumer_offsets Partition: 26 Leader: 0 Replicas: 0,2,3 Isr: 0,3,2
Topic: __consumer_offsets Partition: 27 Leader: 1 Replicas: 1,3,4 Isr: 1,3,4
Topic: __consumer_offsets Partition: 28 Leader: 2 Replicas: 2,4,0 Isr: 0,4,2
Topic: __consumer_offsets Partition: 29 Leader: 3 Replicas: 3,0,1 Isr: 0,1,3
Topic: __consumer_offsets Partition: 30 Leader: 4 Replicas: 4,2,3 Isr: 3,4,2
Topic: __consumer_offsets Partition: 31 Leader: 0 Replicas: 0,3,4 Isr: 0,3,4
Topic: __consumer_offsets Partition: 32 Leader: 0 Replicas: 1,4,0 Isr: 0,1,4
Topic: __consumer_offsets Partition: 33 Leader: 2 Replicas: 2,0,1 Isr: 0,1,2
Topic: __consumer_offsets Partition: 34 Leader: 3 Replicas: 3,1,2 Isr: 1,3,2
Topic: __consumer_offsets Partition: 35 Leader: 4 Replicas: 4,3,0 Isr: 0,3,4
Topic: __consumer_offsets Partition: 36 Leader: 0 Replicas: 0,4,1 Isr: 0,1,4
Topic: __consumer_offsets Partition: 37 Leader: 0 Replicas: 1,0,2 Isr: 0,1,2
Topic: __consumer_offsets Partition: 38 Leader: 2 Replicas: 2,1,3 Isr: 1,3,2
Topic: __consumer_offsets Partition: 39 Leader: 3 Replicas: 3,2,4 Isr: 3,4,2
Topic: __consumer_offsets Partition: 40 Leader: 4 Replicas: 4,0,1 Isr: 0,1,4
Topic: __consumer_offsets Partition: 41 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
Topic: __consumer_offsets Partition: 42 Leader: 1 Replicas: 1,2,3 Isr: 1,3,2
Topic: __consumer_offsets Partition: 43 Leader: 2 Replicas: 2,3,4 Isr: 3,4,2
Topic: __consumer_offsets Partition: 44 Leader: 3 Replicas: 3,4,0 Isr: 0,3,4
Topic: __consumer_offsets Partition: 45 Leader: 4 Replicas: 4,1,2 Isr: 1,4,2
Topic: __consumer_offsets Partition: 46 Leader: 0 Replicas: 0,2,3 Isr: 0,3,2
Topic: __consumer_offsets Partition: 47 Leader: 1 Replicas: 1,3,4 Isr: 1,3,4
Topic: __consumer_offsets Partition: 48 Leader: 2 Replicas: 2,4,0 Isr: 0,4,2
Topic: __consumer_offsets Partition: 49 Leader: 3 Replicas: 3,0,1 Isr: 0,1,3
Topic:TEST PartitionCount:20 ReplicationFactor:1 Configs:
Topic: TEST Partition: 0 Leader: 4 Replicas: 4 Isr: 4
Topic: TEST Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: TEST Partition: 2 Leader: 1 Replicas: 1 Isr: 1
Topic: TEST Partition: 3 Leader: 2 Replicas: 2 Isr: 2
Topic: TEST Partition: 4 Leader: 3 Replicas: 3 Isr: 3
Topic: TEST Partition: 5 Leader: 4 Replicas: 4 Isr: 4
Topic: TEST Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: TEST Partition: 7 Leader: 1 Replicas: 1 Isr: 1
Topic: TEST Partition: 8 Leader: 2 Replicas: 2 Isr: 2
Topic: TEST Partition: 9 Leader: 3 Replicas: 3 Isr: 3
Topic: TEST Partition: 10 Leader: 4 Replicas: 4 Isr: 4
Topic: TEST Partition: 11 Leader: 0 Replicas: 0 Isr: 0
Topic: TEST Partition: 12 Leader: 1 Replicas: 1 Isr: 1
Topic: TEST Partition: 13 Leader: 2 Replicas: 2 Isr: 2
Topic: TEST Partition: 14 Leader: 3 Replicas: 3 Isr: 3
Topic: TEST Partition: 15 Leader: 4 Replicas: 4 Isr: 4
Topic: TEST Partition: 16 Leader: 0 Replicas: 0 Isr: 0
Topic: TEST Partition: 17 Leader: 1 Replicas: 1 Isr: 1
Topic: TEST Partition: 18 Leader: 2 Replicas: 2 Isr: 2
Topic: TEST Partition: 19 Leader: 3 Replicas: 3 Isr: 3
删除topic –delete
[root@hadoop221 bin]# ./kafka-topics.sh --zookeeper ip:port --topic TEST --delete
topic无法删掉
Topic TEST is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
需要增加配置参数:delete.topic.enable=true
修改topic时配置参数 –alter –config
覆盖已经有topic参数
[root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --topic TEST --alter --config flush.messages=1
删除topic级别配置参数
[root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --delete-config flush.messages=1
修改topic分区数量
[root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --partitions 6