kafka常用命令总结

1. 启动kafka
   nohup bin/kafka-server-start.sh config/server.properties > nohup.log 2>&1 &（&表示后台运行）
2. 停止kafka
   bin/kafka-server-stop.sh
3. 创建topic
   bin/kafka-topics.sh –create –zookeeper localhost:2181 –replication-factor 1 –partitions 1 –topic test
4. 展示topic
   bin/kafka-topics.sh –list –zookeeper localhost:2181
5. 描述topic
   bin/kafka-topics.sh –describe –zookeeper localhost:2181 –topic test
6. 生产者：
   bin/kafka-console-producer.sh –broker-list 130.51.23.95:9092 –topic test
7. 消费者：
   bin/kafka-console-consumer.sh –zookeeper 130.51.23.95:2181 –topic test –from-beginnin
8. 删除topic
   bin/kafka-topics.sh –delete –zookeeper localhost:2181 –topic test （删除topic中存储的内容在config/server.properties中找到如下的位置，需要把server.properties设置delete.topic.enable=true）
9. 下线broker
   ./kafka-run-class.sh kafka.admin.ShutdownBroker –zookeeper 127.0.0.1:2181 –broker #brokerId# –num.retries 3 –retry.interval.ms 60
   shutdown broker
10. 为topic增加partition
    ./bin/kafka-topics.sh –zookeeper 127.0.0.1:2181 –alter –partitions 20 –topic testKJ1
11. 查询消费信息：
    bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker –zookeeper localhost:2181 –group group_order_ledger –topic payment_completed_dev
12. 为topic增加副本
    ./kafka-reassign-partitions.sh -zookeeper127.0.0.1:2181-reassignment-json-file json/partitions-to-move.json -execute
    首先来看看我们操作的Topic相关的信息

[iteblog@www.iteblog.com ~]$ kafka-topics.sh --topic iteblog --describe --zookeeper www.iteblog.com:2181
Topic:iteblog  PartitionCount:2  ReplicationFactor:1 Configs:
Topic: iteblog Partition: 0  Leader: 2 Replicas: 2 Isr: 2
Topic: iteblog Partition: 1  Leader: 3 Replicas: 3 Isr: 3

从上面的输出信息可以看出，iteblog主题有两个分区，副本数为1。假如我们需要将这个主题的副本数增加到3，以便提供高可用性。
　　动态地增加相关主题的副本数非常的简单，同样是使用kafka-reassign-partitions.sh工具来执行，所以我们在操作之前得准备一个Json文件，里面详细地记录着每个分区的副本所在机器等信息，具体如下：

{
    "version": 1, 
    "partitions": [
        {
            "topic": "iteblog", 
            "partition": 0, 
            "replicas": [
                2, 
                1, 
                3
            ]
        },
        {
            "topic": "iteblog", 
            "partition": 1, 
            "replicas": [
                3, 
                2, 
                1
            ]
        }
    ]
}

上面已经把主题的副本数变成3了，我们把这个Json数据存放在名为replication.json的文件中。现在我们需要执行这个配置文件，如下：

[iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"iteblog","partition":0,"replicas":[2]},{"topic":"iteblog","partition":1,"replicas":[3]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"iteblog","partition":0,"replicas":[2,1,3]},{"topic":"iteblog","partition":1,"replicas":[3,2,1]}]}

同样我们可以使用verify参数来检查上面的执行计划是否完成，如下：

[iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --verify
Status of partition reassignment:
Reassignment of partition [iteblog,0] is still in progress
Reassignment of partition [iteblog,1] is still in progress

[iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --verify
Status of partition reassignment:
Reassignment of partition [iteblog,0] completed successfully
Reassignment of partition [iteblog,1] completed successfully

好了，我们可以看出上面的执行计划已经完成。我们来查看一下现在iteblog主题的相关信息：

[iteblog@www.iteblog.com ~]$ kafka-topics.sh --topic iteblog --describe --zookeeper www.iteblog.com:2181
Topic:iteblog  PartitionCount:2  ReplicationFactor:3 Configs:
Topic: iteblog Partition: 0  Leader: 2 Replicas: 2,1,3 Isr: 2,1,3
Topic: iteblog Partition: 1  Leader: 3 Replicas: 3,2,1 Isr: 3,2,1

可以看出，iteblog主题的副本数已经变成了3。