启动
- zk.sh start
- kafka.sh start
查看topic列表
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --list
__consumer_offsets
创建topic
[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 3 --partitions 2 --topic mytopic1
Created topic "mytopic1".
//查看主题
[hadoop@hadoop1 ~]$ cd /opt/module/kafka/logs
[hadoop@hadoop1 logs]$ cd mytopic1-0
[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 2 --partitions 3 --topic mytopic2
Created topic "mytopic2".
[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --create --replication-factor 4 --partitions 2 --topic mytopic3
Error while executing topic command : replication factor: 4 larger than available brokers: 3
[2020-10-16 22:27:48,002] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 4 larger than available brokers: 3
(kafka.admin.TopicCommand$)
查看topic
[hadoop@hadoop1 logs]$ kafka-topics.sh --zookeeper hadoop1:2181 --describe --topic mytopic2
Topic:mytopic2 PartitionCount:3 ReplicationFactor:2 Configs:
Topic: mytopic2 Partition: 0 Leader: 2 Replicas: 2,0 Isr: 2,0
Topic: mytopic2 Partition: 1 Leader: 0 Replicas: 0,1 Isr: 0,1
Topic: mytopic2 Partition: 2 Leader: 1 Replicas: 1,2 Isr: 1,2
生产者
[hadoop@hadoop1 logs]$ kafka-console-producer.sh --broker-list hadoop1:9092 --topic mytopic2
>hello
>world
消费者
[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --zookeeper hadoop1:2181 --topic mytopic2
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic mytopic2 --from-beginning
world
hello
设定组名
[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest --consumer-property group.id=g1
删除topic
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --delete --topic mytopic1
Topic mytopic1 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181 --delete --topic mytopic2
Topic mytopic2 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
[hadoop@hadoop1 ~]$ kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic __consumer_offsets --from-beginning --formatter "kafka.coordinator.group.GroupMetadataManager\$OffsetsMessageFormatter"