kafka 环境搭建及验证

下载

git clone https://github.com/wurstmeister/kafka-docker

配置

  进入 root 权限
  sudo su   
  建立目录
  mkdir /usr/local/kafka
  备份一下  docker-compose.yml  然后修改中间的主机名，不要用127.0.0.1
  将 
      KAFKA_ADVERTISED_HOST_NAME: 192.168.1.84   

启动

启动集群

docker-compose -f docker-compose-single-broker.yml up -d

停止集群

docker-compose -f docker-compose-single-broker.yml down

测试

安装 python 环境

apt install python-kafka python3-kafka

运行消费者

python consumer.py

from kafka import KafkaConsumer

#connect to Kafka server and pass the topic we want to consume
consumer = KafkaConsumer('my-topic', bootstrap_servers = ['192.168.1.84:9092'])

for msg in consumer:
   print msg

运行生产者

python product.py

import time
from kafka import SimpleProducer, KafkaClient
from kafka import KafkaProducer


producer = KafkaProducer(bootstrap_servers = ['192.168.1.84:9092'])
# Assign a topic
topic = 'my-topic'

def test():
    print('begin')
    n = 1
    while (n<=100):
        msg="{\"top\":" + str(n) + "}"
        producer.send(topic, msg )
        print "send" + msg
        n += 1
        time.sleep(0.5)
    print('done')

if __name__ == '__main__':
    test()