kafka集群搭建

安装vim

安装ssh sudo apt-get install openssh-server

service ssh start

192.168.1.102

192.168.1.103

192.168.1.104

sudo mkdir /data/zookeeper/data

sudo mkdir /data/zookeeper/log

创建myid

192.168.1.102

echo "1" > /tmp/zookeeper/myid

192.168.1.103

echo "2" > /tmp/zookeeper/myid

192.168.1.104

echo "3" > /tmp/zookeeper/myid

三台机器分别执行如下操作

搭建zookeeper集群

进入kafka根目录 /home/feng/kafka

修改zookeeper配置文件

dataDir=/tmp/zookeeper

# the port at which the clients will connect

clientPort=2181

# disable the per-ip limit on the number of connections since this is a non-production config

maxClientCnxns=200

initLimit=5

syncLimit=2

server.1=192.168.1.102:2888:3888

server.2=192.168.1.103:2888:3888

server.3=192.168.1.104:2888:3888

启动zookeeper

nohup ./bin/zookeeper-server-start.sh config/zookeeper.properties >logs/zookeeper.log 2>1 &

查看zookeeper日志是否报错，如果报错，更改三台机器的启动顺序，重新启动，用来确定leader和follower

最后查看2181端口是否启动以确定是否成功

netstat -anput | grep 2181

搭建kafka集群

进入kafka根目录 /home/feng/kafka

修改kafka配置文件server.properties

分别修改

broker.id=1    //本机myid
host.name=192.168.1.102   //本机ip

advertised.host.name=192.168.1.102   //本机ip

zookeeper.connect=192.168.1.102:2181,192.168.1.103:2181,192.168.1.104:2181   //zookeeper集群

启动kafka集群

nohup ./bin/kafka-server-start.sh config/server.properties >logs/kafka.log 2>1 &

测试kafka集群

1-进入kafka根目录，创建topic--test

bin/kafka-topics.sh --create --zookeeper 192.168.1.xx:2181,192.168.1.xx:2181,192.168.1.xx:2181 --replication-factor 1 --partitions 1 --topic test

2-列出已创建的topic列表

bin/kafka-topics.sh --list --zookeeper localhost:2181

3-模拟客户端去发送消息

bin/kafka-console-producer.sh --broker-list 192.168.1.xx:9092,192.168.1.xx:9092,192.168.1.xx:9092 --topic test

4-模拟客户端去接受消息

bin/kafka-console-consumer.sh --zookeeper 192.168.1.xx:2181,192.168.1.xx:2181,192.168.1.xx:2181 --from-beginning --topic test

7、自己测试结果

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic test

本文转自 无心低语 51CTO博客，原文链接：http://blog.51cto.com/fengzhankui/1927753，如需转载请自行联系原作者