kafka安装

一、安装zookeeper

1.在master节点上操作：

[root@QA-linux-1 src]# tar -zxvf zookeeper-3.4.10.tar.gz -C /usr/local/

[root@QA-linux-1 src]# cd ..

[root@QA-linux-1 local]# ln -s zookeeper-3.4.10 zookeeper

[root@QA-linux-1 local]# cd zookeeper/conf/

[root@QA-linux-1 conf]# cp zoo_sample.cfg zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/tmp/zookeeper

clientPort=2181

server.1=QA-linux-1:2888:3888

server.2=QA-linux-2:2888:3888

server.3=QA-linux-3:2888:3888

2.创建dataDir目录创建/tmp/zookeeper

# 在master节点上

[root@QA-linux-1 conf]# mkdir /tmp/zookeeper

[root@QA-linux-1 conf]# touch /tmp/zookeeper/myid

[root@QA-linux-1 conf]# echo 1 > /tmp/zookeeper/myid

3.将zookeeper文件复制到另外两个节点：

[root@QA-linux-1 local]# scp -r zookeeper-3.4.10/ 10.200.3.86:/usr/local/

[root@QA-linux-1 local]# scp -r zookeeper-3.4.10/ 10.200.3.87:/usr/local/

4.在两个slave节点创建目录和文件

#QA-linux-2节点:

[root@QA-linux-2 local]# ln -s zookeeper-3.4.10 zookeeper

[root@QA-linux-2 local]# mkdir /tmp/zookeeper

[root@QA-linux-2 local]# touch /tmp/zookeeper/myid

[root@QA-linux-2 local]# echo 2 > /tmp/zookeeper/myid

#QA-linux-3节点

[root@QA-linux-3 local]# ln -s zookeeper-3.4.10 zookeeper

[root@QA-linux-3 local]# mkdir /tmp/zookeeper

[root@QA-linux-3 local]# touch /tmp/zookeeper/myid

[root@QA-linux-3 local]# echo 3 > /tmp/zookeeper/myid

5.分别在每个节点上启动 zookeeper测试：

[root@QA-linux-1 zookeeper]# ./bin/zkServer.sh start

[root@QA-linux-2 zookeeper]# ./bin/zkServer.sh start

[root@QA-linux-3 zookeeper]# ./bin/zkServer.sh start

6.查看状态：

[root@QA-linux-1 zookeeper]# ./bin/zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

Mode: follower

[root@QA-linux-2 zookeeper]# ./bin/zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

Mode: leader

[root@QA-linux-3 zookeeper]# ./bin/zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

Mode: follower

至此zookeeper集群安装成功！！！

Kafka集群安装配置

1.安装：

[root@QA-linux-1 src]# tar -zxvf kafka_2.12-1.1.0.tgz -C /usr/local/

[root@QA-linux-1 src]# cd ..

[root@QA-linux-1 local]# ln -s kafka_2.12-1.1.0 kafka

2.修改server.properties文件

# master为0

broker.id=0

listeners=PLAINTEXT://QA-linux-1:9092

advertised.listeners=PLAINTEXT://QA-linux-1:9092

num.network.threads=3

num.io.threads=8

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=/tmp/kafka-logs

num.partitions=5

num.recovery.threads.per.data.dir=1

offsets.topic.replication.factor=1

transaction.state.log.replication.factor=1

transaction.state.log.min.isr=1

log.retention.hours=24

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

# 连接

zookeeper.connect=QA-linux-1:2181,QA-linux-2:2181,QA-linux-3:2181

zookeeper.connection.timeout.ms=6000

group.initial.rebalance.delay.ms=0

# 可删除topic

delete.topic.enable=true

3.将 kafka_2.12-1.1.0 文件夹复制到另外两个节点下

[root@QA-linux-1 local]# scp -r kafka_2.12-1.1.0/ 10.200.3.86:/usr/local/

[root@QA-linux-1 local]# scp -r kafka_2.12-1.1.0/ 10.200.3.87:/usr/local/

并修改每个节点对应的 server.properties 文件的 broker.id和listenrs：

[root@QA-linux-2 config]# cat server.properties

broker.id=1

listeners=PLAINTEXT://QA-linux-2:9092

advertised.listeners=PLAINTEXT://QA-linux-2:9092

 

[root@QA-linux-3 config]# cat server.properties

broker.id=2

listeners=PLAINTEXT://QA-linux-3:9092

advertised.listeners=PLAINTEXT://QA-linux-3:9092

4.启动服务

bin/kafka-server-start.sh config/server.properties &

Zookeeper+Kafka集群测试

创建topic：

[root@QA-linux-1 kafka]# bin/kafka-topics.sh --create --zookeeper QA-linux-1:2181, QA-linux-2:2181, QA-linux-3:2181 --replication-factor 3 --partitions 3 --topic test

显示topic：

[root@QA-linux-1 kafka]# bin/kafka-topics.sh --describe --zookeeper QA-linux-1:2181, QA-linux-2:2181, QA-linux-3:2181 --topic test

列出topic：

[root@QA-linux-1 kafka]# bin/kafka-topics.sh --list --zookeeper QA-linux-1:2181, QA-linux-2:2181, QA-linux-3:2181

test

删除 topic 和关闭服务

[root@QA-linux-1 kafka]# bin/kafka-topics.sh --delete --zookeeper QA-linux-1:2181, QA-linux-2:2181, QA-linux-3:2181 --topic test

启动服务:

bin/kafka-server-start.sh config/server.properties &

停止服务：

bin/kafka-server-stop.sh

 

转自：

• 参考：https://www.cnblogs.com/saneri/p/8762168.html
• 参考：https://www.cnblogs.com/frankdeng/p/9403883.html