Kafka环境搭建

0. Kafka源码包下载

http://mirror.bit.edu.cn/apache/kafka/

1. 集群环境

Master 172.16.11.97

Slave1 172.16.11.98

Slave2 172.16.11.99

2. 下载软件包

#Master

wget http://mirror.bit.edu.cn/apache/kafka/0.10.2.1/kafka_2.11-0.10.2.1.tgz

tar zxvf kafka_2.11-0.10.2.1.tgz

3. 修改Kafka配置文件

#Master

cd kafka_2.11-0.10.2.1/conf

vim server.propertie

log.dirs=/tmp/kafka-logs

zookeeper.connect=master:2181,slave1:2181,slave2:2181

修改kafka的log路径kafka-logs名字不变，把路径改了

输入以下内容：

4. 增加环境变量

#Master、Slave1、Slave2

vim ~/.bashrc

export KAFKA_HOME=/usr/local/src/kafka_2.11-0.10.2.1

export PATH=$KAFKA_HOME/bin:$PATH

#刷新环境变量

source ~/.bashrc

5. 拷贝软件包

scp -r /usr/local/src/kafka_2.11-0.10.2.1 root@slave1:/usr/local/src/kafka_2.11-0.10.2.1

scp -r /usr/local/src/kafka_2.11-0.10.2.1 root@slave2:/usr/local/src/kafka_2.11-0.10.2.1

6. 修改Kafka配置文件

#Master

vim config/server.propertie

broker.id= 0

host.name=kafka0

#Slave1

vim config/server.propertie

broker.id= 1

host.name=kafka1

#Slave2

vim config/server.propertie

broker.id= 2

host.name=kafka2

7. 启动Kafka-Zookeeper集群

#如果启动了Zookeeper集群则跳过此步骤

vim /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka-zookeeper.sh

/usr/local/src/kafka_2.11-0.10.2.1/bin/zookeeper-server-start.sh /usr/local/src/kafka_2.11-0.10.2.1/config/server.properties

chmod +x /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka-zookeeper.sh

start-kafka-zookeeper.sh

8. 启动Kafka集群

#Master、Slave1、Slave2

vim /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka.sh

• 启动server：

/usr/local/src/kafka_2.11-0.10.2.1/bin/kafka-server-start.sh /usr/local/src/kafka_2.11-0.10.2.1/config/server.properties

chmod +x /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka.sh

start-kafka-zookeeper.sh

9. 进程状态

jps

#Master

#Slave1

#Slave2

10. 关闭集群

kafka-server-stop.sh

单节点多broker的配置

cp server.propertie server-1.propertie
cp server.propertie server-2.propertie

启动kafka

/usr/local/src/kafka_2.11-0.10.2.1/bin/kafka-server-start.sh $KAFKA_HOME/config/server-1.properties

/usr/local/src/kafka_2.11-0.10.2.1/bin/kafka-server-start.sh $KAFKA_HOME/config/server-2.properties

/usr/local/src/kafka_2.11-0.10.2.1/bin/kafka-server-start.sh $KAFKA_HOME/config/server-3.properties

查看进程