Kafka安装与配置

目录

准备工作

安装JDK

安装Zookeeper

安装Kafka

---

准备工作

准备3台机器（虚拟机）

192.168.10.141 kafka1

192.168.10.142 kafka2

192.168.10.143 kafka3

并添加到hosts配置

安装JDK

[root@localhost ~]# java -version
openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-b10)
OpenJDK 64-Bit Server VM (build 25.262-b10, mixed mode)

安装Zookeeper

        Apache ZooKeeper

        Zookeeper是安装Kafka集群的必要组件，Kafka通过Zookeeper来实施对元数据信息的管理，包括集群、broker、主题、分区等内容。

        Zookeeper是一个开源的分布式协调服务。三个角色：leader、follower、observer，同一时刻Zookeeper集群中只有一个leader，其他的都是follower和observer，observer不参与投票，默认情况下只有leader和follower两个角色。

注意停掉防火墙：

[root@localhost etc]# cat redhat-release 
CentOS Linux release 7.9.2009 (Core)

[root@localhost ~]# service firewalld.service stop
Redirecting to /bin/systemctl stop firewalld.service

关键配置：

dataDir=/opt/soft/zookeeper-3.7.1/data
dataLogDir=/opt/soft/zookeeper-3.7.1/log

server.1=192.168.10.141:2888:3888
server.2=192.168.10.142:2888:3888
server.3=192.168.10.143:2888:3888

server.A=B:C:D

A:  代表服务器的编号，跟myid文件里的值一致

B：服务器的IP地址

C：服务器与集群中的leader服务器交换信息的端口

D：选举时服务器相互通信的端口

在dataDir目录下编辑myid文件，填写相应的id

[root@localhost zookeeper-3.7.1]# zkServer.sh start 
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/soft/zookeeper-3.7.1/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@localhost zookeeper-3.7.1]# zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/soft/zookeeper-3.7.1/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

安装Kafka

Apache Kafka

下载后，修改config/server.properties，关键配置：

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=1

# The address the socket server listens on. It will get the value returned from 
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://192.168.10.141:9092

# A comma separated list of directories under which to store log files
log.dirs=/opt/soft/kafka_2.12-3.3.1/logs

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=192.168.10.141:2181/kafka,192.168.10.142:2181/kafka,192.168.10.143:2181/kafka

启动Kafka（前台方式），加入-daemon参数或者&后台运行

bin/kafka-server-start.sh config/server.properties