kafka3.0集群搭建过程

1.环境说明

服务器资源规划如下表:

IP	hostname	部署资源
192.168.162.201	m162p201	zookeeper,kafka
192.168.162.202	m162p202	zookeeper,kafka
192.168.162.203	m162p203	zookeeper,kafka

操作系统版本(三个节点均相同)：

[root@m162p201 software]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.7.1908 (Core)
Release:        7.7.1908
Codename:       Core

2.环境准备

2.1 依赖软件下载

计划安装的软件：

软件	版本	安装节点	下载地址
jdk	jdk-8u311-linux-x64.rpm	m162p201,m162p202,m162p203	https://www.oracle.com/java/technologies/downloads/#java8
zookeeper	apache-zookeeper-3.5.6-bin.tar.gz	m162p201,m162p202,m162p203	https://archive.apache.org/dist/zookeeper/zookeeper-3.5.6/
kafka	kafka_2.12-3.0.0.tgz	m162p201,m162p202,m162p203	https://kafka.apache.org/downloads#3.0.0
eagle	kafka-eagle-bin-1.4.8.tar.gz	m162p201	http://download.kafka-eagle.org/

将上述软件上传到对应服务器的/opt/software目录。

2.2 jdk安装

由于下载的是rpm包。因此直接执行rpm安装命令即可：

[root@m161p114 software]# rpm -ivh jdk-8u311-linux-x64.rpm 
warning: jdk-8u311-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:jdk1.8-2000:1.8.0_311-fcs        ################################# [100%]
Unpacking JAR files...
	tools.jar...
	plugin.jar...
	javaws.jar...
	deploy.jar...
	rt.jar...
	jsse.jar...
	charsets.jar...
	localedata.jar...
[root@m161p114 software]# 

2.3 zookeeper安装

在每台服务器都新建一个zookeeper用户：

useradd -d  /opt/zookeeper  zookeeper 

在 /etc/security/limit.conf中增加如下内容：

zookeeper  soft nproc 16384
zookeeper  hard nproc 16384
zookeeper  soft nofile 65535
zookeeper  hard nofile 65536

切换到zookeeper用户 su - zookeeper
建立软链接,便于以后切换版本：

ln -s /opt/zookeeper/zookeeper-3.4.10 /opt/zookeeper/zookeeper

由于采用rpm安装了jdk,因此并不需要重新配置jdk的环境变量。
为了便于后续操作，将zookeeper的环境变量配置上。修改.bash_profile文件。

export ZOOKEEPER_HOME=/opt/zookeeper/apache-zookeeper

export PATH=$ZOOKEEPER_HOME/bin:$PATH

之后重新加载环境变量：

soure .bash_profile

zookeeper配置文件：

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.