Flink集群(standalone)安装

节点规划

进程	IP
JobManager	10.192.192.8
TaskManager	10.192.192.9
TaskManager	10.192.192.10

安装配置

# 三台机器需提前安装jdk1.8
yum install -y java-1.8.0-openjdk-devel.x86_64

# 10.192.192.8执行
# 下载安装包
mkdir /opt/flink && cd /opt/flink

wget https://archive.apache.org/dist/flink/flink-1.14.4/flink-1.14.4-bin-scala_2.11.tgz

tar -zxvf flink-1.14.4-bin-scala_2.11.tgz
# 配置环境变量
vim /etc/profile
export FLINK_HOME=/opt/flink/flink-1.14.4
export PATH=$PATH:$FLINK_HOME/bin

# 重新加载
source /etc/profile
# 验证版本
flink --version

# 修改flink-conf.yaml配置
vim /opt/flink/flink-1.14.4/conf/flink-conf.yaml
taskmanager.numberOfTaskSlots: 5
jobmanager.rpc.address: 10.192.192.8

# 修改masters配置
vim /opt/flink/flink-1.14.4/conf/masters
10.192.192.8:8081

# 修改workers配置
vim /opt/flink/flink-1.14.4/conf/workers
10.192.192.9
10.192.192.10

# 复制配置文件
cd /opt
scp -r flink 10.192.192.9:/opt
scp -r flink 10.192.192.10:/opt

部署测试

# 10.192.192.8执行
# 启动集群（根据提示输入服务器登陆密码）
start-cluster.sh
# 验证进程
jps
[root@iZ2ze3n2wuzr90bp6211riZ conf]# jps
6308 StandaloneSessionClusterEntrypoint
6684 Jps

# 10.192.192.9执行
jps
[root@iZ2ze3n2wuzr90bp6211rkZ ~]# jps
3537 TaskManagerRunner
3823 Jps

# 10.192.192.10执行
[root@iZ2ze3n2wuzr90bp6211rjZ ~]# jps
3797 Jps
3735 TaskManagerRunner

加入系统服务

cat >/usr/lib/systemd/system/flink.service <<EOF
[Unit]
Description=Apache Flink：Stateful Computations over Data Streams
Documentation=https://flink.apache.org
After=network-online.target firewalld.service
Wants=network-online.target

[Service]
ExecStart=/opt/flink/flink-1.14.4/bin/start-cluster.sh
ExecStop=/opt/flink/flink-1.14.4/bin/stop-cluster.sh
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target
EOF

systemctl enable flink.service && \
systemctl daemon-reload && \
systemctl start flink.service  && \
systemctl status flink.service 

备注

如果启动集群时，ssh端口不是默认的22端口，需要将其修改为指定端口：

vim /opt/flink/flink-1.14.4/conf/flink-conf.yaml
# 编辑如下，修改对应配置
env.ssh.opts: -p 指定端口

服务器之间免密登录（如果不知道登录密码）：

# 生成公钥
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa  && cat ~/.ssh/id_dsa.pub
# copy公钥至指定服务器
echo '公钥' >> ~/.ssh/authorized_keys