flink standalone模式不能上传jar包

现象

Caused by: org.apache.flink.runtime.rest.util.RestClientException: [org.apache.flink.runtime.rest.handler.RestHandlerException: Could not upload job files.
        at org.apache.flink.runtime.rest.handler.job.JobSubmitHandler.lambda$uploadJobGraphFiles$4(JobSubmitHandler.java:201)
        at java.base/java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1311)
        at java.base/java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1280)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.apache.flink.util.FlinkException: Could not upload job files.
        at org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:86)
        at org.apache.flink.runtime.rest.handler.job.JobSubmitHandler.lambda$uploadJobGraphFiles$4(JobSubmitHandler.java:195)
        ... 10 more
Caused by: java.io.IOException: Could not connect to BlobServer at address localhost/127.0.0.1:39991

解决办法

本方案针对单节点（192.168.100.106）部署，确保可通过Web UI上传并执行JAR任务。

1. 准备工作

下载Flink

wget https://archive.apache.org/dist/flink/flink-1.17.0/flink-1.17.0-bin-scala_2.12.tgz
tar xzf flink-1.17.0-bin-scala_2.12.tgz
cd flink-1.17.0

2. 关键配置文件

conf/flink-conf.yaml 配置

# 基础配置
jobmanager.rpc.address: 192.168.100.106
jobmanager.rpc.port: 6123
jobmanager.bind-host: 0.0.0.0
jobmanager.memory.process.size: 1600m

taskmanager.bind-host: 0.0.0.0
taskmanager.host: 192.168.100.106
taskmanager.memory.process.size: 4096m
taskmanager.numberOfTaskSlots: 4  # 根据CPU核心数调整

# Web UI和作业提交配置（关键配置）
web.submit.enable: true
web.upload.dir: /tmp/flink-web-upload  # 上传JAR的临时目录
web.tmpdir: /tmp/flink-web-tmp         # 临时目录
rest.address: 192.168.100.106
rest.bind-address: 0.0.0.0
rest.port: 8081

# 检查点配置（可选）
state.backend: filesystem
state.checkpoints.dir: file:///opt/flink/checkpoints
state.savepoints.dir: file:///opt/flink/savepoints

# 安全配置（如需）
security.ssl.enabled: false  # 测试环境可关闭

创建必要目录

mkdir -p /tmp/flink-web-upload
mkdir -p /tmp/flink-web-tmp
chmod 777 /tmp/flink-web-upload
chmod 777 /tmp/flink-web-tmp

3. 单节点部署

修改 conf/workers 文件

192.168.100.106

修改 conf/masters 文件

192.168.100.106:8081

4. 启动集群

# 启动集群
./bin/start-cluster.sh

# 验证状态
./bin/flink list

5. 防火墙配置

sudo ufw allow 8081/tcp  # Web UI端口
sudo ufw allow 6123/tcp  # JobManager RPC端口

6. 访问Web UI

通过浏览器访问：

http://192.168.100.106:8081

7. 通过Web UI提交作业步骤

1. 访问Web UI（http://192.168.100.106:8081）
2. 点击左侧菜单"Submit New Job"
3. 点击"Add New"按钮上传JAR文件
4. 选择上传的JAR文件
5. 填写:
   • Entry Class（主类，如com.example.KafkaWordCount）
   • Program Arguments（可选参数）
6. 点击"Submit"按钮提交作业

8. 常见问题解决

问题1：上传JAR失败

• 检查web.upload.dir目录权限
• 检查磁盘空间

问题2：提交后作业不执行

• 检查JobManager和TaskManager日志
• 确保jobmanager.rpc.address配置正确

问题3：Web UI无法访问

• 检查防火墙设置
• 确认Flink进程正常运行

9. 日志查看

# JobManager日志
tail -f log/flink-*-jobmanager-*.log

# TaskManager日志
tail -f log/flink-*-taskmanager-*.log

10. 停止集群

./bin/stop-cluster.sh

此配置已确保Web上传功能可用，如需更高安全性，可配置HTTPS和用户认证。