flink1.10 在yarn上运行job报内存超出

最新推荐文章于 2024-10-28 21:38:09 发布

原创

最新推荐文章于 2024-10-28 21:38:09 发布 · 5.7k 阅读

5 ·

CC 4.0 BY-SA版权

本文详细解析了Apache Flink在YARN集群模式下遇到的内存配置问题，包括如何正确设置taskmanager和jobmanager的内存，避免因内存配置不当导致的任务失败。

./bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 -s hdfs://master:9000/flink/checkpoints/d15750eebe118cccb93b4450a008e4d3/chk-158/_metadata -c stream.TestKafkaCheckpoint /var/flink/data/jars/flink-1.0-SNAPSHOT.jar

可以看到，我给jobmanager和taskmanager分别分配了1g内存。

但是运行时报了下面的错。

意思是说那几项内存总和（372m）超过了配置的total flink memory（132m）。

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Sum of configured Framework Heap Memory (128.000mb (134217728 bytes)), Framework Off-Heap Memory (128.000mb (134217728 bytes)), Task Off-Heap Memory (0 bytes), Managed Memory (52.800mb (55364813 bytes)) and Network Memory (64.000mb (67108864 bytes)) exceed configured Total Flink Memory (132.000mb (138412032 bytes)).
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Sum of configured Framework Heap Memory (128.000mb (134217728 bytes)), Framework Off-Heap Memory (128.000mb (134217728 bytes)), Task Off-Heap Memory (0 bytes), Managed Memory (52.800mb (55364813 bytes)) and Network Memory (64.000mb (67108864 bytes)) exceed configured Total Flink Memory (132.000mb (138412032 bytes)).

当我调低jobmanager和taskmanager的内存为100m时，又报下面的错。

意思是配置的JVM Metaspace为700m，JVM Overhead为192m，超过了total process memory配置的200m（ytm）。