JAVA Memory and Docker

JAVA Memory and Docker

Recently, I am working on a Java applications on ECS. It is a docker application running on the cluster.
But it keeps get killed by ECS agent. And the exception is saying, docker application is killed by memory usage.

Does that say my Java application have memory leak and go Out of Memory?

First, I checked my memory configuration on Java level. I am using -Xmx3G. My docker hard_limit and soft_limit are both 3G as well.

I am using spring framework, so I have the Perm Gen 256M.
So the VM may use memory like this: -Xmx3G + Perm Gen 256M + Other memory 217M + Xss per thread(Java Stack) = 3.5G

That might be why the ECS kill my docker applications.

Here are all the JVM exception list here
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html

>java \
-Xms2G -Xmx2G -XX:MaxMetaspaceSize=256M \
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/tmp/dump_oom.hprof" \
-Dbuild.env=${RUNNING_ENV} \
-Drunning_module=${RUNNING_MODULE} \
-jar feeds-2g-1.0-jar-with-dependencies.jar 2>&1

I have my Java application running there for a while day, it is not saying any exception about memory. So that is 90% saying it is not Java OOM, it is use memory over the limit of hard_limit on ECS, and ECS agent killed my Java.

I adjust the configuration according to the ECS document.
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html

-Xms2G -Xmx2G and hard_limit and soft_limit 3G. Everything looks good after that.

Reference:
https://blogs.vmware.com/apps/2011/06/taking-a-closer-look-at-sizing-the-java-process.html
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html