Docker and Java issues

Russell Bateman
October 2018
last update:

I've been trying to do some Java vs. Docker limits testing. The JVM is not written for Docker use, though it works just as any other application. The problem is that the JVM goes deep into the host in search of what it should understand as memory and CPUs. This is a short-sightedness that's expected given its history and the relatively recent advent of Docker. See Java and Docker, the limitations.

MemEat.java:
import java.util.ArrayList;
import java.util.List;

public class MemEat
{
  public static void main( String[] args )
  {
    int cpus = Runtime.getRuntime().availableProcessors();

    System.out.println( "Found " + cpus + ( ( cpus == 1 ) ? " CPU." : " CPUs." ) );

    //noinspection MismatchedQueryAndUpdateOfCollection
    List< Object > list = new ArrayList<>();

    //noinspection InfiniteLoopStatement
    while( true )
    {
      byte bytes[] = new byte[ 1048576 ];
      list.add( bytes );

      Runtime runtime = Runtime.getRuntime();
      System.out.println( "free memory: " + runtime.freeMemory() );
    }
  }
}

Create a simple Dockerfile to do the stuff shown by the article noted above, which was written up on 16 May 2018.

FROM      java:openjdk-8u111
WORKDIR   /
COPY      ./MemEat.java /MemEat.java
CMD       /bin/bash

Legend:

FROM The Docker base image, in this case, it appears that there is an image for openjdk-8u111; it's exactly what I want.
WORKDIR Where you want your current working directory to be.
COPY Directive to copy from some file or directory to some place.
CMD Command to run first when the container comes up.

To build an image based on dockerfile:

[email protected]:~/dev/docker-limits$ docker build --tag docker-limits .
Sending build context to Docker daemon   42.5kB
Step 1/4 : FROM    java:openjdk-8u111
 ---> d23bdf5b1b1b
Step 2/4 : WORKDIR /
 ---> Using cache
 ---> 9462a3796ff1
Step 3/4 : COPY    ./src/MemEat.java /MemEat.java
 ---> 136b253704f4
Step 4/4 : CMD     /bin/bash
 ---> Running in 226fddc36f42
Removing intermediate container 226fddc36f42
 ---> 16592d6f505b
Successfully built 16592d6f505b
Successfully tagged docker-limits:latest

To run the image:

[email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
[email protected]:/# javac MemEat.java
[email protected]:/# java -Xmx100m MemEat
free memory: 99090400
free memory: 98041808
free memory: 96993216
.
.
.
free memory: 7603400
free memory: 6554808
free memory: 5506216
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at MemEat.main(MemEat.java:14)

To see all the images:

[email protected]:~/dev/docker-limits$ docker ps -a
CONTAINER ID   IMAGE          COMMAND                 CREATED         STATUS                    PORTS   NAMES
b319b30dfb88   docker-limits  "/bin/sh -c /bin/bash"  6 minutes ago   Exited (1) 26 seconds ago         elegant_hoover

To discard images you no longer want:

[email protected]:~/dev/docker-limits$ docker rm b319b30dfb88
b319b30dfb88

Docker base images that demonstrate brokenness/fixes using the code above:

  1. java:openjdk-8u111, broken.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# javac MemEat.java
    [email protected]:/# java -version
    openjdk version "1.8.0_111"
    OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2~bpo8+1-b14)
    OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
    [email protected]:/# java MemEat
    Found 8 CPUs.
    free memory: 249288432
    free memory: 248239840
    free memory: 247191248
    .
    .
    .
    (runs forever, probably because of kernel not supporting swap limit)
    
  2. java:openjdk-8u111, broken, but fixed using option -Xmx100m.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# javac MemEat.java
    [email protected]:/# java -version
    openjdk version "1.8.0_111"
    OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2~bpo8+1-b14)
    OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
    [email protected]:/# java -Xmx100m MemEat
    Found 8 CPUs.
    free memory: 99090400
    free memory: 98041808
    free memory: 96993216
    .
    .
    .
    free memory: 7603280
    free memory: 6554688
    free memory: 5506096
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)
    
  3. adoptopenjdk/openjdk8, fixed with later build.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# java -version
    Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
    openjdk version "1.8.0_181"
    OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_181-b13)
    OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.181-b13, mixed mode)
    [email protected]:/# javac MemEat.java
    Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
    [email protected]:/# java MemEat
    Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
    Found 8 CPUs.
    free memory: 6430696
    free memory: 5382104
    free memory: 4333512
    .
    .
    .
    free memory: 2204416
    free memory: 1155824
    free memory: 1155808
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)
    
  4. adoptopenjdk/openjdk10 (Java 10u23), still broken.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# javac MemEat.java
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    [email protected]:/# java MemEat
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Found 8 CPUs.
    free memory: 6259312
    free memory: 5215184
    free memory: 4124152
    .
    .
    .
    free memory: 3504160
    free memory: 2455568
    free memory: 1406976
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)
    
  5. adoptopenjdk/openjdk10:nightly (10u46 or later), fixed.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# javac MemEat.java
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    [email protected]:/# java MemEat
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Found 8 CPUs.
    free memory: 6259160
    free memory: 5215208
    free memory: 4124160
    .
    .
    .
    free memory: 3504176
    free memory: 2455584
    free memory: 1406992
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)
    [email protected]:/# export PATH=$PATH:/opt/java/openjdk/jdk-10+46/bin/
    [email protected]:/# javac MemEat.java
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    [email protected]:/# java MemEat
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Found 8 CPUs.
    free memory: 6259176
    free memory: 5215408
    free memory: 4109864
    .
    .
    .
    free memory: 3504408
    free memory: 2455816
    free memory: 1407224
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)
    
  6. adoptopenjdk/openjdk9-openj9, need to try this.
  7. adoptopenjdk/openjdk9-openj9:nightly, need to try this.
  8. adoptopenjdk/openjdk11-openj11, looks correct to me.
    [email protected]:~/dev/docker-limits$ docker run --memory="100m" -it docker-limits
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
    [email protected]:/# javac MemEat.java
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    [email protected]:/# java MemEat
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Found 8 CPUs.
    free memory: 6308328
    free memory: 5264032
    free memory: 4172872
    .
    .
    .
    free memory: 3553264
    free memory: 2504672
    free memory: 1456080
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    	at MemEat.main(MemEat.java:18)