[GitHub] [pulsar] lhotari commented on issue #7859: Use Java 11 for Docker Images

[GitBox <gi...@apache.org>]

lhotari commented on issue #7859:
URL: https://github.com/apache/pulsar/issues/7859#issuecomment-686272487


   > Java 8 and Docker doesn't work that well together. Some things got backported to Java 8 but only Java 10+ supports Memory and CPU limits inside Docker Containers correctly.
   
   This has changed in the last 2 years. The Java 8 version used in Pulsar images already supports container memory & cpu limits. [This blog post explains it](https://blog.softwaremill.com/docker-support-in-new-java-8-finally-fd595df0ca54).
   
   One way to verify that it's supported is to run this docker command:
   ```
   docker run --memory 100m --cpus 1 --rm --entrypoint '/bin/bash' apachepulsar/pulsar:2.6.1 -c 'java -XX:+UnlockDiagnosticVMOptions -XX:+PrintContainerInfo -version'
   ```
   it will printout diagnostics about the limits that the JVM detects:
   ```
   OSContainer::init: Initializing Container Support
   Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
   Memory Limit is: 104857600
   Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
   CPU Quota is: 100000
   Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
   CPU Period is: 100000
   Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
   CPU Shares is: 1024
   CPU Quota count based on quota/period: 1
   OSContainer::active_processor_count: 1
   ```
   
   It's already possible to use the JVM container support in Pulsar containers by specifying `PULSAR_MEM="-XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=50.0 -XX:MinRAMPercentage=50.0"`. 
   
   For example:
   ```
   docker run --name pulsar-standalone --memory 512m --cpus 1 -e PULSAR_MEM="-XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=50.0 -XX:MinRAMPercentage=50.0" -e PULSAR_GC="-XX:+UseG1GC" apachepulsar/pulsar:2.6.1 bin/pulsar standalone
   ```
   
   Verifying in another terminal after startup:
   ```
   $ docker exec pulsar-standalone jcmd 1 GC.heap_info 
   1:
    garbage-first heap   total 262144K, used 146496K [0x00000000f0000000, 0x00000000f0100800, 0x0000000100000000)
     region size 1024K, 75 young (76800K), 7 survivors (7168K)
    Metaspace       used 78433K, capacity 82292K, committed 82460K, reserved 1122304K
     class space    used 9461K, capacity 10287K, committed 10316K, reserved 1048576K
   ```
   Heap size is 256MB, which is expected. Since Pulsar uses also native memory, it's probably not worth increasing the heap limit too much over 50% so that the container doesn't get killed. The [default MaxDirectMemorySize is the maximum heap size](https://stackoverflow.com/a/54447744). The [total memory consumption of a Java process is explained here](https://stackoverflow.com/a/53624438).
   It's also worth noticing that PULSAR_GC should be set for docker containers so that the not-so-good defaults don't get used. [There's a PR already for addressing this issue](https://github.com/apache/pulsar/pull/3650). One issue with the defaults is that there's a huge number of GC threads configured.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org