You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2019/11/07 08:19:23 UTC
[GitHub] [hadoop-docker-ozone] elek opened a new pull request #9: HDDS-2308.
Switch to centos with the apache/ozone-build docker image
elek opened a new pull request #9: HDDS-2308. Switch to centos with the apache/ozone-build docker image
URL: https://github.com/apache/hadoop-docker-ozone/pull/9
## What changes were proposed in this pull request?
I realized multiple JVM crashes in the daily builds:
```
ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs && /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter9018689154779946208.jar /workdir/hadoop-ozone/ozonefs/target/surefire 2019-10-06T14-52-40_697-jvmRun1 surefire7569723928289175829tmp surefire_947955725320624341206tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 139
[ERROR] Crashed tests:
[ERROR] org.apache.hadoop.fs.ozone.contract.ITestOzoneContractRename
[ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs && /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter5429192218879128313.jar /workdir/hadoop-ozone/ozonefs/target/surefire 2019-10-06T14-52-40_697-jvmRun1 surefire7227403571189445391tmp surefire_1011197392458143645283tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 139
[ERROR] Crashed tests:
[ERROR] org.apache.hadoop.fs.ozone.contract.ITestOzoneContractDistCp
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs && /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter1355604543311368443.jar /workdir/hadoop-ozone/ozonefs/target/surefire 2019-10-06T14-52-40_697-jvmRun1 surefire3938612864214747736tmp surefire_933162535733309260236tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 139
[ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs && /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter9018689154779946208.jar /workdir/hadoop-ozone/ozonefs/target/surefire 2019-10-06T14-52-40_697-jvmRun1 surefire7569723928289175829tmp surefire_947955725320624341206tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 139
```
Based on the crash log (uploaded) it's related to the rocksdb JNI interface.
In the current ozone-build docker image (which provides the environment for build) we use alpine where musl libc is used instead of the main glibc. I think it would be more safe to use the same glibc what is used in production.
I tested with centos based docker image and it seems to be more stable. Didn't see any more JVM crashes.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2308
## How this patch can be tested?
Built and pushed to my own dockerhub account:
```
docker build -t elek/ozone-build:20191106-1 .
docker puseh elek/ozone-build:20191106-1
```
And used in the PR builds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org