You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/05/10 02:56:49 UTC
[GitHub] [ozone] kuenishi opened a new pull request #2225: HDDS-5197. Pass option varialbes to OZONE_OPTS
kuenishi opened a new pull request #2225:
URL: https://github.com/apache/ozone/pull/2225
## What changes were proposed in this pull request?
Fix an issue where `OZONE_*_OPTS` are not actually passed to daemon invocations.
## What is the link to the Apache JIRA
[HDDS-5197](https://issues.apache.org/jira/browse/HDDS-5197)
## How was this patch tested?
Partially tested in one of datanodes in our production environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kuenishi commented on pull request #2225: HDDS-5197. Pass option varialbes to OZONE_OPTS
Posted by GitBox <gi...@apache.org>.
kuenishi commented on pull request #2225:
URL: https://github.com/apache/ozone/pull/2225#issuecomment-836298074
@adoroszlai Thank you for the review. I've updated my pull request accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia merged pull request #2225: HDDS-5197. Pass option variables to OZONE_OPTS before adding default GC opts
Posted by GitBox <gi...@apache.org>.
dineshchitlangia merged pull request #2225:
URL: https://github.com/apache/ozone/pull/2225
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kuenishi commented on pull request #2225: HDDS-5197. Pass option variables to OZONE_OPTS before adding default GC opts
Posted by GitBox <gi...@apache.org>.
kuenishi commented on pull request #2225:
URL: https://github.com/apache/ozone/pull/2225#issuecomment-836366279
I updated the PR title. But I don't think I have to update the JIRA title, because it just describes the issue itself, not the fix. @adoroszlai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kuenishi commented on pull request #2225: HDDS-5197. Pass option varialbes to OZONE_OPTS
Posted by GitBox <gi...@apache.org>.
kuenishi commented on pull request #2225:
URL: https://github.com/apache/ozone/pull/2225#issuecomment-836260625
I wasn't aware of the debug option. Give me minutes and let met try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kuenishi commented on pull request #2225: HDDS-5197. Pass option varialbes to OZONE_OPTS
Posted by GitBox <gi...@apache.org>.
kuenishi commented on pull request #2225:
URL: https://github.com/apache/ozone/pull/2225#issuecomment-836291602
I've set
```sh
export OZONE_GC_SETTINGS="-XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=20 -verbose:gc -XX:+PrintGCDetails -XX:MaxGCPauseMillis=100 -XX:ParallelGCThreads=16"
```
and
```sh
export OZONE_OM_OPTS="${OZONE_GC_SETTINGS}"
```
in `ozone-env.sh` , and with `bin/ozone om --verbose` , I got
```
No '-XX:...' jvm parameters are set. Adding safer GC settings '-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled' to the OZONE_OPTS
DEBUG: Appending OZONE_OM_OPTS onto OZONE_OPTS
DEBUG: No secure classname defined.
DEBUG: Profiles: ozone_manager finalize
DEBUG: Prepend CLASSPATH: /opt/ozone/ozone-1.1.0/etc/hadoop
DEBUG: OZONE_OPTS accepted -Dhadoop.log.dir=/opt/ozone/ozone-1.1.0/logs
DEBUG: OZONE_OPTS accepted -Dhadoop.log.file=ozone.log
DEBUG: OZONE_OPTS accepted -Dhadoop.home.dir=/opt/ozone/ozone-1.1.0
DEBUG: OZONE_OPTS accepted -Dhadoop.id.str=ozone
DEBUG: OZONE_OPTS accepted -Dhadoop.root.logger=INFO,console
DEBUG: OZONE_OPTS accepted -Dhadoop.policy.file=hadoop-policy.xml
DEBUG: OZONE_OPTS accepted -Dhadoop.security.logger=INFO,NullAppender
DEBUG: Final CLASSPATH: /opt/ozone/ozone-1.1.0/etc/hadoop:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-simplekdc-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jackson-annotations-2.12.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/okhttp-4.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/protobuf-java-2.5.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/slf4j-log4j12-1.7.30.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/log4j-api-2.13.3.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-hadoop-dependency-server-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-validator-1.6.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-security-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/bcpkix-jdk15on-1.67.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-server-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-core-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/picocli-4.4.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jsr305-3.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/li
b/ratis-server-api-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/j2objc-annotations-1.3.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/leveldbjni-all-1.8.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jaeger-tracerresolver-1.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/htrace-core4-4.1.0-incubating.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-util-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kotlin-stdlib-common-1.4.31.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/javax.activation-api-1.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jaeger-thrift-1.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-logging-1.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-auth-3.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerby-xdr-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jackson-datatype-jsr310-2.12.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/httpcore-4.4.13.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-interface-admin-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/o
pentracing-tracerresolver-0.1.8.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-net-3.6.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-common-3.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-netty-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/snakeyaml-1.26.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jaeger-core-1.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/error_prone_annotations-2.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/httpclient-4.5.13.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-hadoop-dependency-client-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-client-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerby-config-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/okio-2.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-ozone-interface-storage-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-ozone-interface-client-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/metrics-core-3.2.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/g
uava-28.2-jre.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerby-pkix-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-webapp-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/xz-1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jsr311-api-1.1.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-beanutils-1.9.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/javax.servlet-api-3.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-server-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-server-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-util-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jackson-core-2.12.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-client-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/disruptor-3.4.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/libthrift-0.13.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/log4j-core-2.13.3.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/annotations-13.0.jar:/opt/ozone/ozone-1.1.0/share/ozon
e/lib/simpleclient_common-0.7.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jackson-databind-2.12.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-thirdparty-misc-0.6.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/dnsjava-2.1.7.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/bcprov-jdk15on-1.67.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-configuration2-2.1.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/json-smart-2.3.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/token-provider-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/log4j-1.2.17.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-daemon-1.0.13.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-client-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-lang3-3.7.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-compress-1.4.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-util-ajax-9.4.35.v20201120.jar:/opt/ozone/ozon
e-1.1.0/share/ozone/lib/accessors-smart-1.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-io-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-xml-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/re2j-1.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-math3-3.1.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/simpleclient_dropwizard-0.7.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/simpleclient-0.7.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-metrics-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/rocksdbjni-6.8.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/asm-5.0.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerby-util-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/failureaccess-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-ozone-common-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jsp-api-2.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jsch-0.1.54.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-grpc-2.0.0.jar:/opt/ozone/ozone-1.1.0
/share/ozone/lib/hadoop-hdds-config-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/slf4j-api-1.7.30.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-interface-client-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/nimbus-jose-jwt-7.9.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-codec-1.11.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-pool2-2.6.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/gson-2.2.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdfs-3.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/opentracing-api-0.33.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-common-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jcip-annotations-1.0-1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-admin-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-server-framework-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-io-2.5.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-interface-server-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kot
lin-stdlib-1.4.31.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/opentracing-noop-0.33.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jersey-core-1.19.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/checker-qual-2.10.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/ratis-proto-2.0.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/opentracing-util-0.33.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-ozone-client-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-annotations-3.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-cli-1.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdds-common-1.1.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/woodstox-core-5.0.3.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/javax.annotation-api-1.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-servlet-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerby-asn1-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/stax2-api-3.1.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-collections-3.2.2.jar:
/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-digester-1.8.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/commons-text-1.4.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jersey-servlet-1.19.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-identity-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-hdfs-client-3.2.2.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-common-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jaeger-client-1.2.0.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jersey-server-1.19.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/jetty-http-9.4.35.v20201120.jar:/opt/ozone/ozone-1.1.0/share/ozone/lib/kerb-crypto-1.0.1.jar:/opt/ozone/ozone-1.1.0/share/ozone/web:/opt/ozone/ozone-1.1.0/share/ozone/lib/hadoop-ozone-ozone-manager-1.1.0.jar
DEBUG: Final OZONE_OPTS: -Djava.net.preferIPv4Stack=true -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -Dlog4j2.contextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dorg.apache.ratis.thirdparty.io.netty.leakDetection.level=disabled -Dorg.apache.ratis.thirdparty.io.netty.allocator.useCacheForAllThreads=false -XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=20 -verbose:gc -XX:+PrintGCDetails -XX:MaxGCPauseMillis=100 -XX:ParallelGCThreads=16 -Dlog4j.configurationFile=/opt/ozone/ozone-1.1.0/etc/hadoop/om-audit-log4j2.properties -Dhadoop.log.dir=/opt/ozone/ozone-1.1.0/logs -Dhadoop.log.file=ozone.log -Dhadoop.home.dir=/opt/ozone/ozone-1.1.0 -Dhadoop.id.str=ozone -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender
DEBUG: Final JAVA_HOME: /usr/java/jdk1.8.0_162
DEBUG: java: /usr/java/jdk1.8.0_162/bin/java
DEBUG: Class name: org.apache.hadoop.ozone.om.OzoneManagerStarter
DEBUG: Command line options: --verbose
Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
Full console log is [here](https://gist.github.com/kuenishi/073b83f5c8cf192128115dd935c934af#file-ozone-om-verbose-log) and in that gist, I attached full `ozone-env.sh` too.
It seems that before `ozone_subcommand_opts` being processed, due to the lack of GC options in `OZONE_OPTS`, default GC options are added and conflicting GC options prevented JVM from started.
The right fix would be calling `ozone_subcommand_opts` before `ozone_add_default_gc_opts` (currently they are in reverse order).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org