[GitHub] [bigtop] iwasakims commented on pull request #844: BIGTOP-3613 - Upgrade hive 2.3.6 to log4j 2.16.0

[GitBox <gi...@apache.org>]

iwasakims commented on pull request #844:
URL: https://github.com/apache/bigtop/pull/844#issuecomment-996660069


   Thanks for working on this @elukey.
   
   I got following error on starting up HiveServer2 when I ran docker provisioner:
   <pre>
   Exception in thread "main" java.lang.NoSuchMethodError: com.lmax.disruptor.dsl.Disruptor.<init>(Lcom/lmax/disruptor/EventFactory;ILjava/util/concurrent/ThreadFactory;Lcom/lmax/disruptor/dsl/ProducerType;Lcom/lmax/disruptor/WaitStrategy;)V
           at org.apache.logging.log4j.core.async.AsyncLoggerDisruptor.start(AsyncLoggerDisruptor.java:108)
           at org.apache.logging.log4j.core.async.AsyncLoggerContext.maybeStartHelper(AsyncLoggerContext.java:97)
           at org.apache.logging.log4j.core.async.AsyncLoggerContext.start(AsyncLoggerContext.java:86)
           at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:242)
           at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:160)
           at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:132)
           at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:102)
           at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:211)
           at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jDefault(LogUtils.java:154)
           at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:90)
           at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:82)
           at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:65)
           at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:716)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.hadoop.util.RunJar.run(RunJar.java:244)
           at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
   </pre>
   
   disruptor-3.3.0 instead of 3.3.7 is containd in the package.
   ```
   $ rpm -q -l -p output/hive/noarch/hive-2.3.6-3.el8.noarch.rpm | grep disruptor
   /usr/lib/hive/lib/disruptor-3.3.0.jar
   ```
   
   Dependency tree shows that Hive itself depends on disruptor only in test scope. Since hbase-server and druid-common (and more?) depends on disruptor in compile scope, fixing `disruptor.version` in pom.xml of Hive seems not to affect the version of disrupoter pulled as transitive dependency.
   ```
   $ mvn -Dhbase.version=1.5.0 -Dzookeeper.version=3.4.13 -Dhadoop.version=2.10.1 -DskipTests -Dtez.version=0.9.2 -Dspark.version=2.4.5 dependency:tree
   ...
   [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ hive-common ---
   [INFO] org.apache.hive:hive-common:jar:2.3.6
   ...
   [INFO] +- com.lmax:disruptor:jar:3.3.7:test
   ...
   [INFO] +- org.apache.hbase:hbase-server:jar:1.5.0:compile
   ...
   [INFO] |  \- com.lmax:disruptor:jar:3.3.0:compile
   ...
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org