You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Neil Fordyce (JIRA)" <ji...@apache.org> on 2018/06/29 10:47:00 UTC
[jira] [Created] (SAMZA-1767) samza-log4j incompatible with scala
2.12 modules
Neil Fordyce created SAMZA-1767:
-----------------------------------
Summary: samza-log4j incompatible with scala 2.12 modules
Key: SAMZA-1767
URL: https://issues.apache.org/jira/browse/SAMZA-1767
Project: Samza
Issue Type: Bug
Affects Versions: 0.14.1
Reporter: Neil Fordyce
Hit this exception while launching a 0.14.1 job which depends on samza-log4j to use StreamAppender.
{code}
/opt/samza/job/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file:///opt/samza/job/config/job.properties
home_dir=/opt/drone
framework base (location of this script). base_dir=/opt/samza/job
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
java -Dlog4j.configuration=file:/opt/samza/job/bin/log4j-console.xml -Dsamza.log.dir=/opt/samza/job -Djava.io.tmpdir=/opt/samza/job/tmp -Xmx768M -XX:+PrintGCDateStamps -Xloggc:/opt/samza/job/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10241024 -d64 -cp /opt/samza/config:/opt/samza/job/lib/activation-1.1.jar:/opt/samza/job/lib/aopalliance-1.0.jar:/opt/samza/job/lib/apacheds-i18n-2.0.0-M15.jar:/opt/samza/job/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/samza/job/lib/api-asn1-api-1.0.0-M20.jar:/opt/samza/job/lib/api-util-1.0.0-M20.jar:/opt/samza/job/lib/asm-3.1.jar:/opt/samza/job/lib/avro-1.7.4.jar:/opt/samza/job/lib/commons-beanutils-1.7.0.jar:/opt/samza/job/lib/commons-beanutils-core-1.8.0.jar:/opt/samza/job/lib/commons-cli-1.2.jar:/opt/samza/job/lib/commons-codec-1.2.jar:/opt/samza/job/lib/commons-collections-3.2.1.jar:/opt/samza/job/lib/commons-collections4-4.0.jar:/opt/samza/job/lib/commons-compress-1.4.1.jar:/opt/samza/job/lib/commons-configuration-1.6.jar:/opt/samza/job/lib/commons-daemon-1.0.13.jar:/opt/samza/job/lib/commons-digester-1.8.jar:/opt/samza/job/lib/commons-el-1.0.jar:/opt/samza/job/lib/commons-httpclient-3.1.jar:/opt/samza/job/lib/commons-io-2.4.jar:/opt/samza/job/lib/commons-lang-2.6.jar:/opt/samza/job/lib/commons-lang3-3.4.jar:/opt/samza/job/lib/commons-logging-1.1.3.jar:/opt/samza/job/lib/commons-math3-3.1.1.jar:/opt/samza/job/lib/commons-net-3.1.jar:/opt/samza/job/lib/curator-client-2.6.0.jar:/opt/samza/job/lib/curator-framework-2.6.0.jar:/opt/samza/job/lib/curator-recipes-2.6.0.jar:/opt/samza/job/lib/grizzled-slf4j_2.12-1.3.0.jar:/opt/samza/job/lib/gson-2.2.4.jar:/opt/samza/job/lib/guava-17.0.jar:/opt/samza/job/lib/guice-3.0.jar:/opt/samza/job/lib/guice-servlet-3.0.jar:/opt/samza/job/lib/hadoop-annotations-2.6.1.jar:/opt/samza/job/lib/hadoop-auth-2.6.1.jar:/opt/samza/job/lib/hadoop-common-2.6.1.jar:/opt/samza/job/lib/hadoop-hdfs-2.6.1.jar:/opt/samza/job/lib/hadoop-yarn-api-2.6.1.jar:/opt/samza/job/lib/hadoop-yarn-client-2.6.1.jar:/opt/samza/job/lib/hadoop-yarn-common-2.6.1.jar:/opt/samza/job/lib/htrace-core-3.0.4.jar:/opt/samza/job/lib/httpclient-4.1.2.jar:/opt/samza/job/lib/httpcore-4.1.2.jar:/opt/samza/job/lib/jackson-core-asl-1.9.13.jar:/opt/samza/job/lib/jackson-jaxrs-1.9.13.jar:/opt/samza/job/lib/jackson-mapper-asl-1.9.13.jar:/opt/samza/job/lib/jackson-xc-1.9.13.jar:/opt/samza/job/lib/jasper-compiler-5.5.23.jar:/opt/samza/job/lib/jasper-runtime-5.5.23.jar:/opt/samza/job/lib/java-xmlbuilder-0.4.jar:/opt/samza/job/lib/javax.inject-1.jar:/opt/samza/job/lib/javax.servlet-api-3.1.0.jar:/opt/samza/job/lib/jaxb-api-2.2.2.jar:/opt/samza/job/lib/jaxb-impl-2.2.3-1.jar:/opt/samza/job/lib/jersey-client-1.9.jar:/opt/samza/job/lib/jersey-core-1.9.jar:/opt/samza/job/lib/jersey-guice-1.9.jar:/opt/samza/job/lib/jersey-json-1.9.jar:/opt/samza/job/lib/jersey-server-1.9.jar:/opt/samza/job/lib/jets3t-0.9.0.jar:/opt/samza/job/lib/jettison-1.1.jar:/opt/samza/job/lib/jetty-6.1.26.jar:/opt/samza/job/lib/jetty-http-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-io-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-security-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-server-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-servlet-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-util-6.1.26.jar:/opt/samza/job/lib/jetty-util-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-webapp-9.2.7.v20150116.jar:/opt/samza/job/lib/jetty-xml-9.2.7.v20150116.jar:/opt/samza/job/lib/jline-0.9.94.jar:/opt/samza/job/lib/joda-time-2.2.jar:/opt/samza/job/lib/jopt-simple-3.2.jar:/opt/samza/job/lib/jsch-0.1.42.jar:/opt/samza/job/lib/jsp-api-2.1.jar:/opt/samza/job/lib/jsr305-1.3.9.jar:/opt/samza/job/lib/juniversalchardet-1.0.3.jar:/opt/samza/job/lib/kafka-clients-0.11.0.2.jar:/opt/samza/job/lib/kafka_2.12-0.11.0.2.jar:/opt/samza/job/lib/log4j-1.2.17.jar:/opt/samza/job/lib/lz4-1.3.0.jar:/opt/samza/job/lib/metrics-core-2.2.0.jar:/opt/samza/job/lib/mime-util-2.1.3.jar:/opt/samza/job/lib/netty-3.6.2.Final.jar:/opt/samza/job/lib/paranamer-2.3.jar:/opt/samza/job/lib/protobuf-java-2.5.0.jar:/opt/samza/job/lib/rocksdbjni-5.7.3.jar:/opt/samza/job/lib/samza-api-0.14.1.jar:/opt/samza/job/lib/samza-core_2.11-0.14.1.jar:/opt/samza/job/lib/samza-core_2.12-0.14.1.jar:/opt/samza/job/lib/samza-kafka_2.12-0.14.1.jar:/opt/samza/job/lib/samza-kv-inmemory_2.12-0.14.1.jar:/opt/samza/job/lib/samza-kv-rocksdb_2.12-0.14.1.jar:/opt/samza/job/lib/samza-kv_2.12-0.14.1.jar:/opt/samza/job/lib/samza-log4j-0.14.1.jar:/opt/samza/job/lib/samza-yarn_2.12-0.14.1.jar:/opt/samza/job/lib/scala-compiler-2.12.1.jar:/opt/samza/job/lib/scala-library-2.12.1.jar:/opt/samza/job/lib/scala-parser-combinators_2.12-1.0.4.jar:/opt/samza/job/lib/scala-reflect-2.12.1.jar:/opt/samza/job/lib/scala-xml_2.12-1.0.6.jar:/opt/samza/job/lib/scalate-core_2.12-1.8.0.jar:/opt/samza/job/lib/scalate-util_2.12-1.8.0.jar:/opt/samza/job/lib/scalatra-common_2.12-2.5.0.jar:/opt/samza/job/lib/scalatra-scalate_2.12-2.5.0.jar:/opt/samza/job/lib/scalatra_2.12-2.5.0.jar:/opt/samza/job/lib/servlet-api-2.5.jar:/opt/samza/job/lib/slf4j-api-1.6.2.jar:/opt/samza/job/lib/slf4j-log4j12-1.6.1.jar:/opt/samza/job/lib/snappy-java-1.1.2.6.jar:/opt/samza/job/lib/soxtestjob-1.0.0.jar:/opt/samza/job/lib/stax-api-1.0-2.jar:/opt/samza/job/lib/xercesImpl-2.9.1.jar:/opt/samza/job/lib/xml-apis-1.3.04.jar:/opt/samza/job/lib/xmlenc-0.52.jar:/opt/samza/job/lib/xz-1.0.jar:/opt/samza/job/lib/zkclient-0.8.jar:/opt/samza/job/lib/zookeeper-3.4.6.jar org.apache.samza.job.JobRunner --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file:///opt/samza/job/config/job.properties
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.samza.util.Logging.$init$(Lorg/apache/samza/util/Logging;)V
at org.apache.samza.job.yarn.YarnJobFactory.<init>(YarnJobFactory.scala:31)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.samza.job.JobRunner.getJobFactory(JobRunner.scala:175)
at org.apache.samza.job.JobRunner.run(JobRunner.scala:80)
at org.apache.samza.job.JobRunner$.doOperation(JobRunner.scala:53)
at org.apache.samza.job.JobRunner$.main(JobRunner.scala:48)
at org.apache.samza.job.JobRunner.main(JobRunner.scala)
{code}
The dependencies of the job are as follows.
{code}
<dependencies>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-log4j</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-api</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-core_2.12</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-shell</artifactId>
<classifier>dist</classifier>
<type>tgz</type>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-yarn_2.12</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-kv_2.12</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-kv-rocksdb_2.12</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-kv-inmemory_2.12</artifactId>
<version>0.14.1</version>
</dependency>
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-kafka_2.12</artifactId>
<version>0.14.1</version>
</dependency>
</dependencies>
{code}
It appears that the samza-log4j v0.14.1 depends on samza-core_2.11, which conflicts with samza-core_2.12. Interestingly, samza-log4j v0.14.0 depends on samza-core_2.12. The issue is resolved if we exclude the samza-core_2.11 dependency from samza-log4j.
{code}
<dependency>
<groupId>org.apache.samza</groupId>
<artifactId>samza-log4j</artifactId>
<version>0.14.1</version>
<exclusion>
<groupId>org.apache.samza</groupId>
<artifactId>samza-core_2.11</artifactId>
</exclusion>
</exclusions>
</dependency>
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)