You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Grant Henke (Jira)" <ji...@apache.org> on 2020/09/15 19:33:00 UTC
[jira] [Resolved] (KUDU-3157) kudu-spark2 jar is now including
slf4j classes
[ https://issues.apache.org/jira/browse/KUDU-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Henke resolved KUDU-3157.
-------------------------------
Fix Version/s: 1.14.0
Resolution: Fixed
> kudu-spark2 jar is now including slf4j classes
> ----------------------------------------------
>
> Key: KUDU-3157
> URL: https://issues.apache.org/jira/browse/KUDU-3157
> Project: Kudu
> Issue Type: Bug
> Components: spark
> Affects Versions: 1.12.0
> Reporter: Jeff Evans
> Assignee: Grant Henke
> Priority: Major
> Fix For: 1.14.0
>
>
> It seems that sometime between versions {{1.7.0}} and {{1.12.0}} of {{org.apache.kudu:kudu-spark2_2.11}}, some SLF4J classes started being included.
> {code}
> # for 1.7.0
> jar tvf ~/.m2/repository/org/apache/kudu/kudu-spark2_2.11/1.7.0/kudu-spark2_2.11-1.7.0.jar | grep slf4j
> 1203 Wed Jun 29 14:43:04 CDT 2016 META-INF/license/LICENSE.slf4j.txt
> #for 1.12.0
> jar tvf ~/.m2/repository/org/apache/kudu/kudu-spark2_2.11/1.12.0/kudu-spark2_2.11-1.12.0.jar | grep slf4j
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/event/
> 555 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/EventConstants.class
> 9842 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/EventRecodingLogger.class
> 1450 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/Level.class
> 432 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/LoggingEvent.class
> 2615 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/SubstituteLoggingEvent.class
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/helpers/
> 3285 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMarker.class
> 1634 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMarkerFactory.class
> 1233 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMDCAdapter$1.class
> 2618 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMDCAdapter.class
> 1091 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/FormattingTuple.class
> 4773 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/MarkerIgnoringBase.class
> 7553 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/MessageFormatter.class
> 823 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NamedLoggerBase.class
> 3267 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPLogger.class
> 584 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPLoggerFactory.class
> 1249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPMDCAdapter.class
> 10262 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/SubstituteLogger.class
> 2199 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/SubstituteLoggerFactory.class
> 196 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/Util$1.class
> 765 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/Util$ClassContextSecurityManager.class
> 2952 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/Util.class
> 180 Mon Dec 16 22:03:32 CST 2019 org/slf4j/ILoggerFactory.class
> 272 Mon Dec 16 22:03:32 CST 2019 org/slf4j/IMarkerFactory.class
> 1375 Mon Dec 16 22:03:32 CST 2019 org/slf4j/Logger.class
> 12575 Mon Dec 16 22:03:32 CST 2019 org/slf4j/LoggerFactory.class
> 666 Mon Dec 16 22:03:32 CST 2019 org/slf4j/Marker.class
> 1621 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MarkerFactory.class
> 177 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC$1.class
> 749 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC$MDCCloseable.class
> 3675 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC.class
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/spi/
> 455 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/LocationAwareLogger.class
> 249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/LoggerFactoryBinder.class
> 249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/MarkerFactoryBinder.class
> 529 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/MDCAdapter.class
> 0 Mon Apr 27 14:53:30 CDT 2020 META-INF/maven/org.slf4j/
> 0 Mon Apr 27 14:53:30 CDT 2020 META-INF/maven/org.slf4j/slf4j-api/
> 3835 Mon Dec 16 21:59:52 CST 2019 META-INF/maven/org.slf4j/slf4j-api/pom.xml
> 109 Mon Dec 16 22:00:08 CST 2019 META-INF/maven/org.slf4j/slf4j-api/pom.properties
> {code}
> These should be removed from the jar, or at least relocated so as not to clash with Spark vendor dependencies, which can cause {{LinkageError}} in Spark clusters. Real life example of this problem, seen on a Spark job started on CDH 6.3 cluster:
> {code}
> Exception in thread "main" java.lang.LinkageError: loader constraint violation: when resolving method "org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;" the class loader (instance of org/apache/spark/util/ChildFirstURLClassLoader) of the current class, org/slf4j/LoggerFactory, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for the method\'s defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)