You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Randall Hauch (JIRA)" <ji...@apache.org> on 2017/12/06 14:51:00 UTC
[jira] [Comment Edited] (KAFKA-6313) Kafka Core maven dependencies
are missing SLF4J API
[ https://issues.apache.org/jira/browse/KAFKA-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16280260#comment-16280260 ]
Randall Hauch edited comment on KAFKA-6313 at 12/6/17 2:50 PM:
---------------------------------------------------------------
[~ijuma], you are correct that other dependencies in my project caused the SLF4J API artifact to be missed. It wasn't an exclusion, but it was due to including a slightly older version 3.4.6 of {{org.apache.zookeeper:zookeeper}}:
{noformat}
+- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
| +- org.slf4j:slf4j-api:jar:1.6.1:compile
| \- io.netty:netty:jar:3.7.0.Final:compile
{noformat}
Because ZK depends *directly* on SLF4J API while Kafka depends *indirectly*, Maven's [transitive dependency rules|https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies] used ZK's direct dependency to identify the version conflict. Reordering in my project Kafka's dependency first doesn't help, though I could force it by manually adding the SLF4J dependency (now I'd have to manage the version) or the Kafka client dependency (which does have a direct dependency on SLF4J).
So, you are correct that the existing dependencies should have brought in the SLF4J if there were no other conflicts/exclusions, and that we should make Kafka's dependency direct rather than indirect. However, because this is so difficult to fix, I'd still characterize this as more important because of how Maven's dependency conflict resolution works.
was (Author: rhauch):
[~ijuma], you are correct that other dependencies in my project caused the SLF4J API artifact to be missed. It wasn't an exclusion, but it was due to including a slightly older version 3.4.6 of {{org.apache.zookeeper:zookeeper}}:
+- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
| +- org.slf4j:slf4j-api:jar:1.6.1:compile
| \- io.netty:netty:jar:3.7.0.Final:compile
Because ZK depends *directly* on SLF4J API while Kafka depends *indirectly*, Maven's [transitive dependency rules|https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies] used ZK's direct dependency to identify the version conflict. Reordering in my project Kafka's dependency first doesn't help, though I could force it by manually adding the SLF4J dependency (now I'd have to manage the version) or the Kafka client dependency (which does have a direct dependency on SLF4J).
So, you are correct that the existing dependencies should have brought in the SLF4J if there were no other conflicts/exclusions, and that we should make Kafka's dependency direct rather than indirect. However, because this is so difficult to fix, I'd still characterize this as more important because of how Maven's dependency conflict resolution works.
> Kafka Core maven dependencies are missing SLF4J API
> ---------------------------------------------------
>
> Key: KAFKA-6313
> URL: https://issues.apache.org/jira/browse/KAFKA-6313
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 1.1.0
> Reporter: Randall Hauch
> Assignee: Ismael Juma
> Priority: Minor
> Fix For: 1.1.0
>
>
> In an application that depends on the Kafka server artifacts with:
> {code:xml}
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_2.11</artifactId>
> <version>1.1.0-SNAPSHOT</version>
> </dependency>
> {code}
> and then running the server programmatically, the following error occurs:
> {noformat}
> [2017-11-23 01:01:45,029] INFO Shutting down producer (kafka.producer.Producer:63)
> [2017-11-23 01:01:45,051] INFO Closing all sync producers (kafka.producer.ProducerPool:63)
> [2017-11-23 01:01:45,052] INFO Producer shutdown completed in 23 ms (kafka.producer.Producer:63)
> [2017-11-23 01:01:45,052] INFO [KafkaServer id=1] shutting down (kafka.server.KafkaServer:63)
> [2017-11-23 01:01:45,057] ERROR [KafkaServer id=1] Fatal error during KafkaServer shutdown. (kafka.server.KafkaServer:161)
> java.lang.NoClassDefFoundError: org/slf4j/event/Level
> at kafka.utils.CoreUtils$.swallow$default$3(CoreUtils.scala:83)
> at kafka.server.KafkaServer.shutdown(KafkaServer.scala:520)
> ...
> Caused by: java.lang.ClassNotFoundException: org.slf4j.event.Level
> at java.net.URLClassLoader$1.run(URLClassLoader.java:359)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 25 more
> {noformat}
> It appears that KAFKA-1044 and [this PR|https://github.com/apache/kafka/pull/3477] removed the use of Log4J from Core but [added use of|https://github.com/confluentinc/kafka/commit/ed8b0315a6c3705b2a163ce3ab4723234779264f#diff-52505b9374ea885e44bcb73cbc4714d6R34] the {{org.slf4j.event.Level}} in {{CoreUtils.scala}}. The {{org.slf4j.event.Level}} class is in the {{org.slf4j:slf4j-api}} artifact, which is currently not included in the dependencies of {{org.apache.kafka:kafka_2.11:1.1.0-SNAPSHOT}}. Because this is needed by the server, the SLF4J API library probably needs to be added to the dependencies.
> [~viktorsomogyi] and [~ijuma], was this intentional, or is it intended that the SLF4J API be marked as {{provided}}? BTW, I marked this as CRITICAL just because this probably needs to be sorted out before the release.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)