[jira] [Commented] (KAFKA-7741) Bad dependency via SBT

["John Roesler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723514#comment-16723514 ] 

John Roesler commented on KAFKA-7741:
-------------------------------------

Hi [~sachabarber],

Thanks for the report. I've not seen this error before, but I can confirm I also get the same error on my machine with:

build.sbt:
{noformat}
name := "ScalaKafkaStreamsDemo"

version := "1.0"

scalaVersion := "2.12.1"

libraryDependencies += "org.apache.kafka" %% "kafka" % "2.1.0"
libraryDependencies += "org.apache.kafka" % "kafka-clients" % "2.1.0"
libraryDependencies += "org.apache.kafka" % "kafka-streams" % "2.1.0"
libraryDependencies += "javax.ws.rs" % "javax.ws.rs-api" % "2.1.1"
libraryDependencies += "org.apache.kafka" %% "kafka-streams-scala" % "2.1.0"

//TEST
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % Test
libraryDependencies += "org.apache.kafka" % "kafka-streams-test-utils" % "2.1.0" % Test
{noformat}
and:
{noformat}
$ sbt compile
[info] Loading settings for project global-plugins from idea.sbt,plugins.sbt ...
[info] Loading global plugins from /home/john/.sbt/1.0/plugins
[info] Loading project definition from /tmp/test/project
[info] Loading settings for project test from build.sbt ...
[info] Set current project to ScalaKafkaStreamsDemo (in build file:/tmp/test/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] Updating ...
[warn] Detected merged artifact: [FAILED ] javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}: (0ms).
[warn] ==== local: tried
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
[warn] ==== local-preloaded-ivy: tried
[warn] /home/john/.sbt/preloaded/javax.ws.rs/javax.ws.rs-api/2.1.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
[warn] ==== local-preloaded: tried
[warn] file:////home/john/.sbt/preloaded/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
...{noformat}
That URL has an unsubstituted variable in it, so I did a little searching for "sbt packaging.type", and it looks like it might be a bug in the way that Maven and SBT are working together.

See for example: [https://github.com/ronmamo/reflections/issues/169]

It seems like the general approach people are using is to just make SBT replace "${packaging.type}" with "jar" (the actual artifact is at https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.jar").

For example:
{noformat}
$ sbt -Dpackaging.type=jar compile
[info] Loading settings for project global-plugins from idea.sbt,plugins.sbt ...
[info] Loading global plugins from /home/john/.sbt/1.0/plugins
[info] Loading project definition from /tmp/test/project
[info] Loading settings for project test from build.sbt ...
[info] Set current project to ScalaKafkaStreamsDemo (in build file:/tmp/test/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] Updating ...
[info] downloading https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.jar ...
[info] [SUCCESSFUL ] javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.jar (861ms)
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[success] Total time: 2 s, completed Dec 17, 2018, 5:45:42 PM
{noformat}
 

As far as what we can do in Kafka, I'm not sure...

The dependency chain for this dependency is:
{noformat}
$ sbt "whatDependsOn javax.ws.rs"
...
[info] javax.ws.rs:javax.ws.rs-api:2.1.1
[info] +-org.apache.kafka:connect-api:2.1.0
[info] | +-org.apache.kafka:connect-json:2.1.0
[info] | +-org.apache.kafka:kafka-streams:2.1.0
[info] | +-org.apache.kafka:kafka-streams-scala_2.12:2.1.0 [S]
[info] | | +-scalakafkastreamsdemo:scalakafkastreamsdemo_2.12:1.0 [S]
[info] | |
[info] | +-scalakafkastreamsdemo:scalakafkastreamsdemo_2.12:1.0 [S]
[info] |
[info] +-scalakafkastreamsdemo:scalakafkastreamsdemo_2.12:1.0 [S]
{noformat}
You can directly see the connect-api:2.1.0 pom file at: [http://central.maven.org/maven2/org/apache/kafka/connect-api/2.1.0/connect-api-2.1.0.pom]

 

In my web searches, I found [https://github.com/eclipse-ee4j/jaxrs-api/issues/572,] which seems to be the root cause: the javax.ws.rs-api pom doesn't specify "<packaging>jar</packaging>". 

The last comment on that posts suggests that we should change coordinates and update the dependency to fix the issue. But I don't know much about what's been happening with that project to know if this is really ok to do...

Since this seems to be a problem with Connect's dependency, I'm changing the label from "streams" to "connect", to hopefully get some attention from people who might be a little more familiar with javax/jakarta to comment.

 

Sorry for your troubles... Can you confirm if that workaround above works for you?

-john

> Bad dependency via SBT
> ----------------------
>
>                 Key: KAFKA-7741
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7741
>             Project: Kafka
>          Issue Type: Bug
>          Components: core, KafkaConnect
>    Affects Versions: 2.1.0
>         Environment: Windows 10 professional, IntelliJ IDEA 2017.1
>            Reporter: sacha barber
>            Priority: Major
>
> I am using the Kafka-Streams-Scala 2.1.0 JAR.
> And if I create a new Scala project using SBT with these dependencies 
> {code}
> name := "ScalaKafkaStreamsDemo"
> version := "1.0"
> scalaVersion := "2.12.1"
> libraryDependencies += "org.apache.kafka" %% "kafka" % "2.0.0"
> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "2.0.0"
> libraryDependencies += "org.apache.kafka" % "kafka-streams" % "2.0.0"
> libraryDependencies += "org.apache.kafka" %% "kafka-streams-scala" % "2.0.0"
> //TEST
> libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % Test
> libraryDependencies += "org.apache.kafka" % "kafka-streams-test-utils" % "2.0.0" % Test
> {code}
> I get this error
>  
> {code}
> SBT 'ScalaKafkaStreamsDemo' project refresh failed
> Error:Error while importing SBT project:<br/>...<br/><pre>[info] Resolving jline#jline;2.14.1 ...
> [warn] [FAILED ] javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}: (0ms)
> [warn] ==== local: tried
> [warn] C:\Users\sacha\.ivy2\local\javax.ws.rs\javax.ws.rs-api\2.1.1\${packaging.type}s\javax.ws.rs-api.${packaging.type}
> [warn] ==== public: tried
> [warn] https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
> [info] downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka-streams-test-utils/2.1.0/kafka-streams-test-utils-2.1.0.jar ...
> [info] [SUCCESSFUL ] org.apache.kafka#kafka-streams-test-utils;2.1.0!kafka-streams-test-utils.jar (344ms)
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [warn] :: FAILED DOWNLOADS ::
> [warn] :: ^ see resolution messages for details ^ ::
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [warn] :: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
> [trace] Stack trace suppressed: run 'last *:update' for the full output.
> [error] (*:ssExtractDependencies) sbt.ResolveException: download failed: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [error] (*:update) sbt.ResolveException: download failed: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [error] Total time: 8 s, completed 16-Dec-2018 19:27:21
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0</pre><br/>See complete log in <a href="file:/C:/Users/sacha/.IdeaIC2017.1/system/log/sbt.last.log">file:/C:/Users/sacha/.IdeaIC2017.1/system/log/sbt.last.log</a>
> {code}
> This seems to be a common issue with bad dependency from Kafka to javax.ws.rs-api.
> if I drop the Kafka version down to 2.0.0 and add this line to my SBT file this error goes away
> {code}
> libraryDependencies += "javax.ws.rs" % "javax.ws.rs-api" % "2.1" artifacts(Artifact("javax.ws.rs-api", "jar", "jar"))`
> {code}
>  
> However I would like to work with 2.1.0 version.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)