[jira] [Comment Edited] (KAFKA-7741) Bad dependency via SBT

["Randall Hauch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724303#comment-16724303 ] 

Randall Hauch edited comment on KAFKA-7741 at 12/18/18 5:54 PM:
----------------------------------------------------------------

[~vvcephei], thanks for looking into this. A couple of things.

This dependency was added as part of [KIP-285|https://cwiki.apache.org/confluence/display/KAFKA/KIP-285%3A+Connect+Rest+Extension+Plugin], which added a new extension point for custom REST handlers and filters within the Connect worker. This is not needed by client applications, so it's possible that the dependency could be excluded by client applications that need the Connect API. The Connect API was originally intended to be used by projects that implement connectors, transforms, and converter, as all implementations were (at least originally) used within the Connect worker. Apparently Streams and other applications may want to use Connect converters, so we may want to consider a KIP that splits out the portion of the Connect API on which clients can depend. We just haven't needed this distinction so far.

Second, you said the following in an earlier comment:
{quote}
Since this seems to be a problem with Connect's dependency, I'm changing the label from "streams" to "connect", to hopefully get some attention from people who might be a little more familiar with javax/jakarta to comment.
{quote}
Maybe it's a minor point, but I'm not sure I agree that it's a problem with the Connect API dependency -- otherwise Connect would not build. As you mention and reference, the real problem is a malformed POM for the ws.rs-api library. Yes, the problem is exposed transitively via the Connect API dependencies.

Third, isn't another workaround to simply exclude the `{{javax.ws.rs:javax.ws.rs-api:2.1.0}}` dependency from all Kafka Streams artifacts and applications?

Fourth, you suggested that we should either update the docs or update the dependency, but don't we have to do both? AK 2.0.0 and 2.1.0 are out the door, and anyone using those will need to use one of the workarounds. IMO, we should also fix the build to use the more recent and corrected version of the dependency.

Finally, IIUC your suggestion is to change the dependency from `{{javax.ws.rs:javax.ws.rs-api:2.1.0}}` to `{{jakarta.ws.rs:jakarta.ws.rs-api:2.1.3}}`, which does have a well-formed POM. IIUC, upgrading to `{{javax.ws.rs:javax.ws.rs-api:2.1.1}}` won't help because it doesn't have a well-formed POM? And [nothing newer than 2.1.1 is in Maven Central|https://search.maven.org/search?q=g:javax.ws.rs%20AND%20a:javax.ws.rs-api&core=gav]. Is this correct?

If so, then I think it makes sense to upgrade this dependency and merge as far back as the `2.0` branch for inclusion subsequent patch releases. However, we need to be careful -- this dependency was used because it is required by the Jersey implementation that the Connect worker uses, so we'd have to first verify that `{{jakarta.ws.rs:jakarta.ws.rs-api:2.1.3}}` is equivalent and works with the Jetty implementation.



was (Author: rhauch):
[~vvcephei], thanks for looking into this. A could of things.

This dependency was added as part of [KIP-285|https://cwiki.apache.org/confluence/display/KAFKA/KIP-285%3A+Connect+Rest+Extension+Plugin], which added a new extension point for custom REST handlers and filters within the Connect worker. This is not needed by client applications, so it's possible that the dependency could be excluded by client applications that need the Connect API. The Connect API was originally intended to be used by projects that implement connectors, transforms, and converter, as all implementations were (at least originally) used within the Connect worker. Apparently Streams and other applications may want to use Connect converters, so we may want to consider a KIP that splits out the portion of the Connect API on which clients can depend. We just haven't needed this distinction so far.

Second, you said the following in an earlier comment:
{quote}
Since this seems to be a problem with Connect's dependency, I'm changing the label from "streams" to "connect", to hopefully get some attention from people who might be a little more familiar with javax/jakarta to comment.
{quote}
Maybe it's a minor point, but I'm not sure I agree that it's a problem with the Connect API dependency -- otherwise Connect would not build. As you mention and reference, the real problem is a malformed POM for the ws.rs-api library. Yes, the problem is exposed transitively via the Connect API dependencies.

Third, isn't another workaround to simply exclude the `{{javax.ws.rs:javax.ws.rs-api:2.1.0}}` dependency from all Kafka Streams artifacts and applications?

Fourth, you suggested that we should either update the docs or update the dependency, but don't we have to do both? AK 2.0.0 and 2.1.0 are out the door, and anyone using those will need to use one of the workarounds. IMO, we should also fix the build to use the more recent and corrected version of the dependency.

Finally, IIUC your suggestion is to change the dependency from `{{javax.ws.rs:javax.ws.rs-api:2.1.0}}` to `{{jakarta.ws.rs:jakarta.ws.rs-api:2.1.3}}`, which does have a well-formed POM. IIUC, upgrading to `{{javax.ws.rs:javax.ws.rs-api:2.1.1}}` won't help because it doesn't have a well-formed POM? And [nothing newer than 2.1.1 is in Maven Central|https://search.maven.org/search?q=g:javax.ws.rs%20AND%20a:javax.ws.rs-api&core=gav]. Is this correct?

If so, then I think it makes sense to upgrade this dependency and merge as far back as the `2.0` branch for inclusion subsequent patch releases. However, we need to be careful -- this dependency was used because it is required by the Jersey implementation that the Connect worker uses, so we'd have to first verify that `{{jakarta.ws.rs:jakarta.ws.rs-api:2.1.3}}` is equivalent and works with the Jetty implementation.


> Bad dependency via SBT
> ----------------------
>
>                 Key: KAFKA-7741
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7741
>             Project: Kafka
>          Issue Type: Bug
>          Components: core, KafkaConnect
>    Affects Versions: 2.1.0
>         Environment: Windows 10 professional, IntelliJ IDEA 2017.1
>            Reporter: sacha barber
>            Priority: Major
>
> I am using the Kafka-Streams-Scala 2.1.0 JAR.
> And if I create a new Scala project using SBT with these dependencies 
> {code}
> name := "ScalaKafkaStreamsDemo"
> version := "1.0"
> scalaVersion := "2.12.1"
> libraryDependencies += "org.apache.kafka" %% "kafka" % "2.0.0"
> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "2.0.0"
> libraryDependencies += "org.apache.kafka" % "kafka-streams" % "2.0.0"
> libraryDependencies += "org.apache.kafka" %% "kafka-streams-scala" % "2.0.0"
> //TEST
> libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % Test
> libraryDependencies += "org.apache.kafka" % "kafka-streams-test-utils" % "2.0.0" % Test
> {code}
> I get this error
>  
> {code}
> SBT 'ScalaKafkaStreamsDemo' project refresh failed
> Error:Error while importing SBT project:<br/>...<br/><pre>[info] Resolving jline#jline;2.14.1 ...
> [warn] [FAILED ] javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}: (0ms)
> [warn] ==== local: tried
> [warn] C:\Users\sacha\.ivy2\local\javax.ws.rs\javax.ws.rs-api\2.1.1\${packaging.type}s\javax.ws.rs-api.${packaging.type}
> [warn] ==== public: tried
> [warn] https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
> [info] downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka-streams-test-utils/2.1.0/kafka-streams-test-utils-2.1.0.jar ...
> [info] [SUCCESSFUL ] org.apache.kafka#kafka-streams-test-utils;2.1.0!kafka-streams-test-utils.jar (344ms)
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [warn] :: FAILED DOWNLOADS ::
> [warn] :: ^ see resolution messages for details ^ ::
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [warn] :: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
> [trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
> [trace] Stack trace suppressed: run 'last *:update' for the full output.
> [error] (*:ssExtractDependencies) sbt.ResolveException: download failed: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [error] (*:update) sbt.ResolveException: download failed: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
> [error] Total time: 8 s, completed 16-Dec-2018 19:27:21
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0</pre><br/>See complete log in <a href="file:/C:/Users/sacha/.IdeaIC2017.1/system/log/sbt.last.log">file:/C:/Users/sacha/.IdeaIC2017.1/system/log/sbt.last.log</a>
> {code}
> This seems to be a common issue with bad dependency from Kafka to javax.ws.rs-api.
> if I drop the Kafka version down to 2.0.0 and add this line to my SBT file this error goes away
> {code}
> libraryDependencies += "javax.ws.rs" % "javax.ws.rs-api" % "2.1" artifacts(Artifact("javax.ws.rs-api", "jar", "jar"))`
> {code}
>  
> However I would like to work with 2.1.0 version.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)