You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/04/02 19:26:34 UTC
[GitHub] [kafka] dejan2609 opened a new pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
dejan2609 opened a new pull request #10466:
URL: https://github.com/apache/kafka/pull/10466
note: remaining _**'testRuntime'**_ reference changed to _**'testRuntimeOnly'**_
Related link: https://docs.gradle.org/7.0-rc-2/userguide/upgrading_version_6.html#sec:configuration_removal
FYI @ijuma
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r620886540
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
That line should be including test dependencies like log4j, etc. But I don't know why that makes sense for streams. It does make sense for core. So, I would remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821968689
> Could we reintroduce a configuration that behaves the same as `testRuntime` did before it was removed?
@ijuma Can you please expand on this ? Are you referring to a `testRuntime` mentioned in a `copyDependantLibs` task ?
https://github.com/apache/kafka/blob/2.8.0-rc2/build.gradle#L1472
```
project(':streams') {
...
tasks.create(name: "copyDependantLibs", type: Copy) {
from (configurations.testRuntime) {
...
}
...
}
```
Or we now just brainstorm about Gradle behavior _**in-general**_ ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-826400303
@dejan2609 can you please rebase on trunk?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r621504000
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
Ok, we have a deal then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma merged pull request #10466: KAFKA-12417: streams `copyDependentLibs` should not copy testRuntime configuration jars
Posted by GitBox <gi...@apache.org>.
ijuma merged pull request #10466:
URL: https://github.com/apache/kafka/pull/10466
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-828682155
Unrelated failures, merging to trunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-826400303
@dejan2609 can you please rebase on trunk?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r620558374
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
- include('slf4j-log4j12*')
- include('log4j*jar')
- include('*hamcrest*')
+ from (configurations.testCompileClasspath) {
+ include('jackson*')
+ include('slf4j-api*')
}
from (configurations.runtimeClasspath) {
- exclude('kafka-clients*')
+ include('connect*')
+ include('*java*')
+ include('*jni*')
Review comment:
Nice catch @ijuma ! Thing is that my intention was to put forward some solution that just **_works_** (in a same way that previous, see attached snapshots below).
So, I moved around **_java_** and **_java-library_** plugins configurations and after scrapping something I just left solution as-is... of course, **_exclude_** declaration is an optimal solution (because it handles dependencies that will be included into related configuration in the future).
All-in-all: reverting back to an **_'exclude'_** directive.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-826067611
@ijuma quick update: I did made some progress developing a ~~_quick and dirty_~~ **fast** solution for this... will update this ticket today or tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821831500
Also, after some digging I stumbled upon these GitHub resources (I will intentionally leave them formatted as code):
- `https://github.com/gradle/gradle/issues/6353`
- `https://github.com/gradle/gradle/issues/847` (especially comment: `https://github.com/gradle/gradle/issues/847#issuecomment-287515195`):
> Came across this and was looking for a workaround...so in case anyone else is trying to find a way out: it seems sufficient to change `group` for the subprojects such that they end up with different module coordinates
I tested this quick and dirty solution (experimented with changing `group` for subprojects) but that action yield no results... 😐
Also, I played around with dependencies and tasks:
- ./gradlew streams:dependencies > DEPS.txt
- ./gradlew tiTree :streams:copyDependantLibs > TasksGraph.txt
- ./gradlew tiOrder :streams:copyDependantLibs > TasksOrder.txt (appropriate plugin is added in order to create Gradle taks graph and list).
So, after some time, I moved to a phase two.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r619888968
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
- include('slf4j-log4j12*')
- include('log4j*jar')
- include('*hamcrest*')
+ from (configurations.testCompileClasspath) {
+ include('jackson*')
+ include('slf4j-api*')
}
from (configurations.runtimeClasspath) {
- exclude('kafka-clients*')
+ include('connect*')
+ include('*java*')
+ include('*jni*')
Review comment:
Why did we switch from exclude to include?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821841078
So, I started to abandon quick and dirty solution and to lean towards something heavier (i.e. to a some form of refactoring):
- https://discuss.gradle.org/t/gradle-support-for-cyclic-dependencies/14355/2
- https://stackoverflow.com/questions/38062841/how-to-resolve-circular-dependency-in-gradle/38063421#38063421
- https://stackoverflow.com/questions/58178381/resolve-circular-dependency-in-gradle/58186686#58186686
Gradle introduced new thing (among many others): `java-test-fixtures` plugin (that goes hand in hand with `java-library') that is able to fine-grain test related dependency configurations:
- https://docs.gradle.org/6.8.3/userguide/java_testing.html#declaring_dependencies_of_test_fixtures (some browsers show page bottom... in that case scroll up to **_"Declaring dependencies of test fixtures"_*)
- https://stackoverflow.com/questions/5644011/multi-project-test-dependencies-with-gradle/60138176#60138176
- https://stackoverflow.com/questions/64133013/gradle-test-fixtures-plugin-and-core-module-dependencies
- https://stackoverflow.com/questions/5144325/gradle-test-dependency/66138658#66138658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r619888968
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
- include('slf4j-log4j12*')
- include('log4j*jar')
- include('*hamcrest*')
+ from (configurations.testCompileClasspath) {
+ include('jackson*')
+ include('slf4j-api*')
}
from (configurations.runtimeClasspath) {
- exclude('kafka-clients*')
+ include('connect*')
+ include('*java*')
+ include('*jni*')
Review comment:
Why did we switch from exclude to include?
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
@vvcephei do you know why we're copying test dependencies here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-827067268
@ijuma branch is rebased onto trunk (I can also squash all commits into one).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-827844379
@ijuma previous commits are rebased/squashed and PR branch is force-pushed.
Overall: Gradle build looks ok (on my loptop, that is). Some warnings do appear (related to spotless plugin / scala compilation) but I guess those things can be solved elsewhere (I volunteer to create JIRA tickets and see what can be done).
I opted to change commit message in order to emphasize Gradle version upgrade (if that is ok with you we can also change summaries for JIRA ticket and this PR).
One more note: Gradle patch _**7.0.1**_ is around the corner (with lots of issues being solved: https://github.com/gradle/gradle/milestone/173).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-826067611
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-827849319
Sure, let go step-by-step then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 edited a comment on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 edited a comment on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821837798
Digging through project history I managed to find some commits that are seems to be related (see the comment):
```
project(':streams') {
dependencies {
api project(':clients')
....
// testCompileOnly prevents streams from exporting a dependency on test-utils, which would cause a dependency cycle
testCompileOnly project(':streams:test-utils')
....
testRuntimeOnly project(':streams:test-utils')
}
....
}
```
@vvcephei contributed two PRs (#4821 and #4812) 3 years ago and these are related commits:
- **05. Apr 2018**. https://github.com/apache/kafka/commit/d5db4e9b8055536bdbb3b2d2308e36efa4abafcc
- **23. Apr 2018.** https://github.com/apache/kafka/commit/ed51b2cdf5bdac210a6904bead1a2ca6e8411406
Out of curiosity I tried to consolidate these dependency into one `testImplementation project(':streams:test-utils')` and here are the results:
- on top of trunk: build works fine ✔️
- on top of this PR: build fails ❌ (with identical `Circular dependency between the following tasks` error as above)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-827855369
Done, changes are trimmed down.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821837798
Digging through project history I managed to find some commits that are seems to be related (see the comment):
```
project(':streams') {
dependencies {
api project(':clients')
....
// testCompileOnly prevents streams from exporting a dependency on test-utils, which would cause a dependency cycle
testCompileOnly project(':streams:test-utils')
....
testRuntimeOnly project(':streams:test-utils')
}
....
}
```
@vvcephei contributed two PRs (#4821 and #4812) 3 years ago and these are related commits:
- **05. Apr 2018**. https://github.com/apache/kafka/commit/d5db4e9b8055536bdbb3b2d2308e36efa4abafcc
- **23. Apr 2018.** https://github.com/apache/kafka/commit/ed51b2cdf5bdac210a6904bead1a2ca6e8411406
Out of curiosity I tried to consolidate these dependency into one `testImplementation project(':streams:test-utils')` and here are the results:
- on top of trunk: build works fine ✔️
- on top of this PR: build fails ❌ (with identical `Circular dependency between the following tasks` error as above)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821841765
Ok, I finished my writings above (I just had to be as detailed as possible, these comments serve me a lot).
> Could we reintroduce a configuration that behaves the same as `testRuntime` did before it was removed?
Well, I can try to revert to a previous solution, see how that goes and build it from there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-812678436
The change I had attempted was slightly different: https://issues.apache.org/jira/browse/KAFKA-12417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r620580912
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
Hint/note: this **_'from'_** clause could be redundant now:

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-817239263
Gradle 7.0 has been released. Any luck figuring out the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-812677777
I filed a JIRA for this issue, the build failed with this change before. Did it work for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-826381116
@ijuma So, now I have something to show: I managed perform some reverse-engineering and received exactly the same content in `/streams/build/dependent-libs-${version.scala}` folder (that folder is being filled with dependencies by a gradle task `streams:copyDependantLibs`). _Mental note: it would be interesting to fully discover this task background and usage._
Also: Gradle version is bumped (from **6.8.3** to **7.0**).
Pitfall: `./gradlew clean streams:copyDependantLibs` execution times are significantly higher when comparing to a trunk... will try to sort that out, if possible.
|git branch | Gradle 6.8.3 | Gradle 7.0 |
| --- | --: | --: |
| trunk | 16 seconds | **_not applicable_** |
| this PR | 1 minute 55 seconds | 2 minutes 10 seconds |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821826801
I am going to split my findings to a separate comments... here it goes.
I decided to work with Gradle 6.8.3 for the moment (that is, until circular dependency issue is resolved). After that we can try with Gradle 7.0.
So, I am going to change title and align it with a corresponding JIRA ticket that was mentioned above: https://issues.apache.org/jira/browse/KAFKA-12417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-812690240
I just pushed `configurations.testRuntimeOnly.setCanBeResolved(true)` (I forgot to include that directive into my first commit).
And yes, I do receive same circular dependency error with Gradle 7 (as you mentioned in a JIRA ticket above):
```
./gradlew -PscalaVersion=2.12 jar
......
* What went wrong:
Circular dependency between the following tasks:
:streams:copyDependantLibs
+--- :streams:jar
| \--- :streams:copyDependantLibs (*)
\--- :streams:test-utils:jar
\--- :streams:test-utils:copyDependantLibs
\--- :streams:jar (*)
......
```
This Gradle issue seems to be related : https://github.com/gradle/gradle/issues/16604 (see step 4. under ***Steps to Reproduce***).
@ijuma I can take that JIRA ticket and hopefully provide some solution. Is that ok with you ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma edited a comment on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
ijuma edited a comment on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821839178
Could we reintroduce a configuration that behaves the same as `testRuntime` did before it was removed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-812691270
@dejan2609 yes, please take the ticket. It would be great to solve this so that we can build with Java 16.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-827846345
Can you remove the Gradle version bump from this PR then? We can merge the other change and see if there's any impact. We can then go straight to Gradle 7.0.1 once it's released. It may take 1-3 weeks from the link you shared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r619889058
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
@vvcephei do you know why we're copying test dependencies here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821839178
Could we reintroduce a configuration that behaved the same as `testRuntime`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r620603149
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
So... I will test full Gradle build (across all Scala versions and with all tests) with and without this clause.
If turns out that build results are comparable I will remove that clause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 edited a comment on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 edited a comment on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821837798
Digging through project history I managed to find some commits that are seems to be related (see the comment):
```
project(':streams') {
dependencies {
api project(':clients')
....
// testCompileOnly prevents streams from exporting a dependency on test-utils, which would cause a dependency cycle
testCompileOnly project(':streams:test-utils')
....
testRuntimeOnly project(':streams:test-utils')
}
....
}
```
@vvcephei contributed two PRs (#4821 and #4812) 3 years ago and these are related commits:
- **05. Apr 2018**. https://github.com/apache/kafka/commit/d5db4e9b8055536bdbb3b2d2308e36efa4abafcc
- **23. Apr 2018.** https://github.com/apache/kafka/commit/ed51b2cdf5bdac210a6904bead1a2ca6e8411406
Out of curiosity I tried to consolidate these dependency into one `testImplementation project(':streams:test-utils')` and here are the results:
- on top of trunk: build works fine ✔️
- on top of this PR: build fails ❌ (with identical `Circular dependency between the following tasks` error as above)
(...moving to phase 3...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 edited a comment on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)
Posted by GitBox <gi...@apache.org>.
dejan2609 edited a comment on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-821968689
> Could we reintroduce a configuration that behaves the same as `testRuntime` did before it was removed?
@ijuma Can you please expand on this ? Are you referring to a `testRuntime` mentioned in a `copyDependantLibs` task ?
https://github.com/apache/kafka/blob/2.8.0-rc2/build.gradle#L1472
```
project(':streams') {
...
tasks.create(name: "copyDependantLibs", type: Copy) {
from (configurations.testRuntime) {
...
}
...
}
```
~Or we now just brainstorm about Gradle behavior _**in-general**_ ?~
Thing is that this code was introduced back in the year 2015. (as a fresh/newly added code).
Also `testRuntime` configuration was added to an initial Gradle version: https://docs.gradle.org/1.0/userguide/artifact_dependencies_tutorial.html#configurations
I see that `tasks.create(name: "copyDependantLibs", type: Copy)` logic is applied to a dozen of submodules (15 in total, to be more precise). I didn't check, but it is safe to say that this is an prerequisite for `releaseTarGz` tasks (i.e. full distribution tasks).
I will try to compare other 14 (that work as expected) with this one (that fails).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#issuecomment-818929043
@ijuma I narrowed the gap and will post my findings here in a few days.... some more digging will be required in order to solve this (I will try to squeeze it asap).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dejan2609 commented on a change in pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration `testRuntime`
Posted by GitBox <gi...@apache.org>.
dejan2609 commented on a change in pull request #10466:
URL: https://github.com/apache/kafka/pull/10466#discussion_r620594758
##########
File path: build.gradle
##########
@@ -1491,13 +1491,14 @@ project(':streams') {
}
tasks.create(name: "copyDependantLibs", type: Copy) {
- from (configurations.testRuntime) {
Review comment:
One more note: @vvcephei didn't introduced/changed `copyDependentLibs` Gradle task. I just mentioned him above while I was investigating changes related to a `streams` sub-module dependencies block (at the time when I just started to look for an approach).
That specific Gradle task was introduced as a part of this (big) improvement: https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+processor+client
Related (pretty large) commit: https://github.com/apache/kafka/commit/263c10ab7c8e8fde9d3566bf59dccaa454ee2605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org