You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "dejan2609 (via GitHub)" <gi...@apache.org> on 2025/04/18 14:15:22 UTC

[PR] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

dejan2609 opened a new pull request, #19513:
URL: https://github.com/apache/kafka/pull/19513

   **JIRA ticket:** https://issues.apache.org/jira/browse/KAFKA-19174
   
   **Note:** at the moment Gradle 9 is available as a milestone version **_only_**.
   
    **Related links:**
   - https://gradle.org/whats-new/gradle-9
   - https://github.com/gradle/gradle/releases/tag/v9.0.0-M2
   - https://docs.gradle.org/9.0-milestone-2/release-notes.html
   - https://docs.gradle.org/9.0-milestone-2/userguide/userguide.html
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3236158518

   One (last ?) thing stands as a mystery for now: I just compared two CI builds (Kafka 4.1.0 RC3 vs this one) and same results (for `Parse Junit test` step *only*) yield different outcomes:
   - 4.1.0 RC3 build :white_check_mark: https://github.com/apache/kafka/actions/runs/17263427471/job/48991496402
   - this PR: :x: https://github.com/apache/kafka/actions/runs/17308744713/job/49138826668?pr=19513
   
   <img width="1427" height="865" alt="image" src="https://github.com/user-attachments/assets/450d06e8-8ffd-402a-a301-2dba808b57e5" />
   
   
   <img width="1436" height="864" alt="image" src="https://github.com/user-attachments/assets/8a21370c-f69e-4efc-b541-b4c826f534a2" />


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3160881920

   @dejan2609 any update?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3166859319

   @dejan2609 thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3214529467

   Ok, now some paths must he changed (naturally given a fact that distribution .tgz file paths must be changed).
   
   So it seems that I'm closing the gap on this (but I will temporarily move this PR back to a draft status).
   
   <img width="1409" height="338" alt="image" src="https://github.com/user-attachments/assets/15a07aa0-f894-43ac-9d75-c8ab664c5817" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3376482326

   Thank you for merging this @chia7712 ! It was quite a challenge to work on this, I will come up with a separate comment to credit all the people who were involved :slightly_smiling_face: 
   
   I created two more related JIRA tickets: 
   - KAFKA-19761 Reorder Gradle tasks (in order to bump Shadow plugin version)
   - KAFKA-19762 Turn on Gradle reproducible builds feature


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "Goooler (via GitHub)" <gi...@apache.org>.
Goooler commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365635913


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   This is caused by the wrong configuration introduced by #20547.
   
   https://github.com/Goooler/kafka/blob/d067c6c04089a3d24e1f72e6cb1b10b0d85f76da/build.gradle#L3353
   
   
   Reverted that PR and ran `gw :jmh-benchmarks:shadowJar` again, and the task passed.
   



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   This is caused by the wrong configuration introduced by #20547.
   
   https://github.com/apache/kafka/blob/d067c6c04089a3d24e1f72e6cb1b10b0d85f76da/build.gradle#L3353
   
   
   Reverted that PR and ran `gw :jmh-benchmarks:shadowJar` again, and the task passed.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2404971098


##########
wrapper.gradle:
##########
@@ -35,14 +35,12 @@ task bootstrapWrapper() {
         def wrapperBasePath = "\$APP_HOME/gradle/wrapper"
         def wrapperJarPath = wrapperBasePath + "/gradle-wrapper.jar"
 
-        // Add a trailing zero to the version if needed.

Review Comment:
   oh, I get your point that `versions.gradle` take over the job of passing "correct" number. That should work



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370893446

   > I left comments for both conversations: one for the preserveFileTimestamps = true and another for Gradle version trailing zero case.
   
   Excuse me, I don't see any comments.
   
   <img width="1674" height="765" alt="螢幕快照 2025-10-06 18-16-06" src="https://github.com/user-attachments/assets/1eaa9fdd-1117-4fb4-8d0b-e8aa469bdb11" />
   
   Could you share the conversation link with me? for example: https://github.com/apache/kafka/pull/19513#discussion_r2404940333


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370973819

   @chia7712 Related to `preserveFileTimestamps = true` :arrow_double_down: 
   ___
   Prologue (comment from back in August): https://github.com/apache/kafka/pull/19513#issuecomment-3217444662 
   
   Related links:
   - Gradle issue: https://github.com/gradle/gradle/issues/34643
   - Gradle DSL: https://docs.gradle.org/9.1.0/dsl/org.gradle.api.tasks.bundling.Tar.html#org.gradle.api.tasks.bundling.Tar:preserveFileTimestamps
    
   
   I just tested again, with the following procedure: comment out `preserveFileTimestamps = true` for tasks `:core:siteDocsTar` and `:core:releaseTarGz` and execute `./gradlew clean releaseTarGz -i`:
   
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -3 --oneline 
   4278e5621e (HEAD -> KAFKA-19174, origin/KAFKA-19174) KAFKA-19174 Gradle version upgrade (8 -->> 9); build logic is refactored
   9318bd3480 KAFKA-19591 solving issues with `gradlew` content: an old template version (i.e. file `unixStartScript.txt`) was being referenced
   d76442e5a6 (origin/trunk, origin/HEAD, trunk) MINOR: Fix a minor copy&paste typo in comments (#20634)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git diff
   diff --git a/build.gradle b/build.gradle
   index c3cc954e4b..8c7f1d19b4 100644
   --- a/build.gradle
   +++ b/build.gradle
   @@ -1222,7 +1222,7 @@ project(':core') {
        from project.file("$rootDir/docs")
        into 'site-docs'
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
      }
    
      tasks.create(name: "releaseTarGz", dependsOn: configurations.archives.artifacts, type: Tar) {
   @@ -1268,7 +1268,7 @@ project(':core') {
        from(project(':streams:examples').jar) { into("libs/") }
        from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
        filePermissions {
            user {
                read = true
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew clean releaseTarGz -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 4278e562) using Gradle 9.1.0, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 4 Kafka message JSON file(s).
   ...
   ```
   Unpack generated distributions:
   - cd core/build/distributions/
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT.tgz
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   
   **Folders show the wrong date: 02nd Jan 1970:** (with default `preserveFileTimestamps = false`) :arrow_down_small: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$ ls -la
   total 136280
   drwxrwxr-x 4 dejan dejan      4096 Oct  6 10:09 .
   drwxrwxr-x 9 dejan dejan      4096 Oct  6 10:05 ..
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 kafka_2.13-4.2.0-SNAPSHOT
   -rw-rw-r-- 1 dejan dejan   3754359 Oct  6 10:05 kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   -rw-rw-r-- 1 dejan dejan 135767889 Oct  6 10:06 kafka_2.13-4.2.0-SNAPSHOT.tgz
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 site-docs
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2406726608


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   Update: I just pushed a commit that uses the upgrade manual suggestion (mentioned above) for permissions  and goes even further with turning off the reproducible builds feature for Gradle 9 build. 
   
   I assume this is actually a good and safer approach: we can create a separate JIRA ticket and turn on the reproducible builds (at some point in the future and as an isolated change).
   
   Some usual testing yields expected results (file numbers match and cluster starts and sends/receives messages as expected).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2409468533


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   Marking this conversation as **_resolved_**



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2410378413


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   :information_source: Follow up JIRA ticket: 
   - KAFKA-19762 Turn on Gradle reproducible builds feature



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2407799307


##########
wrapper.gradle:
##########
@@ -35,14 +35,12 @@ task bootstrapWrapper() {
         def wrapperBasePath = "\$APP_HOME/gradle/wrapper"
         def wrapperJarPath = wrapperBasePath + "/gradle-wrapper.jar"
 
-        // Add a trailing zero to the version if needed.

Review Comment:
   Resolving conversation; we will keep this change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388830495

   > @dejan2609 I think the command is not running the tests. Normally we should see test execution output like
   
   Are you referring to the command(s) that I used, i.e.`./gradlew <command> -q` ? 
   It runs everything (including tests), but the output is silenced to show errors only (and it shows no errors because they are nowhere to be found).
   
   See here for more details: https://docs.gradle.org/current/userguide/command_line_interface.html#sec:command_line_logging
   ```
   -q, --quiet
   
       Log errors only.
   ```
   
   Now, even your output posted above shows no errors: `BUILD SUCCESSFUL in 1m 28s`
   
   Can't help you unless you file a JIRA ticket with the **exact** error scenario:
   - git commit where the issue occurs
   - stack trace (Gradle output)
   - details about your local box: Java version(s), Java vendor(s). operating system (Windows, Mac, Linux), etc.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3289592416

   Note: I will rebase onto trunk (once more)  in order to get same baseline for testing related JIRA ticket ([KAFKA-19707](https://issues.apache.org/jira/browse/KAFKA-19707)).
   
   This PR stays the same (equivalent) and it is ready for a review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3316119359

   Update: an additional commit has been pushed (to polish some things).
   
   I also did some testing (locally, on my laptop):
   1. `./gradlew :jmh-benchmarks:shadowJar` :white_check_mark: 
   2. Create a distribution and test it:
        - `./gradlew releaseTarGz`
        - unpack (with `tar xvfz`) **_/kafka/core/build/distributions/kafka_2.13-4.2.0-SNAPSHOT.tgz_**
        - test number of files: `find kafka_2.13-4.2.0-SNAPSHOT -type f | wc -l` -->> yields the same number of files as for the trunk distribution :white_check_mark: 
        - test like here: https://kafka.apache.org/quickstart -->> :white_check_mark: 
   
   ___
   Also, I revisited some previous shadow plugin related testing:
   - https://github.com/apache/kafka/pull/15532 
   - https://github.com/apache/kafka/pull/18018#issuecomment-2645879873
   
   **Testing procedure:** 
   - git checkout trunk
   - change `gradle.properties` version to `4.2.0.1-SNAPSHOT`
   - execute `./gradlew :clients:publishToMavenLocal -PskipSigning=true`
   - git reset --hard
   - git checkout KAFKA-19174
   - change `gradle.properties` version to `4.2.0.2-SNAPSHOT`
   - execute (again) `./gradlew :clients:publishToMavenLocal -PskipSigning=true`
   
   Test results:
   - compare two .pom file versions: 
   ```
   diff ~/.m2/repository/org/apache/kafka/kafka-clients/4.2.0.1-SNAPSHOT/kafka-clients-4.2.0.1-SNAPSHOT.pom ~/.m2/repository/org/apache/kafka/kafka-clients/4.2.0.2-SNAPSHOT/kafka-clients-4.2.0.2-SNAPSHOT.pom
   6c6
   <   <version>4.2.0.1-SNAPSHOT</version>
   ---
   >   <version>4.2.0.2-SNAPSHOT</version>
   ```
   - unzip both jar files: ~/.m2/repository/org/apache/kafka/kafka-clients/4.2.0.X-SNAPSHOT/kafka-clients-4.2.0.X-SNAPSHOT.jar: `both META-INF/MANIFEST.MF` files contain only one line: 
   ```
   Manifest-Version: 1.0
   ```
   - test dependencies in a real-life situation:
     - use POC (used previously): https://github.com/dejan2609/poc-kafka-18142.git
     - switch between versions: 
   ```
   dependencies {
       //implementation("org.apache.kafka:kafka-clients:4.2.0.1-SNAPSHOT")
       implementation("org.apache.kafka:kafka-clients:4.2.0.2-SNAPSHOT")
   }
   ```
   - create two files and compare them
     - `./gradlew dependencies > deps-trunk.txt`
     - `./gradlew dependencies > deps-PR.txt`
     - files are equivalent (i.e. Gradle dependencies trees match in everything except for the `kafka-clients` version(s))


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239858678

   Update: I will do multiple force-pushes today.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3146952127

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3200726747

   :warning: it seems that Gradle version 9 (9.0.0 **_and/or_** 9.1.0-rc-1) has some regression issues.
   Real world scenario for these issues (on a large Java monorepo) is described here:  https://github.com/gradle/gradle/issues/34691#issuecomment-3200324603


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2412531614


##########
gradlew:
##########
@@ -212,19 +199,31 @@ if "$cygwin" || "$msys" ; then
 fi
 
 
+
+# Loop in case we encounter an error.
+for attempt in 1 2 3; do
+  if [ ! -e "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" ]; then
+    if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v9.1.0/gradle/wrapper/gradle-wrapper.jar"; then
+      rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
+      # Pause for a bit before looping in case the server throttled us.
+      sleep 5
+      continue
+    fi
+  fi
+done
+
 # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
 DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
 
 # Collect all arguments for the java command:
-#   * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
+#   * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
 #     and any embedded shellness will be escaped.
 #   * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
 #     treated as '${Hostname}' itself on the command line.
 
 set -- \
         "-Dorg.gradle.appname=$APP_BASE_NAME" \
-        -classpath "$CLASSPATH" \
-        org.gradle.wrapper.GradleWrapperMain \
+        -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \

Review Comment:
   This change, introduced in gradle 8.14, seems to be a behavioral change. It causes a `no main manifest attribute` error if developers use the old wrapper. I will file a PR to revert to the previous behavior and open a Jira ticket for 5.0 to remove the workaround
   
   https://github.com/gradle/gradle/commit/5949f6029267fe02e712ff826eaf7c7d6181ff53
   https://github.com/gradle/gradle/commit/f51d28d7520a0cd04f4305576401f2d111a70ff3



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3251633067

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3283644497

   It seems that we have a solution now (only few tests are failing for Java 17 build):
   ``` 
   Found 1 test failures:
   FAILED ❌ RemoteLogManagerTest > "testCopyQuota(boolean).quotaExceeded=false"
   Found 2 flaky test failures:
   FLAKY ⚠️  TransactionsBounceTest > testWithGroupMetadata(String).groupProtocol=consumer
   FLAKY ⚠️  SaslPlaintextConsumerTest > testCoordinatorFailover(String).groupProtocol=consumer
   ```
   I will rebase this PR onto trunk.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3123836832

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3166457327

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3172338803

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3191091835

   One more thing: Gradle team has released 9.1 RC version only 12 days after 9.0 GA version (and typically it would take them more than 5 weeks to do the same thing in a similar situations). 
   (Now, I'm not saying that Gradle 9.0 is unstable, but I would rather wait for a while - just to be on a safe side.)
   ```
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v9.0.0
   Thu Jul 31 09:09:37 2025 +0000
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v9.1.0-RC1
   Tue Aug 12 22:55:39 2025 +0000
   user@hostname:~/gradle$
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v8.0.0
   Fri Feb 10 20:07:20 2023 +0100
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v8.1.0-RC1
   Sun Mar 19 10:30:28 2023 +0100
   user@hostname:~/gradle$
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v7.0.0
   Fri Apr 9 14:33:50 2021 -0400
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v7.1.0-RC1
   Fri May 28 14:36:44 2021 -0400
   user@hostname:~/gradle$
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v6.0.0
   Fri Nov 8 14:19:38 2019 +0100
   user@hostname:~/gradle$ git for-each-ref --format="%(creatordate)" refs/tags/v6.1.0-RC1
   Thu Dec 19 17:20:32 2019 +0100
   user@hostname:~/gradle$
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3235106658

   Update: yet another commit is added (that solves this JIRA sub-ticket):
   - https://issues.apache.org/jira/browse/KAFKA-16801 Streams upgrade :test target doesn't find any junit tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3213881201

   Update 2: I will try to drop new submodule `:distribution' (i.e. just to change `:core` module Gradle build. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239507882

   Ok, I managed to pin-point git commit that makes flaky and new tests to fail.
   
   Will trample revert commits and put this to a draft.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239262474

   Update: rebased/force-pushed; I'm trying to resolve issues with failed results parsing for JUnit tests (for a new/flaky sets of tests only).
   
   _Note: I may have to open a new PR, this one is becoming quite long due to my debugging (also: GitHub Actions bot is constantly reapplying `needs-attention` label)._ 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3355382868

   > @dejan2609 I open the https://issues.apache.org/jira/browse/KAFKA-19750 to describe the root cause of `releaseTarGz`. I think the simple way to fix it is to remove `project(':tools:tools-api')` from `releaseTarGz`, since it is already in runtime path of `core` module. Also, it would be helpful to leave comment on the `implementation project(':tools:tools-api')` to remind developers to add `project(':tools:tools-api')` back if `project(':tools:tools-api')` is removed from the runtime path
   
   Got it. Should I add a commit that resolves that issue here or you want to solve it separately ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388588662

   > @dejan2609 Thanks for working in this. It looks like a few custom tasks (`./gradlew unitTest, ./gradlew integrationTest`) aren’t working as expected after the upgrade.
   > 
   > cc @chia7712
   
   Ok, let me check that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388723818

   @omkreddy  The same goes for a current trunk state (no errors whatsoever):
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ date
   Fri 10 Oct 09:43:26 CEST 2025
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -1 --oneline 
   1e95d04639 (HEAD -> trunk, upstream/trunk, upstream/HEAD, origin/trunk, origin/HEAD) KAFKA-19747: Update ClientTelemetryReporter telemetry push error handling (#20661)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version 
   openjdk version "17.0.16" 2025-07-15
   OpenJDK Runtime Environment Temurin-17.0.16+8 (build 17.0.16+8)
   OpenJDK 64-Bit Server VM Temurin-17.0.16+8 (build 17.0.16+8, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew unitTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 1e95d046) using Gradle 9.1.0, Java 17 and Scala 2.13.17
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 5 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   MessageGenerator: processed 2 Kafka message JSON file(s).
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 26 Kafka message JSON file(s).
   MessageGenerator: processed 44 Kafka message JSON file(s).
   MessageGenerator: processed 197 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   Note: Some input files use or override a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   Note: /home/dejan/kafka/metadata/src/test/java/org/apache/kafka/controller/metrics/QuorumControllerMetricsTest.java uses unchecked or unsafe operations.
   Note: Recompile with -Xlint:unchecked for details.
   Note: /home/dejan/kafka/server/src/test/java/org/apache/kafka/server/AssignmentsManagerTest.java uses or overrides a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew integrationTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 1e95d046) using Gradle 9.1.0, Java 17 and Scala 2.13.17
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ sdk use java 25-tem
   
   Using java version 25-tem in this shell.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version 
   openjdk version "25" 2025-09-16 LTS
   OpenJDK Runtime Environment Temurin-25+36 (build 25+36-LTS)
   OpenJDK 64-Bit Server VM Temurin-25+36 (build 25+36-LTS, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew unitTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 1e95d046) using Gradle 9.1.0, Java 25 and Scala 2.13.17
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 2 Kafka message JSON file(s).
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   MessageGenerator: processed 26 Kafka message JSON file(s).
   MessageGenerator: processed 5 Kafka message JSON file(s).
   MessageGenerator: processed 44 Kafka message JSON file(s).
   MessageGenerator: processed 197 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
   WARNING: sun.misc.Unsafe::arrayBaseOffset has been called by sbt.internal.shaded.com.google.protobuf.UnsafeUtil$MemoryAccessor (file:/home/dejan/.gradle/caches/modules-2/files-2.1/org.scala-sbt/zinc-persist-core-assembly/1.11.0/79a5ffae430cc389d51b03650efb7c2253aa9f30/zinc-persist-core-assembly-1.11.0.jar)
   WARNING: Please consider reporting this to the maintainers of class sbt.internal.shaded.com.google.protobuf.UnsafeUtil$MemoryAccessor
   WARNING: sun.misc.Unsafe::arrayBaseOffset will be removed in a future release
   Note: Some input files use or override a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   Note: /home/dejan/kafka/metadata/src/test/java/org/apache/kafka/controller/metrics/QuorumControllerMetricsTest.java uses unchecked or unsafe operations.
   Note: Recompile with -Xlint:unchecked for details.
   Note: /home/dejan/kafka/server/src/test/java/org/apache/kafka/server/AssignmentsManagerTest.java uses or overrides a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew integrationTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 1e95d046) using Gradle 9.1.0, Java 25 and Scala 2.13.17
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ date
   Fri 10 Oct 09:53:40 CEST 2025
   dejan@dejan-HP-ProBook-450-G7:~/kafka$
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3355463860

   > Got it. Should I add a commit that resolves that issue here or you want to solve it separately ?
   
   What do you think of this approache? Is it a viable replacement for the "distribution" module? If yes, it will be great this PR include the approach. If not, we could have a separate PR for that to fix the warnings for other active branches.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3218117616

   ### Yet another update: it seems that we are good to go !!! :fireworks: :rocket: 
   
   Hopefully last issue was related to permission/execution rights; related links: 
   - https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#removal_of_unix_mode_based_file_permissions  
   - https://docs.gradle.org/9.0.0/userguide/userguide_single.html#using_copyspec_dirpermissions 
   - https://github.com/gradle/gradle/pull/33984
   
   I finally managed to test a new  (i.e. compiled via Gradle 9) kafka distribution :smiley: 
   
   **Steps to test:** 
   - use this PR and execute `./gradlew clean releaseTarGz`
   - find distribution and unpack it: 
      - `cd distribution/build/distributions`
      - `tar xvfz kafka_2.13-4.2.0-SNAPSHOT.tgz`
   - start a cluster, create some topic, write to topic, read events, etc. as described here: here: https://kafka.apache.org/quickstart 
   - everything looks fine ✅
   
   Note: I just need to add related permissions code, force-push and we are all set ! :100:


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3213821218

   ### Update:
   - distribution build works as expected: `./gradlew clean releaseTarGz -i` :white_check_mark:
   - but CI build (rat report) fails :x: 
   
   I opted to introduce another (empty) module `:distribution` in order to extract distribution related tasks out of `:core` module (and hence I had to add some git file placeholder to that new module).
   Will try to fix this somehow.
   
    ``` 
   Run python .github/scripts/rat.py
   Found 1 Rat reports
   Read env GITHUB_WORKSPACE: /home/runner/work/kafka/kafka
   Parsing Rat report file: build/rat/rat-report.xml
   6840 approved licenses
   1 unapproved licenses
   Files with unapproved licenses:
   Notice: File with unapproved license: distribution/git-placeholder-file
   Error: Process completed with exit code 1.
   Run python .github/scripts/rat.py
   Found 1 Rat reports
   Read env GITHUB_WORKSPACE: /home/runner/work/kafka/kafka
   Parsing Rat report file: build/rat/rat-report.xml
   6840 approved licenses
   1 unapproved licenses
   Files with unapproved licenses:
   Notice: File with unapproved license: distribution/git-placeholder-file
   Error: Process completed with exit code 1.
   ```
      - <img width="1283" height="958" alt="image" src="https://github.com/user-attachments/assets/956e0b88-60bc-4d64-b88d-b1547b494af7" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365582134


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   If the shadow upgrade is unnecessary, we could exclude the change from this PR. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2378273533


##########
gradlew:
##########
@@ -84,7 +86,7 @@ done
 # shellcheck disable=SC2034
 APP_BASE_NAME=${0##*/}
 # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
-APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
+APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit

Review Comment:
   Excuse me, why to have those changes? Are those changed generated by gradle 9?



##########
build.gradle:
##########
@@ -3905,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {
+  base {
+    archivesName = "kafka_${versions.baseScala}"
+  }
+
+  configurations {
+    includeIntoDistribution
+  }
+
+  dependencies {
+    includeIntoDistribution project(path: ':core', configuration: 'releaseOnly')
+    includeIntoDistribution project(path: ':core', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':trogdor', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':shell', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:api', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:runtime', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:transforms', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:json', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:file', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:basic-auth-extension', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror-client', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:streams-scala', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:test-utils', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:examples', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools:tools-api', configuration: 'resolvedRuntimeClasspath')
+  }
+
+  task siteDocsTar(dependsOn: [':core:genProtocolErrorDocs', ':core:genProtocolTypesDocs', ':core:genProtocolApiKeyDocs', ':core:genProtocolMessageDocs',
+                               ':core:genAdminClientConfigDocs', ':core:genProducerConfigDocs', ':core:genConsumerConfigDocs',
+                               ':core:genKafkaConfigDocs', ':core:genTopicConfigDocs', ':core:genGroupConfigDocs',
+                               ':connect:runtime:genConnectConfigDocs', ':connect:runtime:genConnectTransformationDocs',
+                               ':connect:runtime:genConnectPredicateDocs',
+                               ':connect:runtime:genSinkConnectorConfigDocs', ':connect:runtime:genSourceConnectorConfigDocs',
+                               ':streams:genStreamsConfigDocs', ':core:genConsumerMetricsDocs', ':core:genProducerMetricsDocs',
+                               ':connect:runtime:genConnectMetricsDocs', ':connect:runtime:genConnectOpenAPIDocs',
+                               ':connect:mirror:genMirrorSourceConfigDocs', ':connect:mirror:genMirrorCheckpointConfigDocs',
+                               ':connect:mirror:genMirrorHeartbeatConfigDocs', ':connect:mirror:genMirrorConnectorConfigDocs',
+                               ':storage:genRemoteLogManagerConfigDoc', ':storage:genRemoteLogMetadataManagerConfigDoc'], type: Tar) {
+    archiveClassifier = 'site-docs'
+    compression = Compression.GZIP
+    from project.file("$rootDir/docs")
+    into 'site-docs'
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')
+    duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+  }
+
+  tasks.create(name: "releaseTarGz", dependsOn: project(':core').configurations.archives.artifacts, type: Tar) {
+    into "kafka_${versions.baseScala}-${archiveVersion.get()}"
+    compression = Compression.GZIP
+    from(project.file("$rootDir/bin")) { into "bin/" }
+    from(project.file("$rootDir/config")) { into "config/" }
+    from(project.file("$rootDir/licenses")) { into "licenses/" }
+    from "$rootDir/LICENSE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from "$rootDir/NOTICE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from(project.siteDocsTar) { into("site-docs/") }
+    from(configurations.includeIntoDistribution) { into("libs/") }
+    from(project(':core').configurations.archives.artifacts.files) { into("libs/") }
+    from(project(':tools').jar) { into("libs/") }
+    from(project(':trogdor').jar) { into("libs/") }
+    from(project(':shell').jar) { into("libs/") }
+    from(project(':connect:api').jar) { into("libs/") }
+    from(project(':connect:runtime').jar) { into("libs/") }
+    from(project(':connect:transforms').jar) { into("libs/") }
+    from(project(':connect:json').jar) { into("libs/") }
+    from(project(':connect:file').jar) { into("libs/") }
+    from(project(':connect:basic-auth-extension').jar) { into("libs/") }
+    from(project(':connect:mirror').jar) { into("libs/") }
+    from(project(':connect:mirror-client').jar) { into("libs/") }
+    from(project(':streams').jar) { into("libs/") }
+    from(project(':streams:streams-scala').jar) { into("libs/") }
+    from(project(':streams:test-utils').jar) { into("libs/") }
+    from(project(':streams:examples').jar) { into("libs/") }
+    from(project(':tools:tools-api').jar) { into("libs/") }
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')

Review Comment:
   Add a comment explaining that the value of `destination` is kept for compatibility



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3333659984

   > @dejan2609 It would be helpful to understand the reasoning behind the changes in this PR, since introducing unnecessary modifications is usually not a good idea. That’s why I left some questions on parts where I couldn’t see the motivation—for example: [#19513 (comment)](https://github.com/apache/kafka/pull/19513#discussion_r2378273533)
   > 
   > Upgrading Gradle is an important task for Kafka, so I’d appreciate your help in clarifying the intent of this PR. Thanks!
   
   Yes, I understand the gravity of the situation. It goes without saying that these types of upgrades must be done with high caution. I have prior experience with upgrading Gradle major versions (from Gradle 6 to 7 and then from 7 to 8) and they were complicated enough (but both of those upgrades pale in comparison to this one).
   
   I left answers to both comments above, but I will sum it up here also:
   - first commit https://github.com/apache/kafka/commit/79f04159d96365d7b9c43c409134b1b1710e8c27 resolves bug KAFKA-19591 (on Gradle 8 level); objectively: this commit is a prerequisite for a Gradle 8 to 9 upgrade
   - all other commits are exclusively related to Gradle 8 to 9 upgrade
   
   I opted to keep separate commits for every phase just for the sake of clarity (and that was a good decision, I guess).
   
   Options to proceed: 
   - option [1]:
     - merge commit https://github.com/apache/kafka/commit/79f04159d96365d7b9c43c409134b1b1710e8c27 (KAFKA-19591) into trunk separately 
     - squash all other commits (related to Gradle 8 -->> 9 upgrade) into one and, after few days, merge that commit into trunk
   - option [2]:
     - put aside commit https://github.com/apache/kafka/commit/79f04159d96365d7b9c43c409134b1b1710e8c27 (KAFKA-19591) and try to concentrate only on Gradle 8 -->> 9 upgrade 
   
   Note: not sure about the possibility of using option [2]: from where I see it https://github.com/apache/kafka/commit/79f04159d96365d7b9c43c409134b1b1710e8c27 (KAFKA-19591) is a must (either separately or as a part of upgrade).
   
   Let me know what you think.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3177573040

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2336213537


##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   Done, will push rebased commit(s) with distributions created in old (i.e. `:core`) paths and then will return here for a full explanation.



##########
build.gradle:
##########
@@ -330,6 +329,19 @@ subprojects {
     tasks.register('uploadArchives').configure { dependsOn(publish) }
   }
 
+    def  submodulesToIncludeInDistribution = ['core', 'tools', 'trogdor', 'shell', 'api', 'runtime', 'transforms',
+                                              'json', 'file', 'basic-auth-extension', 'mirror', 'mirror-client',
+                                              'streams', 'streams-scala', 'test-utils', 'examples', 'tools-api']
+

Review Comment:
   Surprisingly enough: `project.name` for a leaf submodules resolves to a submodule name only, but it seems that we are on a safe side (i.e. submodule leaf names are unique):
   
   https://stackoverflow.com/questions/76320016/is-it-possible-for-2-submodules-to-have-different-paths-but-identical-names/76342122#76342122 



##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   Good news @chia7712 ! I have a solution that  keeps distribution paths as-is. 
   Just needs some testing/double-checking, hopefully will push it tomorrow.



##########
gradle/dependencies.gradle:
##########
@@ -119,13 +119,13 @@ versions += [
   // When updating the scalafmt version please also update the version field in checkstyle/.scalafmt.conf. scalafmt now
   // has the version field as mandatory in its configuration, see
   // https://github.com/scalameta/scalafmt/releases/tag/v3.1.0.
-  scalafmt: "3.7.14",
-  scoverage: "2.0.11",
+  scalafmt: "3.9.9",

Review Comment:
   No, not at all. 
   Just to more precise: both Scala Coverage and Scala formatter are not required for Gradle upgrade and can be left out.



##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   Done, dummy folder `distribution` is removed.



##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   @chia7712 here it is: #20566 



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   Resolving conversation.



##########
gradlew:
##########
@@ -84,7 +86,7 @@ done
 # shellcheck disable=SC2034
 APP_BASE_NAME=${0##*/}
 # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
-APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
+APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit

Review Comment:
   No, they were not generated by Gradle 9 (but here is the twist: they are generated by Gradle 8).
   
   This change resolves the `gradlew` issue at the Gradle 8 level (basically: `gradlew` content has diverged from the proper content for quite some time, and this commit solves those issues).
   
   Strangely enough: Gradle builds via Gradle wrapper still work (obviously) in trunk (with Gradle 8) even without this change, but it is/was only a question of time when this would escalate (and while working on Gradle upgrade from 8 to 9 I managed to discover it and solve it all along).
   
   That's why I included this commit in this PR as the first one (in front of Gradle 8.14.3 to 9.0.0 upgrade, that is):
   - related Jira ticket: KAFKA-19591
   - git commit (that can be merged into trunk separately, i.e. in isolation of Gradle 8-->> 9 upgrade): https://github.com/apache/kafka/pull/19513/commits/79f04159d96365d7b9c43c409134b1b1710e8c27 



##########
build.gradle:
##########
@@ -3905,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {
+  base {
+    archivesName = "kafka_${versions.baseScala}"
+  }
+
+  configurations {
+    includeIntoDistribution
+  }
+
+  dependencies {
+    includeIntoDistribution project(path: ':core', configuration: 'releaseOnly')
+    includeIntoDistribution project(path: ':core', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':trogdor', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':shell', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:api', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:runtime', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:transforms', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:json', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:file', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:basic-auth-extension', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror-client', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:streams-scala', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:test-utils', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:examples', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools:tools-api', configuration: 'resolvedRuntimeClasspath')
+  }
+
+  task siteDocsTar(dependsOn: [':core:genProtocolErrorDocs', ':core:genProtocolTypesDocs', ':core:genProtocolApiKeyDocs', ':core:genProtocolMessageDocs',
+                               ':core:genAdminClientConfigDocs', ':core:genProducerConfigDocs', ':core:genConsumerConfigDocs',
+                               ':core:genKafkaConfigDocs', ':core:genTopicConfigDocs', ':core:genGroupConfigDocs',
+                               ':connect:runtime:genConnectConfigDocs', ':connect:runtime:genConnectTransformationDocs',
+                               ':connect:runtime:genConnectPredicateDocs',
+                               ':connect:runtime:genSinkConnectorConfigDocs', ':connect:runtime:genSourceConnectorConfigDocs',
+                               ':streams:genStreamsConfigDocs', ':core:genConsumerMetricsDocs', ':core:genProducerMetricsDocs',
+                               ':connect:runtime:genConnectMetricsDocs', ':connect:runtime:genConnectOpenAPIDocs',
+                               ':connect:mirror:genMirrorSourceConfigDocs', ':connect:mirror:genMirrorCheckpointConfigDocs',
+                               ':connect:mirror:genMirrorHeartbeatConfigDocs', ':connect:mirror:genMirrorConnectorConfigDocs',
+                               ':storage:genRemoteLogManagerConfigDoc', ':storage:genRemoteLogMetadataManagerConfigDoc'], type: Tar) {
+    archiveClassifier = 'site-docs'
+    compression = Compression.GZIP
+    from project.file("$rootDir/docs")
+    into 'site-docs'
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')
+    duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+  }
+
+  tasks.create(name: "releaseTarGz", dependsOn: project(':core').configurations.archives.artifacts, type: Tar) {
+    into "kafka_${versions.baseScala}-${archiveVersion.get()}"
+    compression = Compression.GZIP
+    from(project.file("$rootDir/bin")) { into "bin/" }
+    from(project.file("$rootDir/config")) { into "config/" }
+    from(project.file("$rootDir/licenses")) { into "licenses/" }
+    from "$rootDir/LICENSE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from "$rootDir/NOTICE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from(project.siteDocsTar) { into("site-docs/") }
+    from(configurations.includeIntoDistribution) { into("libs/") }
+    from(project(':core').configurations.archives.artifacts.files) { into("libs/") }
+    from(project(':tools').jar) { into("libs/") }
+    from(project(':trogdor').jar) { into("libs/") }
+    from(project(':shell').jar) { into("libs/") }
+    from(project(':connect:api').jar) { into("libs/") }
+    from(project(':connect:runtime').jar) { into("libs/") }
+    from(project(':connect:transforms').jar) { into("libs/") }
+    from(project(':connect:json').jar) { into("libs/") }
+    from(project(':connect:file').jar) { into("libs/") }
+    from(project(':connect:basic-auth-extension').jar) { into("libs/") }
+    from(project(':connect:mirror').jar) { into("libs/") }
+    from(project(':connect:mirror-client').jar) { into("libs/") }
+    from(project(':streams').jar) { into("libs/") }
+    from(project(':streams:streams-scala').jar) { into("libs/") }
+    from(project(':streams:test-utils').jar) { into("libs/") }
+    from(project(':streams:examples').jar) { into("libs/") }
+    from(project(':tools:tools-api').jar) { into("libs/") }
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')

Review Comment:
   Resolved.



##########
gradle/dependencies.gradle:
##########
@@ -61,8 +61,8 @@ versions += [
   commonsLang: "3.18.0",
   commonsValidator: "1.9.0",
   classgraph: "4.8.179",
-  gradle: "8.14.3",
-  grgit: "4.1.1",
+  gradle: "9.1.0",
+  grgit: "5.3.0",

Review Comment:
   Done, commit is pushed.



##########
gradle/dependencies.gradle:
##########
@@ -61,8 +61,8 @@ versions += [
   commonsLang: "3.18.0",
   commonsValidator: "1.9.0",
   classgraph: "4.8.179",
-  gradle: "8.14.3",
-  grgit: "4.1.1",
+  gradle: "9.1.0",
+  grgit: "5.3.0",

Review Comment:
   No, the update for `grgit`is certainly not required here. I will remove that and push is separately (as a MINOR PR).



##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   So, Gradle 9 is very strict with submodule boundaries and hence simple version upgrade (from 8.14.3 to 9.0.0 for Gradle and from 8.3.6 to 9.0.2 for shadow plugin) is not enough because `./gradlew releaseTarGz` yields an error: 
   ```
   > Task :core:releaseTarGz FAILED
   
   [Incubating] Problems report is available at: file:///home/dejan/kafka/build/reports/problems/problems-report.html
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   Execution failed for task ':core:releaseTarGz'.
   > Resolution of the configuration ':tools:tools-api:runtimeClasspath' was attempted without an exclusive lock. This is unsafe and not allowed.
   
   * Try:
   > For more information, please refer to https://docs.gradle.org/9.0.0/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors in the Gradle documentation.
   ```
   
   So, in order to bypass this I opted to add these things in order to trick Gradle 9 (so to say):
   - (dummy) submodule `:distribution` and 
   - resolvable configuration `resolvedRuntimeClasspath` for required submodules
   
   Initial solution included changed paths for generated kafka*.tgz files, but now I managed to put them back into a `:core` submodule.



##########
gradle/dependencies.gradle:
##########
@@ -119,13 +119,13 @@ versions += [
   // When updating the scalafmt version please also update the version field in checkstyle/.scalafmt.conf. scalafmt now
   // has the version field as mandatory in its configuration, see
   // https://github.com/scalameta/scalafmt/releases/tag/v3.1.0.
-  scalafmt: "3.7.14",
-  scoverage: "2.0.11",
+  scalafmt: "3.9.9",

Review Comment:
   Resolving conversation.



##########
gradle/dependencies.gradle:
##########
@@ -119,13 +119,13 @@ versions += [
   // When updating the scalafmt version please also update the version field in checkstyle/.scalafmt.conf. scalafmt now
   // has the version field as mandatory in its configuration, see
   // https://github.com/scalameta/scalafmt/releases/tag/v3.1.0.
-  scalafmt: "3.7.14",
-  scoverage: "2.0.11",
+  scalafmt: "3.9.9",

Review Comment:
   Done, removed in rebased commits.



##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   Resolving conversation (paths for distribution .tgz files are intact). 



##########
.github/actions/setup-gradle/action.yml:
##########
@@ -42,7 +42,7 @@ runs:
         distribution: temurin
         java-version: ${{ inputs.java-version }}
     - name: Setup Gradle
-      uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0

Review Comment:
   Done, github action version is changed as suggested.



##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   I may have some good news: it seems that we can avoid this folder :crossed_fingers: 
   
   While it's true that Gradle 9+ fails with an error if subprojects are missing their own folder (see below) that behavior can be easily avoided: https://docs.gradle.org/9.1.0/userguide/multi_project_builds.html#include_existing_projects_only
   
   Will commit in no time.
   
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -2 --oneline 
   218f5a941a (HEAD -> KAFKA-19174, origin/KAFKA-19174) KAFKA-19174 Gradle version upgrade: 9.0.0 -->> 9.1.0
   f0946dbc6a KAFKA-19654 do not fail Github Actions flaky/new build matrix build when no test are discovered (i.e. solution for KAFKA-16801 is expanded)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git status 
   On branch KAFKA-19174
   Your branch is up-to-date with 'origin/KAFKA-19174'.
   
   Changes to be committed:
     (use "git restore --staged <file>..." to unstage)
           deleted:    distribution/.gitkeep
   
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew clean releaseTarGz -q
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   Configuring project ':distribution' without an existing directory is not allowed. The configured projectDirectory '/home/dejan/kafka/distribution' does not exist, can't be written to or is not a directory.
   
   * Try:
   > Make sure the project directory exists and is writable.
   
   BUILD FAILED in 1s
   dejan@dejan-HP-ProBook-450-G7:~/kafka$
   ```
   
   
   
   



##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   On the other hand: Gradle submodule `distribution` is required for build to work with Gradle 9. 
   
   I will provide an in-depth explanation over the weekend.



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   Hi @Goooler
   
   git [commit](https://github.com/apache/kafka/commit/d067c6c04089a3d24e1f72e6cb1b10b0d85f76da) you are referring to was merged into trunk yesterday, while this PR is not synced with trunk for at least 5 days (and tests for https://github.com/GradleUp/shadow/issues/1769 were done locally, not on Github Actions CI - don't ask me why that specific case is not a part of CI test, that is a different pair of shoes).
   
   <img width="1024" height="454" alt="image" src="https://github.com/user-attachments/assets/1baf67e2-27fc-4dbb-8a16-c6efa45fe931" />
   
   So I can't revert that commit for sure in my PR (mind you: I'm not judging  your assessment about issues that specific commit may or may not impose, just stating the obvious from a git revert POV).
   
   Furthermore: Apache Kafka Gradle build is massive/overwhelming/overzealous, and working on it takes a massive toll on members/maintainers (and simple contributors like me share that sentiment).
   
   Due to Gradle build being so complicated, it is really hard to upgrade Gradle versions.
   I do believe you that Shadow plugin 8.3.6 is compatible in (any other ?) Gradle 9 setup but here that is not the case (build doesn't just breaks with error, it bends space-time, so to say).
   That is why I included (among all other changes related to Gradle 9), upgrade for Shadow plugin to version 9 and things were working just fine until we discovered some corner case (as it turns out: on Apache Kafka side) that is not covered via CI.
   
   I thought that it was a corner case on Shadow plugin side and hence I created an issue. It seems I was wrong and I want to make amends (like a real one, not in Git :cowboy_hat_face:).
   
   I can only imagine how complicated things can be on a core side when this implementation stuff can be so intermingled and sketchy.
   
   Having said all of this: I will rewind all of this and try to come up with a plan how/where to proceed.  
   
   Peace be with you.



##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   Marking this as resolved.



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   I'll give it a try.



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   Nice catch, let me try to address this.



##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   Yes, this is a sensitive issue.
   
   I may have an alternative way (that keeps paths as-is); will try to provide different approach and explain rationale for both solutions.



##########
build.gradle:
##########
@@ -3905,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {
+  base {
+    archivesName = "kafka_${versions.baseScala}"
+  }
+
+  configurations {
+    includeIntoDistribution
+  }
+
+  dependencies {
+    includeIntoDistribution project(path: ':core', configuration: 'releaseOnly')
+    includeIntoDistribution project(path: ':core', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':trogdor', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':shell', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:api', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:runtime', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:transforms', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:json', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:file', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:basic-auth-extension', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':connect:mirror-client', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:streams-scala', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:test-utils', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':streams:examples', configuration: 'resolvedRuntimeClasspath')
+    includeIntoDistribution project(path: ':tools:tools-api', configuration: 'resolvedRuntimeClasspath')
+  }
+
+  task siteDocsTar(dependsOn: [':core:genProtocolErrorDocs', ':core:genProtocolTypesDocs', ':core:genProtocolApiKeyDocs', ':core:genProtocolMessageDocs',
+                               ':core:genAdminClientConfigDocs', ':core:genProducerConfigDocs', ':core:genConsumerConfigDocs',
+                               ':core:genKafkaConfigDocs', ':core:genTopicConfigDocs', ':core:genGroupConfigDocs',
+                               ':connect:runtime:genConnectConfigDocs', ':connect:runtime:genConnectTransformationDocs',
+                               ':connect:runtime:genConnectPredicateDocs',
+                               ':connect:runtime:genSinkConnectorConfigDocs', ':connect:runtime:genSourceConnectorConfigDocs',
+                               ':streams:genStreamsConfigDocs', ':core:genConsumerMetricsDocs', ':core:genProducerMetricsDocs',
+                               ':connect:runtime:genConnectMetricsDocs', ':connect:runtime:genConnectOpenAPIDocs',
+                               ':connect:mirror:genMirrorSourceConfigDocs', ':connect:mirror:genMirrorCheckpointConfigDocs',
+                               ':connect:mirror:genMirrorHeartbeatConfigDocs', ':connect:mirror:genMirrorConnectorConfigDocs',
+                               ':storage:genRemoteLogManagerConfigDoc', ':storage:genRemoteLogMetadataManagerConfigDoc'], type: Tar) {
+    archiveClassifier = 'site-docs'
+    compression = Compression.GZIP
+    from project.file("$rootDir/docs")
+    into 'site-docs'
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')
+    duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+  }
+
+  tasks.create(name: "releaseTarGz", dependsOn: project(':core').configurations.archives.artifacts, type: Tar) {
+    into "kafka_${versions.baseScala}-${archiveVersion.get()}"
+    compression = Compression.GZIP
+    from(project.file("$rootDir/bin")) { into "bin/" }
+    from(project.file("$rootDir/config")) { into "config/" }
+    from(project.file("$rootDir/licenses")) { into "licenses/" }
+    from "$rootDir/LICENSE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from "$rootDir/NOTICE-binary" rename {String filename -> filename.replace("-binary", "")}
+    from(project.siteDocsTar) { into("site-docs/") }
+    from(configurations.includeIntoDistribution) { into("libs/") }
+    from(project(':core').configurations.archives.artifacts.files) { into("libs/") }
+    from(project(':tools').jar) { into("libs/") }
+    from(project(':trogdor').jar) { into("libs/") }
+    from(project(':shell').jar) { into("libs/") }
+    from(project(':connect:api').jar) { into("libs/") }
+    from(project(':connect:runtime').jar) { into("libs/") }
+    from(project(':connect:transforms').jar) { into("libs/") }
+    from(project(':connect:json').jar) { into("libs/") }
+    from(project(':connect:file').jar) { into("libs/") }
+    from(project(':connect:basic-auth-extension').jar) { into("libs/") }
+    from(project(':connect:mirror').jar) { into("libs/") }
+    from(project(':connect:mirror-client').jar) { into("libs/") }
+    from(project(':streams').jar) { into("libs/") }
+    from(project(':streams:streams-scala').jar) { into("libs/") }
+    from(project(':streams:test-utils').jar) { into("libs/") }
+    from(project(':streams:examples').jar) { into("libs/") }
+    from(project(':tools:tools-api').jar) { into("libs/") }
+    destinationDirectory = project(':core').layout.buildDirectory.dir('distributions')

Review Comment:
   Done, commit (small comment) is pushed.



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   Resolving conversation, action points from above will be added into JIRA ticket(s).



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   It seems like a shadow plugin version 8.3.9 **_is_** a way to go :rocket: 
   
   Action points for me:
   - update that closed shadow issue for future readers (it would be much easier to sum it up there - this PR is packed with comments) 
   - Create a separate Kafka jira ticket for a Shadow plugin upgrade (to version 9+, at some point in the future)
   - rebase this PR onto Kafka trunk branch and monitor how Gradle build behaves after https://github.com/apache/kafka/commit/d067c6c04089a3d24e1f72e6cb1b10b0d85f76da commit is included 
   - explain to @chia7712 why the new `:distribution` submodule is required (for Gradle build to work)
   
   Last but not least: Thank you for helping us out with this @Goooler.



##########
gradlew:
##########
@@ -84,7 +86,7 @@ done
 # shellcheck disable=SC2034
 APP_BASE_NAME=${0##*/}
 # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
-APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
+APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit

Review Comment:
   Note: explanation was far too long for conversation so I added multiple comments below. 
   I will keep this conversation open.



##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true

Review Comment:
   Prologue (comment from back in August): https://github.com/apache/kafka/pull/19513#issuecomment-3217444662 
   
   Related links:
   - Gradle issue: https://github.com/gradle/gradle/issues/34643
   - Gradle DSL: https://docs.gradle.org/9.1.0/dsl/org.gradle.api.tasks.bundling.Tar.html#org.gradle.api.tasks.bundling.Tar:preserveFileTimestamps
    
   
   I just tested again, with the following procedure: comment out `preserveFileTimestamps = true` for tasks `:core:siteDocsTar` and `:core:releaseTarGz` and execute `./gradlew clean releaseTarGz -i`:
   
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -3 --oneline 
   4278e5621e (HEAD -> KAFKA-19174, origin/KAFKA-19174) KAFKA-19174 Gradle version upgrade (8 -->> 9); build logic is refactored
   9318bd3480 KAFKA-19591 solving issues with `gradlew` content: an old template version (i.e. file `unixStartScript.txt`) was being referenced
   d76442e5a6 (origin/trunk, origin/HEAD, trunk) MINOR: Fix a minor copy&paste typo in comments (#20634)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git diff
   diff --git a/build.gradle b/build.gradle
   index c3cc954e4b..8c7f1d19b4 100644
   --- a/build.gradle
   +++ b/build.gradle
   @@ -1222,7 +1222,7 @@ project(':core') {
        from project.file("$rootDir/docs")
        into 'site-docs'
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
      }
    
      tasks.create(name: "releaseTarGz", dependsOn: configurations.archives.artifacts, type: Tar) {
   @@ -1268,7 +1268,7 @@ project(':core') {
        from(project(':streams:examples').jar) { into("libs/") }
        from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
        filePermissions {
            user {
                read = true
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew clean releaseTarGz -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 4278e562) using Gradle 9.1.0, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 4 Kafka message JSON file(s).
   ...
   ```
   Unpack generated distributions:
   - cd core/build/distributions/
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT.tgz
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   
   **Folders show the wrong date: 02nd Jan 1970:** (with default `preserveFileTimestamps = false`) :arrow_down_small: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$ ls -la
   total 136280
   drwxrwxr-x 4 dejan dejan      4096 Oct  6 10:09 .
   drwxrwxr-x 9 dejan dejan      4096 Oct  6 10:05 ..
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 kafka_2.13-4.2.0-SNAPSHOT
   -rw-rw-r-- 1 dejan dejan   3754359 Oct  6 10:05 kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   -rw-rw-r-- 1 dejan dejan 135767889 Oct  6 10:06 kafka_2.13-4.2.0-SNAPSHOT.tgz
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 site-docs
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$
   ```



##########
wrapper.gradle:
##########
@@ -35,14 +35,12 @@ task bootstrapWrapper() {
         def wrapperBasePath = "\$APP_HOME/gradle/wrapper"
         def wrapperJarPath = wrapperBasePath + "/gradle-wrapper.jar"
 
-        // Add a trailing zero to the version if needed.

Review Comment:
   @chia7712 the thing is that Gradle team aligned/consolidated tag names and version numbers starting with version 9: now all tag names/versions have two dots (and three digits): https://github.com/apache/kafka/pull/20327#pullrequestreview-3131743127 
   
   So, my intention was to remove redundant check (rationale: if all versions are the same there is no need to add a trailing zero :slightly_smiling_face: and I assume it is highly unlikely that Gradle team will revert this decision).
   
   LMK what you think: should we leave as-is in this PR or you want to be extra cautious so I can fall back to trunk for this check ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #19513:
URL: https://github.com/apache/kafka/pull/19513


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2406726608


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   Update: I just pushed a commit that uses the upgrade manual suggestion (mentioned above) for permissions  and goes even further with turning off the reproducible builds feature for Gradle 9 build. 
   
   I assume this is actually a good and safer approach: we can create a separate JIRA ticket and turn on the reproducible builds (at some point in the future and as an isolated change).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "joshua2519 (via GitHub)" <gi...@apache.org>.
joshua2519 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2412808309


##########
gradlew:
##########
@@ -212,19 +199,31 @@ if "$cygwin" || "$msys" ; then
 fi
 
 
+
+# Loop in case we encounter an error.
+for attempt in 1 2 3; do
+  if [ ! -e "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" ]; then
+    if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v9.1.0/gradle/wrapper/gradle-wrapper.jar"; then
+      rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
+      # Pause for a bit before looping in case the server throttled us.
+      sleep 5
+      continue
+    fi
+  fi
+done
+
 # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
 DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
 
 # Collect all arguments for the java command:
-#   * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
+#   * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
 #     and any embedded shellness will be escaped.
 #   * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
 #     treated as '${Hostname}' itself on the command line.
 
 set -- \
         "-Dorg.gradle.appname=$APP_BASE_NAME" \
-        -classpath "$CLASSPATH" \
-        org.gradle.wrapper.GradleWrapperMain \
+        -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \

Review Comment:
   I file the PR #20658 already.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2405861455


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   So, just to be on the same page: you are referring to https://docs.gradle.org/9.1.0/userguide/upgrading_major_version_9.html#reproducible_archives_by_default ?
   <img width="1078" height="696" alt="image" src="https://github.com/user-attachments/assets/96d8de75-95fe-4d0e-a2b7-97f92d0ae256" />
   
   Frankly, this looks like a better solution, let me try to implement it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3260290880

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3243660578

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3241148913

   FYI @chia7712 
   
   **Just to sum it up here:**
   - Gradle version upgrade (8 to 9) is completed (see commits) :white_check_mark: - I did a lots of testing and everything looks fine (when compiled locally, that is)
   - but alas: Github Actions build with Gradle 9 has some issues :x:; see here for more details: #20450 (and please consider providing workflow runs for that PR so I can test some scenarios to bypass these issues)
   
   **In broader context:**
   - GitHub Actions <<--->> Develocity issues are blocking Gradle 8 to 9 upgrade
   - Gradle 9.1 is required for Java 25: https://github.com/gradle/gradle/releases/tag/v9.1.0-RC1 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-2831792793

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3189180031

   Just and update: I'm working on a POC (skeleton project/stripped down Kafka) in order to provide some solution.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2393862730


##########
gradle/dependencies.gradle:
##########
@@ -61,8 +61,8 @@ versions += [
   commonsLang: "3.18.0",
   commonsValidator: "1.9.0",
   classgraph: "4.8.179",
-  gradle: "8.14.3",
-  grgit: "4.1.1",
+  gradle: "9.1.0",
+  grgit: "5.3.0",

Review Comment:
   Are those changed related to gradle upgrade? It would be good to ensure the code changed by this PR is "related" to gradle upgrade. Perhaps, we could have a follow-up to upgrade the plugins version.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3358664101

   I may have some good news for you @chia7712 ! 
   Closing the gap on this issue: we will likely be able to drop Gradle helper `:submodule` :crossed_fingers: 
   
   So, I just scrapped a skeleton POC solution #20625 with just two things:
   - Gradle 8 -->> 9 upgrade 
   - `project(':tools:tools-api').configurations.runtimeClasspath` removed out of `releaseTarGz` task
   
   Results look encouraging: `releaseTarGz` task works fine (same goes for `:jmh-benchmarks:shadowJar`) :white_check_mark: 
   
   Action points for me would be to reorganize all remaining commits into a new solution (that keeps `releaseTarGz` task in `:core`) and perform some tests (should be doable in a few days.).
   
   If I may suggest something (my 2 cents on this): we could opt to solve KAFKA-19750 separately (i.e. on a Gradle 8 level).
   Rationale: it would be much easier to spot that single isolated change (and revert it back if/when the time comes) rather than bundle it here. 
   But it's your call, let me know which option you prefer.
   
   All-in-all: it is safe to say that things are wrapping up. 
   Stay tuned, we are so close :dart: :slightly_smiling_face: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3194549951

   @chia7712 I think I have a solution, but I need at least one week for testing before I push it here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239545985

   Update: I think I am closing the gap on this (at least I will isolate the problem).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3218056724

   Update: just one more thing needs to be solved.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370782408

   > I just left some follow-up answers.
   
   Pardon me, could you please provide the answer and put it in the relevant conversation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3294036384

   @chia7712 just a reminder for this :crossed_fingers: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365581447


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   It seems this update causes following error. 
   
   ```
   > Task :jmh-benchmarks:shadowJar FAILED
   
   [Incubating] Problems report is available at: file:///home/chia7712/project/kafka/build/reports/problems/problems-report.html
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   Execution failed for task ':jmh-benchmarks:shadowJar'.
   > Cannot expand ZIP '/home/chia7712/project/kafka/core/build/classes/java/test' as it does not exist.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3208865051

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3214232245

   Unfortunately Rat licence check fails on Github Action CI server even for a proper git placeholder file `.gitkeep`:
   
   ```
   Run python .github/scripts/rat.py
   Found 1 Rat reports
   Read env GITHUB_WORKSPACE: /home/runner/work/kafka/kafka
   Parsing Rat report file: build/rat/rat-report.xml
   6840 approved licenses
   1 unapproved licenses
   Files with unapproved licenses:
   Notice: File with unapproved license: distribution/.gitkeep
   Error: Process completed with exit code 1.
   ```
   
   Let's discuss @chia7712 about all of these tickets: 
   - included into this PR: 
     - https://issues.apache.org/jira/browse/KAFKA-19591 `gradlew` is referencing an old Groovy template version 
     - https://issues.apache.org/jira/browse/KAFKA-19174 Gradle version upgrade 8 -->> 9
   - newly created:
      - https://issues.apache.org/jira/browse/KAFKA-19636 Exclude some files (like .gitkeep) from Rat checks ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3229457946

   One additional commit is pushed - files and folders permissions are tidied up (in order to make them exactly the same as in distribution generated via Gradle version in trunk branch).
   
   All-in-all: everything looks just fine now (and on top of that: I made a **_lots_** of testing, see my previous comment above for details).
   
   @chia7712 now that Kafka 4.1.0 RC 3 is released I hope you are able to squeeze this into your schedule.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3316130108

   > @dejan2609 could you please take a look at my previous comment? [#19513 (comment)](https://github.com/apache/kafka/pull/19513#discussion_r2362766270)
   
   Sure, I am creating POC to explain this case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3316124655

   :warning: [Signing](https://docs.gradle.org/9.1.0/userguide/signing_plugin.html) is another thing that could be tested (in addition to those performed above).
   
   @chia7712 let me know what you think: is this type of tests required ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3316125823

   @dejan2609 could you please take a look at my previous comment? https://github.com/apache/kafka/pull/19513#discussion_r2362766270


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3168501977

   Pending: I will also explain second problem (specific to Gradle 9, that is).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370985553

   @chia7712 Related to `// Add a trailing zero to the version if needed.` :arrow_double_down: 
    ___
    
   The thing is that Gradle team aligned/consolidated tag names and version numbers starting with version 9: now all tag names/versions have two dots (and three digits): https://github.com/apache/kafka/pull/20327#pullrequestreview-3131743127 
   
   So, my intention was to remove redundant check (rationale: if all versions are the same there is no need to add a trailing zero :slightly_smiling_face: and I assume it is highly unlikely that Gradle team will revert this decision).
   
   LMK what you think: should we leave as-is in this PR or you want to be extra cautious so I can fall back to trunk for this check ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3282145505

   So, I just got deeper into Develocity test results for a flaky/new matrix, herewith some links and results:
   - flaky tests: https://develocity.apache.org/scans???search.buildToolType=gradle&list.sortColumn=tags&scanList.help=custom-values&search.buildOutcome=failure&search.buildToolType=gradle&search.buildToolVersions=9.0.0&search.relativeStartTime=P28D&search.rootProjectNames=kafka&search.tags=new&search.timeZoneId=Europe%2FBelgrade#
     - JDK 24: https://develocity.apache.org/s/7wmqzebwtylmu
     - JDK 17: https://develocity.apache.org/s/ml6j7v4omex64
   - new tests: https://develocity.apache.org/scans???search.buildToolType=gradle&list.sortColumn=tags&scanList.help=custom-values&search.buildOutcome=failure&search.buildToolType=gradle&search.buildToolVersions=9.0.0&search.relativeStartTime=P28D&search.rootProjectNames=kafka&search.tags=flaky&search.timeZoneId=Europe%2FBelgrade#
     - JDK 24: https://develocity.apache.org/s/we6qsz7zqfmn4
     - JDK 17: https://develocity.apache.org/s/qbegmh7pat4ha
   
   Same error pops up in all cases:   
    ``` 
   There are test sources present and no filters are applied, 
   but the test task did not discover any tests to execute. 
   This is likely due to a misconfiguration. 
   Please check your test configuration. 
   If this is not a misconfiguration, this error can be disabled 
   by setting the 'failOnNoDiscoveredTests' property to false.
   ```
   :bulb: This sounds like an issue I already solved for `streams:upgrade-system-tests-XY` submodules (see my commit `KAFKA-16801 do not fail Gradle 9 build when no test are discovered in a upgrade-system-tests* submodules`
   
   Will check this asap.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3338308713

   Ok, I'll split this POC into multiple comments, starting from my local environment: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version 
   openjdk version "17.0.16" 2025-07-15
   OpenJDK Runtime Environment (build 17.0.16+8-Ubuntu-0ubuntu124.04.1)
   OpenJDK 64-Bit Server VM (build 17.0.16+8-Ubuntu-0ubuntu124.04.1, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ javac -version 
   javac 17.0.16
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ sdk list gradle     
   ================================================================================
   Available Gradle Versions
   ================================================================================
      * 9.1.0               7.4                 5.6.1               3.2.1          
      * 9.0.0               7.3.3               5.6                 3.2            
    > * 8.14.3              7.3.2               5.5.1               3.1            
        8.14.2              7.3.1               5.5                 3.0            
      * 8.14.1              7.3                 5.4.1               2.14.1         
        8.14                7.2                 5.4                 2.14           
        8.13                7.1.1               5.3.1               2.13           
        8.12.1              7.1                 5.3                 2.12           
        8.12                7.0.2               5.2.1               2.11           
        8.11.1              7.0.1               5.2                 2.10           
        8.11                7.0                 5.1.1               2.9            
        8.10.2              6.9.4               5.1                 2.8            
        8.10.1              6.9.3               5.0                 2.7            
        8.10                6.9.2               4.10.3              2.6            
        8.9                 6.9.1               4.10.2              2.5            
      * 8.8                 6.9                 4.10.1              2.4            
      * 8.7                 6.8.3               4.10                2.3            
        8.6                 6.8.2               4.9                 2.2.1 
   ```
   ___
   
   These are Kafka trunk `gradlew` changes (up until now):
   
   Overview of `gradlew` changes in Kafka trunk:
   <img width="1889" height="581" alt="image" src="https://github.com/user-attachments/assets/8d6da280-0ac9-4970-b348-0579157ed850" />
   ___
   Related GitHub Gradle links for `unixStartScript.txt` (file that is used as a template for `gradlew` generation):
   
   The thing is that Gradle decided to change the path for `unixStartScript.txt` (template for `gradlew`) in Gradle version 8.8:
   - Gradle version 8.7 path for `unixStartScript.txt`: https://github.com/gradle/gradle/blob/v8.7.0/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
   - Gradle version 8.8 path for `unixStartScript.txt`: https://github.com/gradle/gradle/blob/v8.8.0/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
   
   🪧Loosely related Gradle issue: https://github.com/gradle/gradle/issues/30101


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "omkreddy (via GitHub)" <gi...@apache.org>.
omkreddy commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388441851

   @dejan2609  Thanks for working in this. It looks like a few custom tasks (`./gradlew unitTest, ./gradlew integrationTest`) aren’t working as expected after the upgrade.
   
   cc @chia7712 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388680745

   @omkreddy Can't find issues here:
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ date
   Fri 10 Oct 09:28:15 CEST 2025
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git status 
   On branch trunk
   Your branch is up-to-date with 'origin/trunk'.
   
   nothing to commit, working tree clean
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git checkout c6bbbbe24d
   Note: switching to 'c6bbbbe24d'.
   
   You are in 'detached HEAD' state. You can look around, make experimental
   changes and commit them, and you can discard any commits you make in this
   state without impacting any branches by switching back to a branch.
   
   If you want to create a new branch to retain commits you create, you may
   do so (now or later) by using -c with the switch command. Example:
   
     git switch -c <new-branch-name>
   
   Or undo this operation with:
   
     git switch -
   
   Turn off this advice by setting config variable advice.detachedHead to false
   
   HEAD is now at c6bbbbe24d KAFKA-19174 Gradle version upgrade 8 -->> 9 (#19513)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -1 --oneline 
   c6bbbbe24d (HEAD) KAFKA-19174 Gradle version upgrade 8 -->> 9 (#19513)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version
   openjdk version "17.0.16" 2025-07-15
   OpenJDK Runtime Environment Temurin-17.0.16+8 (build 17.0.16+8)
   OpenJDK 64-Bit Server VM Temurin-17.0.16+8 (build 17.0.16+8, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew unitTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id c6bbbbe2) using Gradle 9.1.0, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 2 Kafka message JSON file(s).
   MessageGenerator: processed 5 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 44 Kafka message JSON file(s).
   MessageGenerator: processed 26 Kafka message JSON file(s).
   MessageGenerator: processed 197 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   Note: Some input files use or override a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   Note: /home/dejan/kafka/metadata/src/test/java/org/apache/kafka/controller/metrics/QuorumControllerMetricsTest.java uses unchecked or unsafe operations.
   Note: Recompile with -Xlint:unchecked for details.
   Note: /home/dejan/kafka/server/src/test/java/org/apache/kafka/server/AssignmentsManagerTest.java uses or overrides a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew integrationTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id c6bbbbe2) using Gradle 9.1.0, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ sdk use java 24.0.2-tem
   
   Using java version 24.0.2-tem in this shell.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version 
   openjdk version "24.0.2" 2025-07-15
   OpenJDK Runtime Environment Temurin-24.0.2+12 (build 24.0.2+12)
   OpenJDK 64-Bit Server VM Temurin-24.0.2+12 (build 24.0.2+12, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew unitTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id c6bbbbe2) using Gradle 9.1.0, Java 24 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   MessageGenerator: processed 2 Kafka message JSON file(s).
   MessageGenerator: processed 1 Kafka message JSON file(s).
   MessageGenerator: processed 5 Kafka message JSON file(s).
   MessageGenerator: processed 26 Kafka message JSON file(s).
   MessageGenerator: processed 44 Kafka message JSON file(s).
   MessageGenerator: processed 197 Kafka message JSON file(s).
   MessageGenerator: processed 4 Kafka message JSON file(s).
   WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
   WARNING: sun.misc.Unsafe::arrayBaseOffset has been called by sbt.internal.shaded.com.google.protobuf.UnsafeUtil$MemoryAccessor (file:/home/dejan/.gradle/caches/modules-2/files-2.1/org.scala-sbt/zinc-persist-core-assembly/1.10.8/f29192c40cd73f2880b38e33ed9331ff70652d76/zinc-persist-core-assembly-1.10.8.jar)
   WARNING: Please consider reporting this to the maintainers of class sbt.internal.shaded.com.google.protobuf.UnsafeUtil$MemoryAccessor
   WARNING: sun.misc.Unsafe::arrayBaseOffset will be removed in a future release
   Note: Some input files use or override a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   Note: /home/dejan/kafka/metadata/src/test/java/org/apache/kafka/controller/metrics/QuorumControllerMetricsTest.java uses unchecked or unsafe operations.
   Note: Recompile with -Xlint:unchecked for details.
   Note: /home/dejan/kafka/server/src/test/java/org/apache/kafka/server/AssignmentsManagerTest.java uses or overrides a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew integrationTest -q
   Starting build with version 4.2.0-SNAPSHOT (commit id c6bbbbe2) using Gradle 9.1.0, Java 24 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ date
   Fri 10 Oct 09:38:24 CEST 2025
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388734158

   > @dejan2609 Thanks for working in this. It looks like a few custom tasks (`./gradlew unitTest, ./gradlew integrationTest`) aren’t working as expected after the upgrade.
   > 
   > cc @chia7712
   
   All in all: I see no errors here @omkreddy 
   Please explain your case in more detail and create a ticket here: https://issues.apache.org/jira/projects/KAFKA/issues 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2404954107


##########
wrapper.gradle:
##########
@@ -35,14 +35,12 @@ task bootstrapWrapper() {
         def wrapperBasePath = "\$APP_HOME/gradle/wrapper"
         def wrapperJarPath = wrapperBasePath + "/gradle-wrapper.jar"
 
-        // Add a trailing zero to the version if needed.

Review Comment:
   @dejan2609 Have you checked the comment before deleting it? for example, the following link is broken
   https://raw.githubusercontent.com/gradle/gradle/v9.1/gradle/wrapper/gradle-wrapper.jar
   
   it works if we add a trailing zero.
   
   https://raw.githubusercontent.com/gradle/gradle/v9.1.0/gradle/wrapper/gradle-wrapper.jar
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "Goooler (via GitHub)" <gi...@apache.org>.
Goooler commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365677702


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   ```suggestion
     id 'com.gradleup.shadow' version '8.3.9' apply false
   ```



##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   You can use 8.3.9 instead, which is also Gradle 9 compatible.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3143759025

   Update: project compilation works as expected (when powered by Gradle 9) but binary release creation fails. 
   More work has to be done in order to accommodate to Gradle breaking changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3212837033

   _**Few notes for reviewers:**_ 
   -  **Gradle team confirms that version 9.0.0 is stable** (regression mentioned above is hitting only **9.1.0-rc-1** version: https://github.com/gradle/gradle/issues/34691#issuecomment-3208979878)
   - I opted to split solution into three commits (just for the sake of clarity) but will eventually squash them into one
   - distributions generated by Gradle 9 (both **_main_** and **_site-docs_**) are looking just fine (when it comes to a size and/or number of files/folders)
   - size for both and old and new **_main_** distribution is 130 MB and number of files is identical
   - there is a slight difference in size for a site-docs distribution (Gradle 9.0: 3.7 MB vs Gradle 8.14: 3.6 MB, but files and folders are identical - _**note: will double-check this tomorrow**_)
   - also I will try to do some additional testing (i.e. to setup Kafka cluster out of this distribution) ASAP
   
   I invite you all to review (and hopefully someone will pick this up :smiley:): 
   @chia7712 @apoorvmittal10 @mumrah @ijuma @showuon


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3214875168

   Ok, I will also try to bump Github actions setup gradle action version in order to fix this issue: 
   
   <img width="1324" height="740" alt="image" src="https://github.com/user-attachments/assets/ee6bbd1c-2a66-434d-a3cd-33791d6db7df" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3217238796

   :pushpin: _**Feel free to start your review here**_ (there were some changes above in order to solve issues with Github Actions job/workflows). 
   
   You will find three git commits (eventually I will squash them into one, now there are separated just for clarity):
   1. commit solves [KAFKA-19591](https://issues.apache.org/jira/browse/KAFKA-19591):`gradlew` is referencing an old Groovy template version
   2. commit upgrades Gradle [KAFKA-19174](https://issues.apache.org/jira/browse/KAFKA-19174): Gradle version upgrade 8 -->> 9
   3. _[note: optional commit]:_ Gradle plugins version upgrades


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3263579498

   Thank you for a review @chia7712. 
   Good points, will address them ASAP.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3315169063

   Seems ok, a few unrelated tests have failed: https://github.com/apache/kafka/actions/runs/17882006645/job/50851190821?pr=19513


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365565530


##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   @dejan2609 Curious about your thoughts here - would adding a new module be necessary, or could we simply continue handling the distributions within the core module?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3312887483

   Test results look fine (with only one test flaky and another failed; both issues seem unrelated to this build change): 
   
   <img width="1663" height="796" alt="image" src="https://github.com/user-attachments/assets/4267a388-3d50-4eba-9686-2bbca099ee30" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3333523460

   @dejan2609 It would be helpful to understand the reasoning behind the changes in this PR, since introducing unnecessary modifications is usually not a good idea. That’s why I left some questions on parts where I couldn’t see the motivation—for example: https://github.com/apache/kafka/pull/19513#discussion_r2378273533
   
   Upgrading Gradle is an important task for Kafka, so I’d appreciate your help in clarifying the intent of this PR. Thanks!
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3334752759

   @chia7712 Come to think of it: there is an option [4] also: I'll create a POC to narrow these options down :slightly_smiling_face: 
   
   Stay tuned, I'll send an update here ASAP.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3199081286

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3217444662

   :white_check_mark:  **Good news**: .tgz distribution files (old and new) are almost the same in size (and they have a same number of files).
   :x: **Pitfall**:  unpacked new archive creates folder with a wrong date - I will have to sort that out:
   
   ```
   user@host:~/kafka/distribution/build/distributions$ ls -lh
   total 134M
   drwxr-xr-x 7 dejan dejan 4.0K Jan  2  1970 kafka_2.13-4.2.0-SNAPSHOT
   -rw-rw-r-- 1 dejan dejan 3.6M Aug 23 23:25 kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   -rw-rw-r-- 1 dejan dejan 130M Aug 23 23:25 kafka_2.13-4.2.0-SNAPSHOT.tgz
   drwxr-xr-x 7 dejan dejan 4.0K Jan  2  1970 site-docs
   ```
   <img width="888" height="543" alt="image" src="https://github.com/user-attachments/assets/d2f31cbf-c689-4c8c-9fc4-adb5f7994e1f" /> 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3368517551

   @chia7712 done, I kept only two commits (but I can squash them into one):
   - one for the Gradle 8 level fix
   - another one for Gradle 8 to 9 upgrade
   
   No version are changed during this rebase (except for `gradle/actions/setup-gradle`: initial version upgrade was from 4.3.0 to 4.4.3, but I just bumped it to 4.4.4 - that version was tested against Gradle 9.1.0).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370543704

   No problem @chia7712 ! 
   
   I just left some follow-up answers.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370846646

   @chia7712 Yes, I left comments for both conversations: one for the `preserveFileTimestamps = true` and another for Gradle version trailing zero case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3371051778

   > If you are in review mode, you must click the `Submit review` button to finalize your comment
   
   @chia7712  Thanks for helping out! 
   Truth to te told I find this entire GitHub setup to be not intuitive and overzealously complicated. 
   
   I submitted my comments, and you should be able to see them now :crossed_fingers: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370999111

   @chia7712 maybe this PR managed to break some limit in comments :slightly_smiling_face: 
   Anyway: I copied and pasted my follow-ups on your reviews in the comments above (I hope you can see them).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3355289025

   Hi @chia7712 !   I did some cleanup (resolved old/outdated conversations). 
   
   You probably have a lot of things in your schedule, but please try to squeeze this in somehow.
   
   TLDR version for you:
   - a `distribution` Gradle submodule is required (but that solution is as light as possible: paths for generated tgz files stay the same)
   - as for `gradlew` changes (KAFKA-19591): they are in fact a fix (missing pieces) generated by Gradle 8 - see my detailed explanation above 
   
   Waiting for your input on this.
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3389955876

   @omkreddy Thanks for your report. We will take a look and test other commands to ensure everything works correctly


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3390127934

   Another task, `aggregatedJavadoc`, is also faling with an error similar to the one in #20628 - will dig in it later


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3116295492

   This PR is being marked as stale since it has not had any activity in 90 days. If you
   would like to keep this PR alive, please leave a comment asking for a review. If the PR has 
   merge conflicts, update it with the latest from the base branch.
   <p>
   If you are having difficulty finding a reviewer, please reach out on the 
   [mailing list](https://kafka.apache.org/contact).
   <p>
   If this PR is no longer valid or desired, please feel free to close it. If no activity
   occurs in the next 30 days, it will be automatically closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3274834504

   Good news ✅:
   - overall Gradle build looks fine 
   - all build scans are now published to Develocity :tada: (I had some issues recently so I created new PR here to experiment: #20450)
   
   There are some issues with one Github Actions :x:: flaky/new test matrix has one step that fails (`Parse Junit tests`) but that same step works fine for majority of tests.
   I assume that `.github/scripts/junit.py` Python script should be checked.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "Goooler (via GitHub)" <gi...@apache.org>.
Goooler commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365635913


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   This is caused by the wrong configuration introduced by #20547.
   
   https://github.com/apache/kafka/blob/d067c6c04089a3d24e1f72e6cb1b10b0d85f76da/build.gradle#L3353
   
   
   Reverted that PR and ran `gw :jmh-benchmarks:shadowJar` again, and the task passed.
   
   If you want to keep that, need to rearrange your task dependencies.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3215376032

   Ok, it seems to me that this PR is (**_finally_**) ready for a review. 
   Rationale: tests results (for both Java 17 and 24) are excellent (under 1% of tests are skipped/flaky and almost none of them end up with errors).
   
   I still prefer to leave multiple commits just during the review (presumably it will be easier for reviewers to notice how/when/why somethings has been changed).
   
   FYI @chia7712 @apoorvmittal10 @mumrah @ijuma @showuon
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3216191094

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "yeikel (via GitHub)" <gi...@apache.org>.
yeikel commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3390047209

   > @omkreddy Thanks for your report. We will take a look and test other commands to ensure everything works correctly
   
   Can we add a CI task that validates these commands? I'd assume that the goal should be to avoid regressions for these moving forward 
   
    A separate workflow would be fine, I think


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3390113268

   I will file a PR to fix the task `unittest` and `integrationTest` according to https://docs.gradle.org/current/userguide/upgrading_major_version_9.html#test_tasks_may_no_longer_execute_expected_tests
   
   > A separate workflow would be fine, I think
   
   there are many custom tasks in kafka, but yes it is worth having the CI - I have opened https://issues.apache.org/jira/browse/KAFKA-19780 to discuss that later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3355501251

   Aha, now I see it clearly :bulb:
    It goes without saying that we should avoid a `:distribution` module (it that's possible).
   
   Allow me some time to test this idea, will update you asap. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3355314891

   @dejan2609 I open the https://issues.apache.org/jira/browse/KAFKA-19750 to describe the root cause of `releaseTarGz`. I think the simple way to fix it is to remove `project(':tools:tools-api')` from `releaseTarGz`, since it is already in runtime path of `core` module. Also, it would be helpful to leave comment on the `implementation project(':tools:tools-api')` to remind developers to add `project(':tools:tools-api')` back if `project(':tools:tools-api')` is removed from the runtime path


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239714559

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3240990397

   I just accidentally deleted remote branch (from my command line). Will recover it instantly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3240992380

   Done, branch recovered :relieved: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3239393015

   Ok, I will just try to revert commits one-by-one in order to find _**the**_ one that has issues with JUnit test results parsing :thinking: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3373008540

   @chia7712 A few unrelated tests are broken (for both Java 17 and 24) but other than that the build is looking fine.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2405684688


##########
build.gradle:
##########
@@ -330,6 +329,19 @@ subprojects {
     tasks.register('uploadArchives').configure { dependsOn(publish) }
   }
 
+    def  submodulesToIncludeInDistribution = ['core', 'tools', 'trogdor', 'shell', 'api', 'runtime', 'transforms',
+                                              'json', 'file', 'basic-auth-extension', 'mirror', 'mirror-client',
+                                              'streams', 'streams-scala', 'test-utils', 'examples', 'tools-api']
+

Review Comment:
   Not important anymore (another solution was implemented). 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3368827888

   Only a few unrelated Github Actions CI tests failed for both Java 17 and 24, so we are good to go.
   
   I also did some local testing (counted files in distributions and did some Kafka cluster testing for Gradle 9 generated distribution):
   
   **Environment:**
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ java -version 
   openjdk version "17.0.16" 2025-07-15
   OpenJDK Runtime Environment Temurin-17.0.16+8 (build 17.0.16+8)
   OpenJDK 64-Bit Server VM Temurin-17.0.16+8 (build 17.0.16+8, mixed mode, sharing)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -3 --oneline 
   4278e5621e (HEAD -> KAFKA-19174, origin/KAFKA-19174) KAFKA-19174 Gradle version upgrade (8 -->> 9); build logic is refactored
   9318bd3480 KAFKA-19591 solving issues with `gradlew` content: an old template version (i.e. file `unixStartScript.txt`) was being referenced
   d76442e5a6 (upstream/trunk, upstream/HEAD, origin/trunk, origin/HEAD, trunk) MINOR: Fix a minor copy&paste typo in comments (#20634)
   ```
   **Steps:** 
   - create distributions: `./gradlew clean releaseTarGz -i`
   - move them into some folder and untar them: 
     - tar xvfz kafka_2.13-4.2.0-SNAPSHOT.tgz 
     - tar xvfz kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz 
   - count number of files: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions-Gradle-9$ find kafka_2.13-4.2.0-SNAPSHOT -type f | wc -l 
   223
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions-Gradle-9$ find site-docs/ -type f | wc -l 
   145
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions-Gradle-9$
   ```
   - repeat the same procedure for the trunk: the number of files is identical
   ___
   Finally: I started Kafka cluster from the Gradle-9-generated distribution (as noted here: https://kafka.apache.org/quickstart) and everything works just fine (I was able to send and receive messages).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3168442040

   @chia7712 feel free to check this related PR: https://github.com/apache/kafka/pull/20327 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3193344479

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3166720779

   @chia7712 
   Will provide an update during the weekend (there are two problems with Gradle 9 build, will explain in more details and provide some insights).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3217140937

   :information_source:  I will just reorder git commits once more (there are no reviews yet, so I m not interrupting anyone).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3235589490

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2327711607


##########
gradle/dependencies.gradle:
##########
@@ -119,13 +119,13 @@ versions += [
   // When updating the scalafmt version please also update the version field in checkstyle/.scalafmt.conf. scalafmt now
   // has the version field as mandatory in its configuration, see
   // https://github.com/scalameta/scalafmt/releases/tag/v3.1.0.
-  scalafmt: "3.7.14",
-  scoverage: "2.0.11",
+  scalafmt: "3.9.9",

Review Comment:
   Are those changes necessary?



##########
.github/actions/setup-gradle/action.yml:
##########
@@ -42,7 +42,7 @@ runs:
         distribution: temurin
         java-version: ${{ inputs.java-version }}
     - name: Setup Gradle
-      uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0

Review Comment:
   Could you please use the git commit hash instead, to follow the Apache policy (see below)?
   
   > You MUST pin all external actions to the specific git hash (SHA1) of the action that has been reviewed for use by the project. For instance, you MUST pin foobar/baz-action@8843d7f92416211de9ebb963ff4ce28125932878.
   
   reference: https://infra.apache.org/github-actions-policy.html



##########
.github/workflows/build.yml:
##########
@@ -164,7 +164,7 @@ jobs:
         # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
         # message and list the empty files
         run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz

Review Comment:
   Changing the path may break downstream CI or workflows. Could you please provide more details about this change?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3284082346

   @chia7712 all tests in green :100: 
   This is ready for another review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3338706171

   @chia7712 you can review this again.
   
   To sum it up (and please pardon me for such a long comments above, this things are quite tricky to catch and hence I wanted to provide you as much details as possible):
   - root cause for this issue is Gradle 8.8 path change for the template file `unixStartScript.txt` (that is used by Gradle to generate `gradlew`)
   - the issue is tricky to find: if we are using gradle wrapper only (i.e. gradle wrapper jar older versions) then this issue stays hidden
   - one has to use his local Gradle distribution to find it / fix it
   
   Where to proceed from here: 
   1. merge this commit for KAFKA-19591 separately or
   2. bundle it with Gradle 8 to 9 upgrade 
   
   Note: maybe it would be a good idea to describe this corner case in README.md file.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "brandboat (via GitHub)" <gi...@apache.org>.
brandboat commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3361120926

   Many thanks to the help @chia7712, @dejan2609. Here is the pr link: https://github.com/apache/kafka/pull/20628


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3390654001

   I noticed another error that was introduced by #12280 and it unrelated to Gradle9
   ```
   * What went wrong:
   A problem occurred evaluating root project 'kafka'.
   > No such property: name for class: org.gradle.language.scala.tasks.KeepAliveMode
   ```
   
   It can be fixed by changing the `name` to `name()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2404940333


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true

Review Comment:
   @dejan2609 why to add this change?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2405312571


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true

Review Comment:
   @chia7712 Prologue (comment from back in August): https://github.com/apache/kafka/pull/19513#issuecomment-3217444662 
   
   Related links:
   - Gradle issue: https://github.com/gradle/gradle/issues/34643
   - Gradle DSL: https://docs.gradle.org/9.1.0/dsl/org.gradle.api.tasks.bundling.Tar.html#org.gradle.api.tasks.bundling.Tar:preserveFileTimestamps
    
   
   I just tested again, with the following procedure: comment out `preserveFileTimestamps = true` for tasks `:core:siteDocsTar` and `:core:releaseTarGz` and execute `./gradlew clean releaseTarGz -i`:
   
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -3 --oneline 
   4278e5621e (HEAD -> KAFKA-19174, origin/KAFKA-19174) KAFKA-19174 Gradle version upgrade (8 -->> 9); build logic is refactored
   9318bd3480 KAFKA-19591 solving issues with `gradlew` content: an old template version (i.e. file `unixStartScript.txt`) was being referenced
   d76442e5a6 (origin/trunk, origin/HEAD, trunk) MINOR: Fix a minor copy&paste typo in comments (#20634)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git diff
   diff --git a/build.gradle b/build.gradle
   index c3cc954e4b..8c7f1d19b4 100644
   --- a/build.gradle
   +++ b/build.gradle
   @@ -1222,7 +1222,7 @@ project(':core') {
        from project.file("$rootDir/docs")
        into 'site-docs'
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
      }
    
      tasks.create(name: "releaseTarGz", dependsOn: configurations.archives.artifacts, type: Tar) {
   @@ -1268,7 +1268,7 @@ project(':core') {
        from(project(':streams:examples').jar) { into("libs/") }
        from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
        duplicatesStrategy 'exclude'
   -    preserveFileTimestamps = true
   +    //preserveFileTimestamps = true
        filePermissions {
            user {
                read = true
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ./gradlew clean releaseTarGz -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 4278e562) using Gradle 9.1.0, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   MessageGenerator: processed 4 Kafka message JSON file(s).
   ...
   ```
   Unpack generated distributions:
   - cd core/build/distributions/
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT.tgz
   - tar xvfz kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   
   **Folders show the wrong date: 02nd Jan 1970:** (with default `preserveFileTimestamps = false`) :arrow_down_small: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$ ls -la
   total 136280
   drwxrwxr-x 4 dejan dejan      4096 Oct  6 10:09 .
   drwxrwxr-x 9 dejan dejan      4096 Oct  6 10:05 ..
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 kafka_2.13-4.2.0-SNAPSHOT
   -rw-rw-r-- 1 dejan dejan   3754359 Oct  6 10:05 kafka_2.13-4.2.0-SNAPSHOT-site-docs.tgz
   -rw-rw-r-- 1 dejan dejan 135767889 Oct  6 10:06 kafka_2.13-4.2.0-SNAPSHOT.tgz
   drwxr-xr-x 7 dejan dejan      4096 Jan  2  1970 site-docs
   dejan@dejan-HP-ProBook-450-G7:~/kafka/core/build/distributions$
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2362766270


##########
build.gradle:
##########
@@ -3904,6 +3856,109 @@ project(':connect:test-plugins') {
   }
 }
 
+project(':distribution') {

Review Comment:
   Just wondering - does this change matter for the upgrade from gradle 8 to 9



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3226609376

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3237287563

   Update: I will try to rebase all commits onto upstream/trunk current state.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3172464870

   **So, just to sum it up:** Gradle 9 task **_:core:releaseTarGz_** (for a binary tgz file creation) fails :x::
   
   ```
   :core:releaseTarGz (Thread[Execution worker Thread 3,5,main]) started.
   
   > Task :core:releaseTarGz FAILED
   
   [Incubating] Problems report is available at: file:///home/dejan/kafka/build/reports/problems/problems-report.html
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   Execution failed for task ':core:releaseTarGz'.
   > Resolution of the configuration ':tools:tools-api:runtimeClasspath' was attempted without an exclusive lock. This is unsafe and not allowed.
   ```
   
   **What needs to be done** :bulb:: it seems to me that Gradle [producer-consumer concept](https://docs.gradle.org/9.0.0/userguide/declaring_dependencies.html#sub:producers-vs-consumers-basics) needs to be applied for Kafka submodules.
   
   **Other related links (mental note for me)** :ledger::
   - https://docs.gradle.org/9.0.0/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors
   - https://docs.gradle.org/9.0.0/userguide/multi_project_builds.html#sec:depending_on_output_of_another_project
   - https://docs.gradle.org/9.0.0/samples/sample_cross_project_output_sharing.html
   - https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#packaging_and_artifact_behavior_changes
   
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [KAFKA-19174] [KAFKA-19591] Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3214429289

   :bulb:  Will try to skip Apache Rat check for .gitkeep :crossed_fingers: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3217966391

   >❌ **Pitfall**: unpacked new archive creates folder with a wrong date (on my Ubuntu laptop, at least) - I will have to doublecheck that:
   
   :bulb: It's not a bug :beetle:  it's a feature :smiley::
   https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#reproducible_archives_by_default
   
   Also discussed here:  
   - https://github.com/gradle/gradle/issues/34643#issuecomment-3182810708 
   
   Will solve this, should be easy.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3218711196

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 closed pull request #19513: KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9
URL: https://github.com/apache/kafka/pull/19513


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2365649274


##########
build.gradle:
##########
@@ -29,22 +29,21 @@ buildscript {
 }
 
 plugins {
-  id 'com.github.ben-manes.versions' version '0.48.0'
+  id 'com.github.ben-manes.versions' version '0.52.0'
   id 'idea'
   id 'jacoco'
   id 'java-library'
-  id 'org.owasp.dependencycheck' version '8.2.1'
+  id 'org.owasp.dependencycheck' version '12.1.3'
   id 'org.nosphere.apache.rat' version "0.8.1"
   id "io.swagger.core.v3.swagger-gradle-plugin" version "${swaggerVersion}"
 
-  id "com.github.spotbugs" version '6.2.3' apply false
+  id "com.github.spotbugs" version '6.2.5' apply false
   id 'org.scoverage' version '8.0.3' apply false
-  id 'com.gradleup.shadow' version '8.3.6' apply false
-  id 'com.diffplug.spotless' version "6.25.0"
+  id 'com.gradleup.shadow' version '9.0.2' apply false

Review Comment:
   The import is required due to `org.apache.kafka.coordinator.common.runtime.MockCoordinatorExecutor`.
   
   > If you want to keep that, need to rearrange your task dependencies.
   
   Perhaps we can revisit it in the follow-up. If not addressed, it could prevent the upgrade 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "omkreddy (via GitHub)" <gi...@apache.org>.
omkreddy commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3388760324

   @dejan2609 I think the command is not running the tests. Normally we should see test running output like
   
   ```
   ➜  kafka git:(f5a87b3703) ./gradlew  unitTest
   
   > Configure project :
   Starting build with version 4.2.0-SNAPSHOT (commit id f5a87b37) using Gradle 8.14.3, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=10, maxScalacThreads=8, maxTestRetries=0
   
   > Task :metadata:processMessages
   MessageGenerator: processed 26 Kafka message JSON file(s).
   
   > Task :clients:processMessages
   MessageGenerator: processed 197 Kafka message JSON file(s).
   
   > Task :raft:processMessages
   MessageGenerator: processed 1 Kafka message JSON file(s).
   
   > Task :storage:processMessages
   MessageGenerator: processed 5 Kafka message JSON file(s).
   
   > Task :transaction-coordinator:processMessages
   MessageGenerator: processed 2 Kafka message JSON file(s).
   
   > Task :group-coordinator:processMessages
   MessageGenerator: processed 44 Kafka message JSON file(s).
   
   > Task :share-coordinator:processMessages
   MessageGenerator: processed 4 Kafka message JSON file(s).
   
   > Task :streams:processMessages
   MessageGenerator: processed 1 Kafka message JSON file(s).
   
   > Task :generator:unitTest
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 146 > EntityTypeTest > testVerifyTypeMismatches() PASSED
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 146 > EntityTypeTest > testVerifyTypeMatches() PASSED
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 146 > EntityTypeTest > testUnknownEntityType() PASSED
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 148 > CodeBufferTest > testWrite() PASSED
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 148 > CodeBufferTest > testIndentMustBeNonNegative() PASSED
   
   Gradle Test Run :generator:unitTest > Gradle Test Executor 148 > CodeBufferTest > testEquals() PASSED
   ...
   ...
   ```
   
   On trunk: 
   ```
   ➜  kafka git:(trunk) ./gradlew clean  unitTest
   
   > Configure project :
   Starting build with version 4.2.0-SNAPSHOT (commit id 1e95d046) using Gradle 9.1.0, Java 17 and Scala 2.13.17
   Build properties: ignoreFailures=false, maxParallelForks=10, maxScalacThreads=8, maxTestRetries=0
   
   > Task :streams:processMessages
   MessageGenerator: processed 1 Kafka message JSON file(s).
   
   > Task :transaction-coordinator:processMessages
   MessageGenerator: processed 2 Kafka message JSON file(s).
   
   > Task :share-coordinator:processMessages
   MessageGenerator: processed 4 Kafka message JSON file(s).
   
   > Task :raft:processMessages
   MessageGenerator: processed 1 Kafka message JSON file(s).
   
   > Task :storage:processMessages
   MessageGenerator: processed 5 Kafka message JSON file(s).
   
   > Task :group-coordinator:processMessages
   MessageGenerator: processed 44 Kafka message JSON file(s).
   
   > Task :metadata:processMessages
   MessageGenerator: processed 26 Kafka message JSON file(s).
   
   > Task :clients:processMessages
   MessageGenerator: processed 197 Kafka message JSON file(s).
   
   > Task :clients:processTestMessages
   MessageGenerator: processed 4 Kafka message JSON file(s).
   
   > Task :clients:compileTestJava
   Note: Some input files use or override a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   
   > Task :metadata:compileTestJava
   Note: /Users/manikumar/workspace/kafka/metadata/src/test/java/org/apache/kafka/controller/metrics/QuorumControllerMetricsTest.java uses unchecked or unsafe operations.
   Note: Recompile with -Xlint:unchecked for details.
   
   > Task :server:compileTestJava
   Note: /Users/manikumar/workspace/kafka/server/src/test/java/org/apache/kafka/server/AssignmentsManagerTest.java uses or overrides a deprecated API.
   Note: Recompile with -Xlint:deprecation for details.
   
   [Incubating] Problems report is available at: file:///Users/manikumar/workspace/kafka/build/reports/problems/problems-report.html
   
   Deprecated Gradle features were used in this build, making it incompatible with Gradle 10.
   
   You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
   
   For more on this, please refer to https://docs.gradle.org/9.1.0/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
   
   BUILD SUCCESSFUL in 1m 28s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3390249764

   > Another task, aggregatedJavadoc, is also faling with an error similar to the one in https://github.com/apache/kafka/pull/20628 - will dig in it later
   
   The simple fix is to add `--no-parallel` to the `aggregatedJavadoc` task, The build time increased from `9s` to `12s`, so the regression should be acceptable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3338624270

   Now that we have an environment in place, we need to do the following things: 
   - checkout Kafka `trunk`
   - make sure that there are no changes whatsoever
   - remove all files that are not committed/pushed into GitHub repo (we want to make sure that we don't have `gradle/wrapper/gradle-wrapper.jar` - hence we are forced to use our local Gradle installation)
   - use our local Gradle installation (same version as in trunk: 8.14.3)
   - execute: `gradle wrapper -q`
   - review git diff :slightly_smiling_face: 
   ```
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git diff
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git status
   On branch trunk
   Your branch is up-to-date with 'origin/trunk'.
   
   nothing to commit, working tree clean
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git clean -fdx
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ ls gradle/wrapper/
   gradle-wrapper.properties
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git log -5 --oneline 
   9e9d2a23ef (HEAD -> trunk, upstream/trunk, upstream/HEAD, origin/trunk, origin/HEAD) MINOR: fix flaky sys test for static membership (#20594)
   857b1e92cc KAFKA-19719: --no-initial-controllers should not assume kraft.version=1 (#20551)
   f4e00e9cf0 MINOR: Remove unnecessary check in ReplicaManager (#20588)
   97c8c6b595 KAFKA-19733 Fix arguments to assertEquals() in clients module (#20586)
   14917ae727 MINOR: Handle envelope response in AutoTopicCreationManager (#20569)
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ gradle -version
   
   ------------------------------------------------------------
   Gradle 8.14.3
   ------------------------------------------------------------
   
   Build time:    2025-07-04 13:15:44 UTC
   Revision:      e5ee1df3d88b8ca3a8074787a94f373e3090e1db
   
   Kotlin:        2.0.21
   Groovy:        3.0.24
   Ant:           Apache Ant(TM) version 1.10.15 compiled on August 25 2024
   Launcher JVM:  17.0.16 (Ubuntu 17.0.16+8-Ubuntu-0ubuntu124.04.1)
   Daemon JVM:    /usr/lib/jvm/java-17-openjdk-amd64 (no JDK specified, using current Java home)
   OS:            Linux 6.8.0-84-generic amd64
   
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ gradle wrapper -q
   Starting build with version 4.2.0-SNAPSHOT (commit id 9e9d2a23) using Gradle 8.14.3, Java 17 and Scala 2.13.16
   Build properties: ignoreFailures=false, maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git status
   On branch trunk
   Your branch is up-to-date with 'origin/trunk'.
   
   Changes not staged for commit:
     (use "git add <file>..." to update what will be committed)
     (use "git restore <file>..." to discard changes in working directory)
   	modified:   gradlew
   
   no changes added to commit (use "git add" and/or "git commit -a")
   dejan@dejan-HP-ProBook-450-G7:~/kafka$ git diff
   diff --git a/gradlew b/gradlew
   index f4bb3360e1..0eaf5d7382 100755
   --- a/gradlew
   +++ b/gradlew
   @@ -15,6 +15,8 @@
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
   +# SPDX-License-Identifier: Apache-2.0
   +#
    
    ##############################################################################
    #
   @@ -55,7 +57,7 @@
    #       Darwin, MinGW, and NonStop.
    #
    #   (3) This script is generated from the Groovy template
   -#       https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
   +#       https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
    #       within the Gradle project.
    #
    #       You can find Gradle at https://github.com/gradle/gradle/.
   @@ -84,7 +86,7 @@ done
    # shellcheck disable=SC2034
    APP_BASE_NAME=${0##*/}
    # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
   -APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
   +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
    
    # Use the maximum available, or set MAX_FD != -1 to use that value.
    MAX_FD=maximum
   @@ -112,20 +114,7 @@ case "$( uname )" in                #(
      NONSTOP* )        nonstop=true ;;
    esac
    
   -
   -# Loop in case we encounter an error.
   -for attempt in 1 2 3; do
   -  if [ ! -e "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" ]; then
   -    if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v8.14.3/gradle/wrapper/gradle-wrapper.jar"; then
   -      rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
   -      # Pause for a bit before looping in case the server throttled us.
   -      sleep 5
   -      continue
   -    fi
   -  fi
   -done
   -
   -CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
   +CLASSPATH="\\\"\\\""
    
    
    # Determine the Java command to use to start the JVM.
   @@ -216,7 +205,7 @@ fi
    DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
    
    # Collect all arguments for the java command:
   -#   * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
   +#   * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
    #     and any embedded shellness will be escaped.
    #   * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
    #     treated as '${Hostname}' itself on the command line.
   @@ -224,7 +213,20 @@ DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
    set -- \
            "-Dorg.gradle.appname=$APP_BASE_NAME" \
            -classpath "$CLASSPATH" \
   -        org.gradle.wrapper.GradleWrapperMain \
   +
   +# Loop in case we encounter an error.
   +for attempt in 1 2 3; do
   +  if [ ! -e "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" ]; then
   +    if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v8.14.3/gradle/wrapper/gradle-wrapper.jar"; then
   +      rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
   +      # Pause for a bit before looping in case the server throttled us.
   +      sleep 5
   +      continue
   +    fi
   +  fi
   +done
   +
   +        -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
            "$@"
    
    # Stop when "xargs" is not available.
   (END)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370958561

   Quite strange: I'm not able to copy links for my comments (and both of them are marked as `Pending` as if they are waiting for some approval or something like that):
   
   **First one:**
   <img width="1313" height="788" alt="image" src="https://github.com/user-attachments/assets/59d0b1c9-568e-4352-97dd-76c26c461f3d" />
   ___
   **Second one:**
   <img width="1245" height="785" alt="image" src="https://github.com/user-attachments/assets/8a9035de-b8d9-42cf-92e0-8380db591e3f" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3370997706

   > Quite strange: I'm not able to copy links for my comments (and both of them are marked as Pending as if they are waiting for some approval or something like that):
   
   If you are in review mode, you must click the `Submit review` button to finalize your comment
   
   <img width="1182" height="956" alt="螢幕快照 2025-10-06 18-44-35" src="https://github.com/user-attachments/assets/4f5ee32d-6739-4d0c-af5f-4452f9d88e5c" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3371020032

   > LMK what you think: should we leave as-is in this PR or you want to be extra cautious so I can fall back to trunk for this check ?
   
   The change LGTM, and we can keep it in this PR :smiley: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2405757923


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   Should we use the simple solution provided by gradle? 
   
   https://docs.gradle.org/current/userguide/upgrading_major_version_9.html#task_changes?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "dejan2609 (via GitHub)" <gi...@apache.org>.
dejan2609 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2406726608


##########
build.gradle:
##########
@@ -1264,6 +1268,27 @@ project(':core') {
     from(project(':streams:examples').jar) { into("libs/") }
     from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
     duplicatesStrategy 'exclude'
+    preserveFileTimestamps = true
+    filePermissions {

Review Comment:
   Update: I just pushed a commit that uses the upgrade manual suggestion (mentioned above) for permissions  and goes even further with turning of the reproducible builds feature for Gradle 9 build. 
   
   I assume this is actually a good and safer approach: we can create a separate JIRA ticket and turn on the reproducible builds (at some point in the future and as an isolated change).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3359161351

   > If I may suggest something (my 2 cents on this): we could opt to solve [KAFKA-19750](https://issues.apache.org/jira/browse/KAFKA-19750) separately (i.e. on a Gradle 8 level).
   Rationale: it would be much easier to spot that single isolated change (and revert it back if/when the time comes) rather than bundle it here.
   
   You are right. @brandboat Do you have free time to file a patch for it first?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #19513:
URL: https://github.com/apache/kafka/pull/19513#issuecomment-3368354081

   @dejan2609 please rebase code, thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19174 Gradle version upgrade 8 -->> 9 [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #19513:
URL: https://github.com/apache/kafka/pull/19513#discussion_r2410107053


##########
gradlew:
##########
@@ -1,7 +1,7 @@
 #!/bin/sh
 
 #
-# Copyright © 2015-2021 the original authors.

Review Comment:
   I have filed https://issues.apache.org/jira/browse/KAFKA-19756 to write down the correct steps for upgrading gradle



##########
.github/actions/setup-gradle/action.yml:
##########
@@ -42,7 +42,7 @@ runs:
         distribution: temurin
         java-version: ${{ inputs.java-version }}
     - name: Setup Gradle
-      uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
+      uses: gradle/actions/setup-gradle@748248ddd2a24f49513d8f472f81c3a07d4d50e1 # v4.4.4

Review Comment:
   Version 5.0.0 has been released. As a follow-up, I will file a minor patch to revisit all actions



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org