You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "snazy (via GitHub)" <gi...@apache.org> on 2025/08/22 16:56:14 UTC
[PR] Built: improve reproducible archive files [polaris]
snazy opened a new pull request, #2432:
URL: https://github.com/apache/polaris/pull/2432
As part of the effort for #2204, this change fixes a few aspects around reproducible builds:
Some Gradle projects produce archive files, but don't get the necessary Gradle archive-tasks settings applied: one not-published project but also the tarball&zip of the distribution. This change moves the logic to the new build-plugin `polaris-reproducible`.
Another change is to have some Quarkus generated jar files adhere to the same conventions, which are constant timestamps for the zip entries and a deterministic order of the entries. That's sadly not a full fix, as the classes that are generated or instumented by Quarkus differ in each build.
Contributes to #2204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #2432:
URL: https://github.com/apache/polaris/pull/2432#discussion_r2294266665
##########
build-logic/src/main/kotlin/Utilities.kt:
##########
@@ -61,3 +66,41 @@ fun JavaForkOptions.addSparkJvmOptions() {
"-Djdk.reflect.useDirectMethodHandle=false",
)
}
+
+/**
+ * Rewrites the given ZIP file.
+ *
+ * The timestamps of all entries are set to `1980-02-01 00:00`, zip entries appear in a
+ * deterministic order.
+ */
+fun makeZipReproducible(source: File) {
+ val t = FileTime.fromMillis(318211200_000) // 1980-02-01 00:00 GMT
Review Comment:
Gradle uses the same timestamp. IIRC it's got something to do with ZIP file format specialties...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b merged PR #2432:
URL: https://github.com/apache/polaris/pull/2432
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2432:
URL: https://github.com/apache/polaris/pull/2432#discussion_r2294269812
##########
build-logic/src/main/kotlin/Utilities.kt:
##########
@@ -61,3 +66,41 @@ fun JavaForkOptions.addSparkJvmOptions() {
"-Djdk.reflect.useDirectMethodHandle=false",
)
}
+
+/**
+ * Rewrites the given ZIP file.
+ *
+ * The timestamps of all entries are set to `1980-02-01 00:00`, zip entries appear in a
+ * deterministic order.
+ */
+fun makeZipReproducible(source: File) {
+ val t = FileTime.fromMillis(318211200_000) // 1980-02-01 00:00 GMT
Review Comment:
Ah, ok :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #2432:
URL: https://github.com/apache/polaris/pull/2432#discussion_r2294270253
##########
build-logic/src/main/kotlin/polaris-java.gradle.kts:
##########
@@ -237,9 +238,20 @@ configurations.all {
}
}
-// ensure jars conform to reproducible builds
-// (https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives)
-tasks.withType<AbstractArchiveTask>().configureEach {
- isPreserveFileTimestamps = false
- isReproducibleFileOrder = true
+if (plugins.hasPlugin("io.quarkus")) {
Review Comment:
We have a dedicated build-plugin for the "distributed" Quarkus builds, but also other projects that use/build with Quarkus.... So I put it into the non-java version specific one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2432:
URL: https://github.com/apache/polaris/pull/2432#discussion_r2294251931
##########
build-logic/src/main/kotlin/polaris-java.gradle.kts:
##########
@@ -237,9 +238,20 @@ configurations.all {
}
}
-// ensure jars conform to reproducible builds
-// (https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives)
-tasks.withType<AbstractArchiveTask>().configureEach {
- isPreserveFileTimestamps = false
- isReproducibleFileOrder = true
+if (plugins.hasPlugin("io.quarkus")) {
Review Comment:
Would it be worth moving it to `polaris-server.gradle.kts`?.. or perhaps make a separate plugin dedicated to quarkus?
##########
build-logic/src/main/kotlin/polaris-java.gradle.kts:
##########
@@ -237,9 +238,20 @@ configurations.all {
}
}
-// ensure jars conform to reproducible builds
-// (https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives)
-tasks.withType<AbstractArchiveTask>().configureEach {
- isPreserveFileTimestamps = false
- isReproducibleFileOrder = true
+if (plugins.hasPlugin("io.quarkus")) {
+ tasks.named("quarkusBuild") {
+ actions.addLast {
+ listOf(
+ "quarkus-app/quarkus-run.jar",
Review Comment:
Does Quarkus not respect `isReproducibleFileOrder`, etc.?
##########
build-logic/src/main/kotlin/Utilities.kt:
##########
@@ -61,3 +66,41 @@ fun JavaForkOptions.addSparkJvmOptions() {
"-Djdk.reflect.useDirectMethodHandle=false",
)
}
+
+/**
+ * Rewrites the given ZIP file.
+ *
+ * The timestamps of all entries are set to `1980-02-01 00:00`, zip entries appear in a
+ * deterministic order.
+ */
+fun makeZipReproducible(source: File) {
+ val t = FileTime.fromMillis(318211200_000) // 1980-02-01 00:00 GMT
Review Comment:
Why not `0` :wink:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Built: improve reproducible archive files [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #2432:
URL: https://github.com/apache/polaris/pull/2432#discussion_r2294267342
##########
build-logic/src/main/kotlin/polaris-java.gradle.kts:
##########
@@ -237,9 +238,20 @@ configurations.all {
}
}
-// ensure jars conform to reproducible builds
-// (https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives)
-tasks.withType<AbstractArchiveTask>().configureEach {
- isPreserveFileTimestamps = false
- isReproducibleFileOrder = true
+if (plugins.hasPlugin("io.quarkus")) {
+ tasks.named("quarkusBuild") {
+ actions.addLast {
+ listOf(
+ "quarkus-app/quarkus-run.jar",
Review Comment:
Ehm - nope. That's a Gradle thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org