You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xtable.apache.org by "vinishjail97 (via GitHub)" <gi...@apache.org> on 2026/07/27 19:43:01 UTC

[PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

vinishjail97 opened a new pull request, #867:
URL: https://github.com/apache/incubator-xtable/pull/867

   Fixes #864
   
   ## What is the purpose of the pull request
   
   A source release must build from its own artifact, and right now it does not. While validating `0.4.0-incubating-rc1`, `./mvnw clean package` on the extracted source tarball fails in `xtable-service`:
   
   ```
   Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.2.12.Final:generate-code-tests
     (default) on project xtable-service: Quarkus code generation phase has failed:
     Failed to bootstrap application in TEST mode: Failed to resolve dependencies for
     org.apache.xtable:xtable-service:jar:0.4.0-incubating: The following artifacts could not be
     resolved: org.apache.xtable:xtable-core_2.12:jar:0.4.0-incubating (absent),
     org.apache.xtable:xtable-core_2.12:jar:tests:0.4.0-incubating (absent)
   ```
   
   `quarkus-maven-plugin` bootstraps the application in order to generate code, which means it resolves this module's dependencies through Quarkus' own bootstrap resolver instead of through the reactor. That resolver discovers the workspace from the raw poms, without interpolating properties, so the module declaring
   
   ```xml
   <artifactId>xtable-core_${scala.binary.version}</artifactId>
   ```
   
   is never matched against the `xtable-core_2.12` dependency. Quarkus then falls back to the remote repositories, where that artifact only exists once a release has been published — so the build of an unpublished release fails. The error names only `xtable-core_2.12`; `xtable-api` and `xtable-hudi-support-utils` have literal artifactIds and resolve from the reactor without trouble.
   
   This has been invisible because every build path we have runs `mvn install`, which puts `xtable-core` in the local repository and lets Quarkus' repository fallback succeed. Plain `mvn package` — what a release validator runs — is the one path never exercised.
   
   ## Brief change log
   
   - Set `-Dquarkus.bootstrap.effective-model-builder=true` in `.mvn/maven.config` so Quarkus' workspace discovery uses the effective model and resolves the property. Quarkus reads this only as a JVM system property (a pom `<properties>` entry has no effect, verified), and `.mvn/` already ships inside the source release tarball, so the setting travels with the release.
   - Documented the reason above the `quarkus-maven-plugin` declaration in `xtable-service/pom.xml`, since `maven.config` cannot carry comments.
   - Added a `package`-only CI job that first removes locally installed XTable artifacts, so the reactor is the only place they can come from. The `rm` matters: `cache: maven` restores a `~/.m2` that the `install` job has populated, which would otherwise hide exactly this failure again.
   
   ## Verify this pull request
   
   Verified by building with `package` only, in a tree whose version cannot be resolved from Maven Central or from a stale `~/.m2`:
   
   | Build | Result |
   | --- | --- |
   | rc1 source tarball, `clean package` | fails, reproduces #864 |
   | rc1 source tarball + the property as a pom `<properties>` entry | fails — confirms it must be a system property |
   | rc1 source tarball + `-Dquarkus.bootstrap.effective-model-builder=true` on the command line | succeeds |
   | rc1 source tarball + `.mvn/maven.config` | succeeds |
   | this branch with `.mvn/maven.config` removed | fails |
   | this branch as-is | succeeds |
   
   A matching PR targets `branch-0.4`, where this blocks the 0.4.0 release candidate.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Make the source release build with mvn package [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 merged PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3661030443


##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -43,5 +43,15 @@ jobs:
           distribution: 'temurin'
           cache: maven
 
+      # Anyone building the source release runs `mvn package` without a prior
+      # `mvn install`, so xtable-service has to resolve xtable-core from the
+      # reactor rather than from a repository. Delete locally installed XTable
+      # artifacts first, otherwise the ~/.m2 restored by `cache: maven` hides
+      # such a failure.
+      - name: Verify xtable-service builds without installed XTable artifacts
+        run: |
+          rm -rf ~/.m2/repository/org/apache/xtable
+          ./mvnw clean package -pl xtable-service -am -DskipTests -ntp -B
+

Review Comment:
   Good catch, thanks. I had only dealt with one of the two caches here.
   
   The `rm -rf` covers the dependency cache that `cache: maven` restores, but the maven-build-cache-extension from .mvn/extensions.xml is also active in CI and I had left it on. From the last run on this PR:
   
   ```
   [INFO] Cache configuration is not available at configured path .../.mvn/maven-build-cache-config.xml, cache is enabled with defaults
   [INFO] Saved Build to local file: /home/runner/.m2/build-cache/v1/org.apache.xtable/xtable/...
   ```
   
   If that ever restores a module instead of building it, the step passes without resolving xtable-core from the reactor at all, so the check quietly stops checking anything. Added `-Dmaven.build.cache.enabled=false` in bf970d38 so both caches are out of the way, and expanded the comment to say why each one is there.
   
   On moving it to its own workflow, that does give the same isolation structurally rather than through two flags someone has to keep, which I like. The reason it is a step here is that Vinish asked earlier in this PR to fold it into the existing xtable-service CI rather than run a second job on every push, and without `cache: maven` a separate workflow cold-downloads Spark, Hudi and Iceberg each time, roughly 4 to 6 minutes against the 73 seconds this step took in the last run. Happy to move it if you would still rather have the separate workflow, just say the word.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3660477616


##########
xtable-service/pom.xml:
##########
@@ -250,6 +250,15 @@
     </dependencies>
     <build>
         <plugins>
+            <!--

Review Comment:
   Shortened in a227a3e1 — down to four lines plus that link, thanks for the pointer. It now just says the plugin resolves its own dependencies and needs `effective-model-builder` to match `xtable-core_${scala.binary.version}` against the reactor, and that the property lives in `.mvn/maven.config` because Quarkus only reads it as a system property.



##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -45,3 +45,28 @@ jobs:
 
       - name: Build all module with Maven
         run: ./mvnw clean install -ntp -B
+
+  package:

Review Comment:
   Fair point, a second full build on every push and PR is too much. Scoped it down in a227a3e1: the separate job is gone and it is now a step in the existing build job, limited to `-pl xtable-service -am -DskipTests`, so it only builds xtable-api, xtable-hudi-support-utils, xtable-core and xtable-service, and skips the shade modules.
   
   I checked that the narrower reactor still catches the failure. With `.mvn/maven.config` removed, `./mvnw clean package -pl xtable-service -am -DskipTests` fails with `xtable-core_2.12:jar:<version> (absent)`; with it in place the same command succeeds.
   
   The `rm -rf ~/.m2/repository/org/apache/xtable` is the part that makes it meaningful. `cache: maven` restores a `~/.m2` that the `install` step populated on an earlier run, and Quarkus falls back to repository resolution, so without deleting those artifacts the check passes even when the source release is broken. That is why rc1 built fine for all of us locally.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3660800453


##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -43,5 +43,15 @@ jobs:
           distribution: 'temurin'
           cache: maven
 
+      # Anyone building the source release runs `mvn package` without a prior
+      # `mvn install`, so xtable-service has to resolve xtable-core from the
+      # reactor rather than from a repository. Delete locally installed XTable
+      # artifacts first, otherwise the ~/.m2 restored by `cache: maven` hides
+      # such a failure.
+      - name: Verify xtable-service builds without installed XTable artifacts
+        run: |
+          rm -rf ~/.m2/repository/org/apache/xtable
+          ./mvnw clean package -pl xtable-service -am -DskipTests -ntp -B
+

Review Comment:
   This could also be put in its own workflow where we don't leverage a build cache 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3660446968


##########
xtable-service/pom.xml:
##########
@@ -250,6 +250,15 @@
     </dependencies>
     <build>
         <plugins>
+            <!--

Review Comment:
   Shorten this comment and link this page? 
   https://quarkus.io/guides/maven-tooling#bootstrap-maven-properties 



##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -45,3 +45,28 @@ jobs:
 
       - name: Build all module with Maven
         run: ./mvnw clean install -ntp -B
+
+  package:

Review Comment:
   Do we really need this change here? Wouldn't it affect all the builds? Can we validate this as part of xtable-service CI only? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3661066385


##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -43,5 +43,15 @@ jobs:
           distribution: 'temurin'
           cache: maven
 
+      # Anyone building the source release runs `mvn package` without a prior
+      # `mvn install`, so xtable-service has to resolve xtable-core from the
+      # reactor rather than from a repository. Delete locally installed XTable
+      # artifacts first, otherwise the ~/.m2 restored by `cache: maven` hides
+      # such a failure.
+      - name: Verify xtable-service builds without installed XTable artifacts
+        run: |
+          rm -rf ~/.m2/repository/org/apache/xtable
+          ./mvnw clean package -pl xtable-service -am -DskipTests -ntp -B
+

Review Comment:
   Added `source-build.check` yaml



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [864] Build the source release with mvn package (Quarkus workspace discovery) [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #867:
URL: https://github.com/apache/incubator-xtable/pull/867#discussion_r3661030443


##########
.github/workflows/mvn-ci-build.yml:
##########
@@ -43,5 +43,15 @@ jobs:
           distribution: 'temurin'
           cache: maven
 
+      # Anyone building the source release runs `mvn package` without a prior
+      # `mvn install`, so xtable-service has to resolve xtable-core from the
+      # reactor rather than from a repository. Delete locally installed XTable
+      # artifacts first, otherwise the ~/.m2 restored by `cache: maven` hides
+      # such a failure.
+      - name: Verify xtable-service builds without installed XTable artifacts
+        run: |
+          rm -rf ~/.m2/repository/org/apache/xtable
+          ./mvnw clean package -pl xtable-service -am -DskipTests -ntp -B
+

Review Comment:
   Good catch, thanks. I had only dealt with one of the two caches here.
   
   The `rm -rf` covers the dependency cache that `cache: maven` restores, but the maven-build-cache-extension from .mvn/extensions.xml is also active in CI and I had left it on. From the last run on this PR:
   
   ```
   [INFO] Cache configuration is not available at configured path .../.mvn/maven-build-cache-config.xml, cache is enabled with defaults
   [INFO] Saved Build to local file: /home/runner/.m2/build-cache/v1/org.apache.xtable/xtable/...
   ```
   
   If that ever restores a module instead of building it, the step passes without resolving xtable-core from the reactor at all, so the check quietly stops checking anything. Added `-Dmaven.build.cache.enabled=false` in bf970d38 so both caches are out of the way, and expanded the comment to say why each one is there.
   
   On moving it to its own workflow, that does give the same isolation structurally rather than through two flags someone has to keep, which I like. The reason it is a step here is that Vinish asked earlier in this PR to fold it into the existing xtable-service CI rather than run a second job on every push, and without `cache: maven` a separate workflow cold-downloads Spark, Hudi and Iceberg each time, roughly 4 to 6 minutes against the 73 seconds this step took in the last run. Happy to move it if you would still rather have the separate workflow, just say the word.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org