You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zhouyuan (via GitHub)" <gi...@apache.org> on 2026/08/18 13:56:29 UTC
[PR] [GLUTEN-12288][CI] Move more jobs to nightly [gluten]
zhouyuan opened a new pull request, #12814:
URL: https://github.com/apache/gluten/pull/12814
<!--
Thank you for submitting a pull request! Here are some tips:
1. For first-time contributors, please read our contributing guide:
https://github.com/apache/gluten/blob/main/CONTRIBUTING.md
2. If necessary, create a GitHub issue for discussion beforehand to avoid duplicate work.
3. If the PR is specific to a single backend, include [VL] or [CH] in the PR title to indicate the
Velox or ClickHouse backend, respectively.
4. If the PR is not ready for review, please mark it as a draft.
-->
## What changes are proposed in this pull request?
Move more jobs to nightly tests
<!--
Provide a clear and concise description of the changes introduced in this PR.
Ensure the PR description aligns with the code changes, especially after updates.
If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the corresponding issue
when the PR is merged.
-->
## How was this patch tested?
pass GHA
<!--
Describe how the changes were tested, if applicable.
Include new tests to validate the functionality, if necessary.
For UI-related changes, attach screenshots to demonstrate the updates.
-->
## Was this patch authored or co-authored using generative AI tooling?
IBM Bob
<!--
If generative AI tooling has been used in the process of authoring this patch, please include the
phrase: 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12288][CI] Move more jobs to nightly [gluten]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12814:
URL: https://github.com/apache/gluten/pull/12814#discussion_r3805689625
##########
.github/workflows/velox_backend_x86.yml:
##########
@@ -103,6 +155,10 @@ jobs:
build-native-lib-centos-7:
needs: detect-changes
+ if: >-
+ needs.detect-changes.outputs.cpp == 'true' ||
+ needs.detect-changes.outputs.java == 'true' ||
+ needs.detect-changes.outputs.tools_it == 'true'
Review Comment:
This condition drops `build-native-lib-centos-7` for pure shims-only changes (e.g. `shims35=true` with `java=false`), which can inadvertently skip any downstream jobs that `needs: build-native-lib-centos-7` even though the workflow comments indicate shims-only PRs should still run a subset of spark jobs. Consider including the shims flags in this `if` (shims33/34/35/40/41) or introducing a single aggregated output (e.g. `needs_native_lib=true`) from `detect-changes` that covers all cases where the native artifact is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12288][CI] Move more jobs to nightly [gluten]
Posted by "zhouyuan (via GitHub)" <gi...@apache.org>.
zhouyuan merged PR #12814:
URL: https://github.com/apache/gluten/pull/12814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12288][CI] Move more jobs to nightly [gluten]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12814:
URL: https://github.com/apache/gluten/pull/12814#discussion_r3804869390
##########
.github/workflows/velox_backend_x86.yml:
##########
@@ -85,6 +106,16 @@ jobs:
- name: Detect changed paths
id: filter
run: |
+ if [ "${{ github.event_name }}" != "pull_request" ]; then
+ # schedule / workflow_dispatch: no PR diff to inspect -- run
+ # everything, including the extended matrices PR runs skip.
+ for flag in cpp java shims33 shims34 shims35 shims40 shims41 tools_it; do
+ echo "$flag=true" >> $GITHUB_OUTPUT
+ done
+ echo "full_run=true" >> $GITHUB_OUTPUT
+ exit 0
+ fi
Review Comment:
The comment says `full_run` is for schedule/workflow_dispatch runs, but the condition enables `full_run` for *any* non-`pull_request` event (including `push`). If this workflow runs on `push` (e.g., main/default branch), it will unintentionally trigger the expanded matrices and chaos jobs on every push/merge and largely negate the PR-time savings. Consider tightening the condition to only match `schedule` + `workflow_dispatch` (or explicitly exclude `push`), and keep `push` in the regular change-filtered path if applicable.
##########
.github/workflows/velox_backend_x86.yml:
##########
@@ -159,7 +200,16 @@ jobs:
matrix:
os: [ "ubuntu:22.04" ]
spark: [ "spark-3.3", "spark-3.4", "spark-3.5", "spark-4.0", "spark-4.1" ]
- java: [ "java-8", "java-11", "java-17", "java-21", "java-25" ]
+ # PR runs test the primary JDK per Spark line (8 for 3.x, 17 for 4.x --
+ # after the static excludes below that is 6 combos instead of 11). The
+ # alternative-JDK combos (11/21/25) validate build + TPC run under JDKs
+ # that Gluten changes rarely break in a JDK-specific way; the nightly
+ # full_run keeps covering them daily so a break surfaces within a day
+ # instead of costing ~75 runner-minutes on every PR.
+ java: >-
+ ${{ fromJSON(needs.detect-changes.outputs.full_run == 'true'
+ && '["java-8", "java-11", "java-17", "java-21", "java-25"]'
+ || '["java-8", "java-17"]') }}
Review Comment:
The workflow repeats several long `fromJSON(...)` matrix expressions (Spark versions, Java versions, Celeborn versions). This duplication makes it easier for the nightly vs PR matrices to drift over time. Consider centralizing these JSON lists (e.g., via reusable workflow inputs/outputs, job-level `env` JSON strings, or YAML anchors) so changes to supported versions only need to be made in one place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org