You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "felipepessoto (via GitHub)" <gi...@apache.org> on 2026/08/25 06:10:26 UTC
[PR] [MINOR][CI] Publish safe nightly Delta baseline updates [gluten]
felipepessoto opened a new pull request, #12855:
URL: https://github.com/apache/gluten/pull/12855
## What changes are proposed in this pull request?
Automate maintenance of the Delta Spark UT `known-failures.txt` baseline from the scheduled daily run.
We recently had to remove now-passing tests manually in two separate PRs:
- [#12815](https://github.com/apache/gluten/pull/12815) removed four tests fixed by cast-mode preservation.
- [#12851](https://github.com/apache/gluten/pull/12851) removed two type-widening tests fixed by a Velox update.
Those fixes reached core libraries without triggering Delta CI when merged. The daily run eventually detected the now-passing tests, but updating the baseline still required a manual PR.
This change makes the scheduled workflow publish an automation-safe,
deletion-only baseline artifact. It removes tests confirmed as passing, never
adds new failures, and preserves skipped, unseen, and flaky entries as well as
existing comments. Regressions continue to fail the Delta gate and cannot be
accepted automatically.
Apache repositories do not permit `GITHUB_TOKEN` workflows to create pull
requests, so PR publication is intentionally kept outside this repository. A
lightweight workflow in `felipepessoto/gluten` consumes only the latest recent,
successful, scheduled `apache/gluten` run and maintains one fork branch and one
upstream PR. It skips failed, incomplete, non-scheduled, stale, unchanged, or
non-deletion-only results, avoiding duplicate Delta test runs and noisy PRs.
A focused unit test covers the pruning behavior, including preservation of regressions, skipped tests, stale entries, flaky tests, and baseline comments.
Related to #12743.
## How was this patch tested?
- Ran the `compare-test-results.py` unit test.
- Checked both Python files with Black.
- Validated `delta_spark_ut.yml` with actionlint.
- Ran `git diff --check`.
Validated the complete scheduled flow end to end in
[`felipepessoto/gluten`](https://github.com/felipepessoto/gluten):
- [Scheduled run #32615561149](https://github.com/felipepessoto/gluten/actions/runs/32615561149)
completed every build, test, aggregate, and update job successfully. The
aggregate reported zero regressions and two now-passing tests, with no
synthetic unit-test fixtures leaking into the job summary.
- The run produced exactly the two expected baseline deletions and no additions.
- The fork publisher's missing-credential guard was validated in
[run #32815741328](https://github.com/felipepessoto/gluten/actions/runs/32815741328):
it completed successfully without attempting to publish or creating noise.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.80
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5400717030
This is the option, if it is not available I'll close this PR
<img width="1597" height="705" alt="image" src="https://github.com/user-attachments/assets/39bb98b5-8a44-40d5-bb2e-36e0de45db76" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5406426609
I'll close it for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Publish safe nightly Delta baseline updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5406293684
Yes. I revised this PR so the Apache workflow no longer attempts to create a
PR; it only publishes the validated deletion-only artifact.
The publisher is now configured in my fork:
https://github.com/felipepessoto/gluten/blob/main/.github/workflows/delta_known_failures_publisher.yml
It does not duplicate the expensive Delta suite. It polls Apache's latest
scheduled `main` run and proceeds only when that run is:
- scheduled (never a PR/manual run),
- completed successfully,
- less than 36 hours old, and
- carrying the deletion-only artifact produced by this PR.
It independently rejects additions and comment/header removals, keeps one fixed
fork branch and one upstream PR, skips unchanged output, and closes a stale open
PR if the latest successful run has no removals. Two staggered schedules provide
a same-day attempt plus a previous-run fallback without repeatedly updating an
unchanged PR.
The fork is intentionally configured to allow only local workflows, so the
publisher uses a read-only `git fetch` rather than third-party actions. Its
missing-credential path was also verified as a successful no-op:
https://github.com/felipepessoto/gluten/actions/runs/32815741328
The remaining operational step is adding a dedicated `UPSTREAM_PR_TOKEN` secret
to the fork; the fork's `GITHUB_TOKEN` can push the update branch but cannot open
a PR in `apache/gluten`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "philo-he (via GitHub)" <gi...@apache.org>.
philo-he commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5400397454
> @philo-he is the option “Allow GitHub Actions to create and approve pull requests” enabled in the https://github.com/apache/gluten/settings/actions ?
@felipepessoto, I didn't find this option in the settings page. Perhaps, the action permission is not allowed to be set for Apache projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on code in PR #12855:
URL: https://github.com/apache/gluten/pull/12855#discussion_r3835571723
##########
.github/workflows/delta_spark_ut.yml:
##########
@@ -494,11 +498,112 @@ jobs:
--expected-shards "${{ env.DELTA_NUM_SHARDS }}" \
--known-failures .github/workflows/util/delta-spark-ut/known-failures.txt \
--flaky-tests .github/workflows/util/delta-spark-ut/flaky-tests.txt \
- --baseline-out aggregated/known-failures.txt
+ --baseline-out aggregated/known-failures.txt \
+ --pruned-baseline-out aggregated/pruned-known-failures.txt
Review Comment:
Good catch. Addressed in f7f0149ef by requiring both the Delta test matrix and aggregate job to succeed before the scheduled update job runs. I used the direct matrix gate rather than failing aggregation so failed nightlies still publish aggregate diagnostics/artifacts, while regressions and other shard failures all block PR automation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12855:
URL: https://github.com/apache/gluten/pull/12855#discussion_r3835553821
##########
.github/workflows/delta_spark_ut.yml:
##########
@@ -494,11 +498,112 @@ jobs:
--expected-shards "${{ env.DELTA_NUM_SHARDS }}" \
--known-failures .github/workflows/util/delta-spark-ut/known-failures.txt \
--flaky-tests .github/workflows/util/delta-spark-ut/flaky-tests.txt \
- --baseline-out aggregated/known-failures.txt
+ --baseline-out aggregated/known-failures.txt \
+ --pruned-baseline-out aggregated/pruned-known-failures.txt
Review Comment:
The scheduled workflow may still open/update the baseline PR even when the run has global regressions: `delta-spark-aggregate` does not fail on regressions by default, so `delta-spark-update-baseline` can run as long as aggregation succeeds. This can create a baseline-update PR that will be red/unmergeable due to the regressions, adding noise. Pass `--fail-on-regression true` (or otherwise gate the update job on a regression-free run) so the PR automation only runs on clean nightlies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "philo-he (via GitHub)" <gi...@apache.org>.
philo-he commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5402823957
> This is the option, if it is not available I'll close this PR
>
> <img alt="image" width="1597" height="705" src="https://private-user-images.githubusercontent.com/1336227/640518094-39bb98b5-8a44-40d5-bb2e-36e0de45db76.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODc2MTM1MjEsIm5iZiI6MTc4NzYxMzIyMSwicGF0aCI6Ii8xMzM2MjI3LzY0MDUxODA5NC0zOWJiOThiNS04YTQ0LTQwZDUtYmIyZS0zNmUwZGU0NWRiNzYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDgyNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjA4MjRUMjMxMzQxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjU1ZGYwZGQ2YTM0N2E1NGUzNTJjNzBjMDkxMTViNTEwZTU2ZTY0NzU2MzkxNWEzMTIyMGY5MTY5MGFiNjVjMiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmcmVzcG9uc2UtY29udGVudC10eXBlPWltYWdlJTJGcG5nIn0.JJeGoa_njtTwg5O_8KC0WmPEk9xuG4DJXJfPZufvFVc">
@felipepessoto, it is indeed unavailable in this Apache repo, although I can see it in my fork. Can the automated PR-creation workflow be configured in your Gluten fork repo? If so, it should be configured carefully to avoid false-positive triggers and excessive noisy PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on PR #12855:
URL: https://github.com/apache/gluten/pull/12855#issuecomment-5392197036
@philo-he is the option “Allow GitHub Actions to create and approve pull requests” in the https://github.com/apache/gluten/settings/actions ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto closed pull request #12855: [MINOR][CI] Automate nightly Delta known-failures updates
URL: https://github.com/apache/gluten/pull/12855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto closed pull request #12855: [MINOR][CI] Automate nightly Delta known-failures updates
URL: https://github.com/apache/gluten/pull/12855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][CI] Automate nightly Delta known-failures updates [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto closed pull request #12855: [MINOR][CI] Automate nightly Delta known-failures updates
URL: https://github.com/apache/gluten/pull/12855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org