You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "Smallfu666 (via GitHub)" <gi...@apache.org> on 2026/07/24 15:13:51 UTC
[PR] [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses [gluten]
Smallfu666 opened a new pull request, #12623:
URL: https://github.com/apache/gluten/pull/12623
## What changes are proposed in this pull request?
Follow-up to #12610 (merged as 86df8560c). Apache Stash only supports exact-key restore and has no `restore-keys` mechanism. When `hashFiles('ep/build-velox/src/**')` changes for a PR branch, the exact Stash key misses and the native build starts cold. This PR adds a fallback to the legacy `actions/cache/restore` with the original `restore-keys` prefix, which can still pull the latest warm cache seeded on `main`.
### Design
```
Stash exact key hit → native build (fast)
Stash miss → actions/cache/restore with restore-keys → native build (still warm)
```
Each consumer restore now works in two steps:
1. `stash/restore` first (exact `ccache-<variant>-${{ hashFiles('ep/build-velox/src/**') }}` key);
2. if `stash-hit` is not `'true'`, fall back to `actions/cache/restore` with the pre-migration `restore-keys`, which pulls the latest cache seeded on main.
The existing `actions/cache/save` steps are kept, so both the Stash and actions/cache paths stay populated.
### Scope
- **Modified** (3 files, 57 additions): `velox_backend_arm.yml`, `velox_backend_enhanced.yml`, `velox_backend_x86.yml`
- **Not modified**: `velox_backend_cache.yml` (producer), `velox_nightly.yml`, `build_bundle_package.yml`, `flink.yml`, `velox_backend_ansi.yml`
The cache producer, nightly and bundle workflows, and self-contained caches (flink, ansi) are untouched — they keep the legacy `actions/cache` method as-is.
### Changes per consumer
| Consumer | Variants | Container job? |
|---|---|---|
| ARM | centos8 static, centos9 shared | yes (vcpkg-centos-9, centos-9-jdk8) |
| Enhanced | centos7 enhanced | no (host runner) |
| x86 | centos7 static, centos9 shared, cuDF centos9 shared | only centos9 shared is container |
Container jobs also add an unsupported-architecture fail-fast branch to the `gh`/`jq` bootstrap step.
## How was this patch tested?
- YAML parsing passes for all three modified workflow files.
- All 6 fallback restore keys and their `restore-keys` prefixes have been verified against the producer's `actions/cache/save` entries in `velox_backend_cache.yml`.
- No trailing whitespace, no unrelated formatting changes, no modifications outside the three scope files.
- **Forced-miss CI validation is in progress.** A temporary commit appends a `-forced-miss-validation` suffix to the six Stash keys to confirm the fallback executes correctly. Evidence (stash-hit status, fallback execution, build duration, ccache hit rate) will be added here once collected.
- **Not yet end-to-end PASS** — the fallback path's CI evidence is pending the forced-miss run.
### Relationship to other PRs
- #12599 (umbrella issue)
- #12602 (producer Stash seeding)
- #12610 (consumer Stash restore)
- PR #12619 currently contains an overlapping x86 centos7 fallback hunk used during its CI investigation. This PR centralizes the complete six-consumer fallback behavior; the overlapping hunk should be removed from #12619.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses [gluten]
Posted by "Smallfu666 (via GitHub)" <gi...@apache.org>.
Smallfu666 commented on PR #12623:
URL: https://github.com/apache/gluten/pull/12623#issuecomment-5072892690
@zhouyuan Thanks a lot for the review and for including the commit in #12615 — glad it was useful.
I’m still new to Gluten and currently learning the project through the Velox CI/cache work and the Spark–Velox correctness side. I’d also like to understand the project’s current direction better. Are there particular Velox backend areas or ongoing priorities where more contributor help would be especially useful?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses [gluten]
Posted by "Smallfu666 (via GitHub)" <gi...@apache.org>.
Smallfu666 closed pull request #12623: [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses
URL: https://github.com/apache/gluten/pull/12623
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses [gluten]
Posted by "zhouyuan (via GitHub)" <gi...@apache.org>.
zhouyuan commented on PR #12623:
URL: https://github.com/apache/gluten/pull/12623#issuecomment-5078126744
@Smallfu666
Welcome to the team! We’ve put together a great list of good first issues to help you get started:: https://github.com/apache/gluten/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22good%20first%20issue%22
If you're looking for a specific place to dive in, the ANSI support issue (#10134) is a good choice. You can start by reading the Spark code for those functions and then implementing them in the Velox backend.
https://github.com/apache/gluten/issues/10134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12599][VL] Fall back to actions/cache when Apache Stash misses [gluten]
Posted by "zhouyuan (via GitHub)" <gi...@apache.org>.
zhouyuan commented on PR #12623:
URL: https://github.com/apache/gluten/pull/12623#issuecomment-5072600114
@Smallfu666 thanks a lot for the contribution! I have included this commit in (https://github.com/apache/gluten/pull/12615) to speed up a bit on the rebase work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org