You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2026/08/05 10:33:51 UTC
[PR] Refresh the image registry cache in a workflow that cannot be cancelled [airflow]
potiuk opened a new pull request, #71171:
URL: https://github.com/apache/airflow/pull/71171
`v3-3-test` builds its `linux/amd64` CI images from scratch, ~20 minutes per Python version, because the registry has no cache for them:
```
#20 importing cache manifest from ghcr.io/apache/airflow/v3-3-test/ci/python3.11:cache-linux-amd64
#20 ERROR: failed to configure registry cache importer:
ghcr.io/apache/airflow/v3-3-test/ci/python3.11:cache-linux-amd64: not found
```
The registry holds `cache-linux-amd64` for Python 3.10 alone on that branch, while `cache-linux-arm64` exists for all five.
### Why
A branch built on `push` refreshed its cache from inside the CI run, which cancels in progress on the next push. With no cache the build runs cold, which widens the window the next push cancels in, which leaves the cache missing. Run `30973271401` is the loop in one screenshot:
```
success 04:09–04:46 Build CI linux/amd64 image 3.10 … 3.14 (all five built)
success 04:58–05:17:11 Push CI Early:3.10 image cache ← the one tag that exists
cancelled 04:58–05:17:44 Push CI Early:3.11 image cache
cancelled 04:58–05:17:43 Push CI Early:3.13 image cache
cancelled 04:58–05:17:41 Push CI Early:3.12 image cache
cancelled 04:57–05:17:44 Push CI Early:3.14 image cache
```
The next push landed at 05:17:26 and killed four of the five, fifteen seconds after the first finished. Of the last 60 AMD runs on that branch, 45 were cancelled and 14 failed — so the Regular cache push in `finalize-tests.yml`, which sits behind the whole test matrix, never runs there either.
A job cannot opt out of its run being cancelled, so the refresh has to be a workflow of its own.
### What this does
**`refresh-image-registry-cache.yml`** — refreshes the CI cache for both platforms, on `workflow_dispatch` and on `push` to release-prep / providers branches. Push runs are never cancelled (a newer one queues behind); manual runs are grouped separately, so a merge cannot cancel one, and re-firing it replaces the run in flight.
**Every Python version, both paths.** Selective checks narrows `python-versions` to the default one on a push where only text files changed (`_should_run_all_tests_and_versions`), which is a second, independent way a branch ends up with cache for one version. The workflow forces the `all versions` label; a new test in `test_selective_checks.py` pins that.
**The early cache push is removed from the regular build.** It is what this workflow replaces, so `additional-ci-image-checks.yml` loses that job and the seven `workflow_call` inputs only it consumed (removed from both wrappers identically, so `check-ci-workflows-in-sync` stays green). `main` is unaffected: it is not built on push, and its Push Image Cache job at the end of the scheduled canary runs to completion.
**`refresh-constraints.yml`** — the manual constraints refresh, split out of `update-constraints-on-push.yml` so it is not filtered by that workflow's `paths: uv.lock` and appears under its own name. It calls that workflow rather than copying it, and refreshes all three flavours — source-providers, no-providers and PyPI — since refreshing only PyPI leaves CI and Breeze on the older pins. `release-constraints.yml` is untouched.
**Docs** — both procedures in `dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md` now lead with the workflows; the local buildx / qemu / `refresh_images.sh` route is kept but demoted to a fallback. The automated-CI section no longer describes the early cache push this PR deletes.
Also fixed in `dev/MANUALLY_BUILDING_IMAGES.md`: the merge step was documented as `docker release-management merge-prod-images` in both the regular and slim variants — it is a `breeze` subcommand, so the documented command fails as written.
### Follow-ups this does not do
- `v3-3-test` needs the workflow backported before its cache can be refreshed there — `workflow_dispatch` runs the file from the branch you select.
- `dev/MANUALLY_BUILDING_IMAGES.md` contradicts itself on the buildx version (a warning says "at least 0.13", the prose says 0.23 is needed for `docker-container`). Left alone rather than guessed at.
- The buildx setup instructions now exist in both dev docs and could be collapsed to one copy.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5)
Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Refresh the image registry cache in a workflow that cannot be cancelled [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #71171:
URL: https://github.com/apache/airflow/pull/71171#issuecomment-5190795036
OK Yolo :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Refresh the image registry cache in a workflow that cannot be cancelled [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #71171:
URL: https://github.com/apache/airflow/pull/71171
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Refresh the image registry cache in a workflow that cannot be cancelled [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #71171:
URL: https://github.com/apache/airflow/pull/71171#issuecomment-5190809242
### Backport failed to create: v3-3-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/30998950270'> Run details </a>
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>❌</td>
<td>v3-3-test</td>
<td><a href="https://github.com/apache/airflow/commit/e69c1881b32c36abb827bae3717eaf46424427bd"><img src='https://img.shields.io/badge/Commit-e69c188-red' alt='Commit Link'></a></td>
</tr>
</table>
You can attempt to backport this manually by running:
```bash
cherry_picker e69c188 v3-3-test
```
This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.
After you have resolved the conflicts, you can continue the backport process by running:
```bash
cherry_picker --continue
```
If you don't have cherry-picker installed, see the [installation guide](https://github.com/apache/airflow/blob/main/dev/README.md#how-to-backport-pr-with-cherry-picker-cli).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org