You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2025/05/04 14:02:38 UTC
(airflow) branch use-merge-for-multiplatform-images updated (9e6ff9d3e54 -> 19bdde82564)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch use-merge-for-multiplatform-images
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 9e6ff9d3e54 Separate building release images to separate hardware
new 19bdde82564 Separate building release images to separate hardware
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9e6ff9d3e54)
\
N -- N -- N refs/heads/use-merge-for-multiplatform-images (19bdde82564)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/release_dockerhub_image.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
(airflow) 01/01: Separate building release images to separate hardware
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch use-merge-for-multiplatform-images
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 19bdde82564f06f5228a9c1ba62868f9054cd99f
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sun May 4 15:18:09 2025 +0200
Separate building release images to separate hardware
So far, when we built release images in CI or locally we relied on
either the emulation or having a builder configured that could handle
having docker engines running at the same time on different hardware.
That had some disadvantages:
* for emulation, it was terribly slow.It was 10/15 times slower than building
appropriate platform on appropriate hardware - making our image relase
process taking more than hour. Also build output from both runs was
interleaved which made it rather difficult to diagnose problems
sometimes.
* for using local builder, you needed to setup and authenticate with
another docker engine, which required SSH connectivity and port
forwarding and it was really not feasible for GitHub Actions where
you cannot easily authenticate between separate runners and make sure
they are reachable from each other.
This PR implements different appreach - merging build images. Instead of
building them together, the images are built as ``digest-only`` images
and pushed to the registry and their metadata with digest is uploaded as
artifacts. Then a separate merge step downloads the metadata and creates
merged manifest where both images are referred to.
This allows to build each image separately on a separate hardware - way
faster in general (providing that we have an appropriate hypervisor
capable hardware for each platform that we can build the images on.
---
.github/workflows/release_dockerhub_image.yml | 169 ++++++----
dev/MANUALLY_BUILDING_IMAGES.md | 91 +++++-
dev/breeze/doc/09_release_management_tasks.rst | 82 ++++-
dev/breeze/doc/images/output-commands.svg | 6 +-
.../doc/images/output_release-management.svg | 20 +-
.../doc/images/output_release-management.txt | 2 +-
...output_release-management_merge-prod-images.svg | 166 ++++++++++
...output_release-management_merge-prod-images.txt | 1 +
...tput_release-management_release-prod-images.svg | 34 +-
...tput_release-management_release-prod-images.txt | 2 +-
.../output_setup_check-all-params-in-groups.svg | 2 +-
.../output_setup_check-all-params-in-groups.txt | 2 +-
.../output_setup_regenerate-command-images.svg | 2 +-
.../output_setup_regenerate-command-images.txt | 2 +-
dev/breeze/doc/images/output_shell.svg | 6 +-
dev/breeze/doc/images/output_shell.txt | 2 +-
dev/breeze/doc/images/output_start-airflow.svg | 6 +-
dev/breeze/doc/images/output_start-airflow.txt | 2 +-
.../output_testing_providers-integration-tests.svg | 4 +-
.../output_testing_providers-integration-tests.txt | 2 +-
.../commands/release_management_commands.py | 355 ++++++++++++++-------
.../commands/release_management_commands_config.py | 14 +
dev/breeze/src/airflow_breeze/global_constants.py | 1 +
.../airflow_breeze/utils/docker_command_utils.py | 53 +++
24 files changed, 802 insertions(+), 224 deletions(-)
diff --git a/.github/workflows/release_dockerhub_image.yml b/.github/workflows/release_dockerhub_image.yml
index e10670389c5..e3fcdca038b 100644
--- a/.github/workflows/release_dockerhub_image.yml
+++ b/.github/workflows/release_dockerhub_image.yml
@@ -46,11 +46,22 @@ jobs:
allPythonVersions: ${{ steps.selective-checks.outputs.all-python-versions }}
defaultPythonVersion: ${{ steps.selective-checks.outputs.default-python-version }}
skipLatest: ${{ steps.selective-checks.outputs.skip-latest }}
+ platform-matrix: ${{ steps.determine-matrix.outputs.platform-matrix }}
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
VERBOSE: true
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
AMD_ONLY: ${{ github.event.inputs.amdOnly }}
+ if: contains(fromJSON('[
+ "ashb",
+ "eladkal",
+ "ephraimbuddy",
+ "jedcunningham",
+ "kaxil",
+ "pierrejeambrun",
+ "potiuk",
+ "utkarsharma2"
+ ]'), github.event.sender.login)
steps:
- name: "Input parameters summary"
shell: bash
@@ -82,26 +93,27 @@ jobs:
env:
VERBOSE: "false"
run: breeze ci selective-check 2>> ${GITHUB_OUTPUT}
+ - name: "Determine build matrix"
+ shell: bash
+ id: determine-matrix
+ run: |
+ if [[ "${AMD_ONLY}" = "true" ]]; then
+ echo 'platform-matrix=["linux/amd64"]' >> "${GITHUB_OUTPUT}"
+ else
+ echo 'platform-matrix=["linux/amd64", "linux/arm64"]' >> "${GITHUB_OUTPUT}"
+ fi
- release-images:
- timeout-minutes: 120
- name: "Release images: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}"
- runs-on: ["ubuntu-22.04"]
+ build-images:
+ timeout-minutes: 50
+ # yamllint disable rule:line-length
+ name: "Build images: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}, ${{ matrix.platform }}"
+ runs-on: ${{ matrix.platform == 'linux/amd64' && fromJSON('["ubuntu-22.04"]') || fromJSON('["asf-arm"]') }}
needs: [build-info]
strategy:
fail-fast: false
matrix:
python-version: ${{ fromJSON(needs.build-info.outputs.pythonVersions) }}
- if: contains(fromJSON('[
- "ashb",
- "eladkal",
- "ephraimbuddy",
- "jedcunningham",
- "kaxil",
- "pierrejeambrun",
- "potiuk",
- "utkarsharma2"
- ]'), github.event.sender.login)
+ platform: ${{ fromJSON(needs.build-info.outputs.platform-matrix) }}
steps:
- name: "Cleanup repo"
shell: bash
@@ -145,79 +157,129 @@ jobs:
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt install docker-buildx-plugin
- - name: "Install emulation support"
- run: docker run --privileged --rm tonistiigi/binfmt --install all
- - name: "Install regctl"
- # yamllint disable rule:line-length
- run: |
- mkdir -p ~/bin
- curl -L https://github.com/regclient/regclient/releases/latest/download/regctl-linux-amd64 >${HOME}/bin/regctl
- chmod 755 ${HOME}/bin/regctl
- echo "${HOME}/bin" >>${GITHUB_PATH}
- name: "Create airflow_cache builder"
- run: docker buildx create --name airflow_cache
+ run: docker buildx create --name airflow_cache --driver docker-container
- name: >
- Release regular images AMD64 first: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
+ Build regular images: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}, ${{ matrix.platform }}
env:
COMMIT_SHA: ${{ github.sha }}
REPOSITORY: ${{ github.repository }}
PYTHON_VERSION: ${{ matrix.python-version }}
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
SKIP_LATEST: ${{ needs.build-info.outputs.skipLatest == 'true' && '--skip-latest' || '' }}
+ LIMIT_PLATFORM: ${{ matrix.platform }}
run: >
breeze release-management release-prod-images --dockerhub-repo "${REPOSITORY}"
--airflow-version "${AIRFLOW_VERSION}" ${SKIP_LATEST}
- --limit-python ${PYTHON_VERSION} --limit-platform linux/amd64
+ --limit-python ${PYTHON_VERSION}
+ --metadata-folder dist
- name: >
- Verify regular AMD64 image: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
+ Verify regular image: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}, ${{ matrix.platform }}
env:
PYTHON_VERSION: ${{ matrix.python-version }}
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
REPOSITORY: ${{ github.repository }}
run: >
- breeze prod-image verify
- --pull
- --image-name
+ breeze prod-image verify --pull --image-name
${REPOSITORY}:${AIRFLOW_VERSION}-python${PYTHON_VERSION}
- name: >
- Release slim images AMD64 first: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
+ Release slim images: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}, ${{ matrix.platform }}
env:
COMMIT_SHA: ${{ github.sha }}
REPOSITORY: ${{ github.repository }}
PYTHON_VERSION: ${{ matrix.python-version }}
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
SKIP_LATEST: ${{ needs.build-info.outputs.skipLatest == 'true' && '--skip-latest' || '' }}
+ LIMIT_PLATFORM: ${{ matrix.platform }}
run: >
breeze release-management release-prod-images --dockerhub-repo "${REPOSITORY}"
--airflow-version "${AIRFLOW_VERSION}" ${SKIP_LATEST}
- --limit-python ${PYTHON_VERSION} --slim-images --limit-platform linux/amd64
+ --limit-python ${PYTHON_VERSION} --slim-images
+ --metadata-folder dist
- name: >
- Verify slim AMD64 image: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
+ Verify slim image: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}, ${{ matrix.platform }}
env:
PYTHON_VERSION: ${{ matrix.python-version }}
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
REPOSITORY: ${{ github.repository }}
run: >
- breeze prod-image verify
- --pull
- --slim-image
- --image-name
+ breeze prod-image verify --pull --slim-image --image-name
${REPOSITORY}:slim-${AIRFLOW_VERSION}-python${PYTHON_VERSION}
- - name: >
- Release regular images for both AMD / ARM: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
- env:
- COMMIT_SHA: ${{ github.sha }}
- REPOSITORY: ${{ github.repository }}
- PYTHON_VERSION: ${{ matrix.python-version }}
- AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
- SKIP_LATEST: ${{ needs.build-info.outputs.skipLatest == 'true' && '--skip-latest' || '' }}
+ - name: "Upload metadata artifacts"
+ uses: actions/upload-artifact@v4
+ with:
+ name: metadata-${{ matrix.python-version }}-${{ matrix.platform }}
+ path: ./dist/metadata-*
+ retention-days: 7
+ if-no-files-found: error
+ - name: "Docker logout"
+ run: docker logout
+ if: always()
+
+ merge-images:
+ timeout-minutes: 5
+ name: "Merge images: ${{ github.event.inputs.airflowVersion }}"
+ runs-on: ["ubuntu-22.04"]
+ needs: [build-info, build-images]
+ strategy:
+ fail-fast: false
+ matrix:
+ python-version: ${{ fromJSON(needs.build-info.outputs.pythonVersions) }}
+ steps:
+ - name: "Cleanup repo"
+ shell: bash
+ run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"
+ - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
+ uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: "Cleanup docker"
+ run: ./scripts/ci/cleanup_docker.sh
+ - name: "Install Breeze"
+ uses: ./.github/actions/breeze
+ with:
+ use-uv: "false"
+ - name: Free space
+ run: breeze ci free-space --answer yes
+ - name: "Cleanup dist and context file"
+ run: rm -fv ./dist/* ./docker-context-files/*
+ - name: "Login to hub.docker.com"
run: >
- breeze release-management release-prod-images --dockerhub-repo "${REPOSITORY}"
- --airflow-version "${AIRFLOW_VERSION}" ${SKIP_LATEST}
- --limit-python ${PYTHON_VERSION}
- if: github.event.inputs.amdOnly != 'true'
- - name: >
- Release slim images for both AMD / ARM: ${{ github.event.inputs.airflowVersion }}, ${{ matrix.python-version }}
+ echo ${{ secrets.DOCKERHUB_TOKEN }} |
+ docker login --password-stdin --username ${{ secrets.DOCKERHUB_USER }}
+ - name: Login to ghcr.io
+ env:
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ ACTOR: ${{ github.actor }}
+ run: echo "${GITHUB_TOKEN}" | docker login ghcr.io -u ${ACTOR} --password-stdin
+ - name: "Download metadata artifacts"
+ uses: actions/download-artifact@v4
+ with:
+ name: metadata-${{ matrix.python-version }}-*
+ - name: "Install buildx plugin"
+ # yamllint disable rule:line-length
+ run: |
+ sudo apt-get update
+ sudo apt-get install ca-certificates curl
+ sudo install -m 0755 -d /etc/apt/keyrings
+ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+ sudo chmod a+r /etc/apt/keyrings/docker.asc
+
+ # Add the repository to Apt sources:
+ echo \
+ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+ sudo apt-get update
+ sudo apt install docker-buildx-plugin
+ - name: "Install regctl"
+ # yamllint disable rule:line-length
+ run: |
+ mkdir -p ~/bin
+ curl -L https://github.com/regclient/regclient/releases/latest/download/regctl-linux-amd64 >${HOME}/bin/regctl
+ chmod 755 ${HOME}/bin/regctl
+ echo "${HOME}/bin" >>${GITHUB_PATH}
+ - name: "Merge images"
env:
COMMIT_SHA: ${{ github.sha }}
REPOSITORY: ${{ github.repository }}
@@ -225,10 +287,9 @@ jobs:
AIRFLOW_VERSION: ${{ github.event.inputs.airflowVersion }}
SKIP_LATEST: ${{ needs.build-info.outputs.skipLatest == 'true' && '--skip-latest' || '' }}
run: >
- breeze release-management release-prod-images --dockerhub-repo "${REPOSITORY}"
+ breeze release-management merge-prod-images --dockerhub-repo "${REPOSITORY}"
--airflow-version "${AIRFLOW_VERSION}" ${SKIP_LATEST}
- --limit-python ${PYTHON_VERSION} --slim-images
- if: github.event.inputs.amdOnly != 'true'
+ --limit-python ${PYTHON_VERSION} --metadata-folder dist
- name: "Docker logout"
run: docker logout
if: always()
diff --git a/dev/MANUALLY_BUILDING_IMAGES.md b/dev/MANUALLY_BUILDING_IMAGES.md
index b0537fe6c5b..7df3717accf 100644
--- a/dev/MANUALLY_BUILDING_IMAGES.md
+++ b/dev/MANUALLY_BUILDING_IMAGES.md
@@ -24,7 +24,8 @@
- [Building docker images](#building-docker-images)
- [Keeping your docker context small](#keeping-your-docker-context-small)
- [Setting environment with emulation](#setting-environment-with-emulation)
-- [Setting up cache refreshing with hardware ARM/AMD support](#setting-up-cache-refreshing-with-hardware-armamd-support)
+- [Setting up airflow_cache builder for your local hardware](#setting-up-airflow_cache-builder-for-your-local-hardware)
+- [Building images separately on different hardware and merging them into multi-platform image](#building-images-separately-on-different-hardware-and-merging-them-into-multi-platform-image)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -76,26 +77,35 @@ More information can be found [here](https://docs.docker.com/engine/reference/co
However, emulation is very slow - more than 10x slower than hardware-backed builds.
-## Setting up cache refreshing with hardware ARM/AMD support
+## Setting up airflow_cache builder for your local hardware
-If you plan to build a number of images, it's probably better to set up a hardware remote builder
-for your ARM or AMD builds (depending which platform you build images on - the "other" platform should be
-remote).
+> [!WARNING]
+>
+> You need to have at least 0.13 version of the docker buildx plugin installed with your container engine to
+> perform manual release of the images. See https://github.com/docker/buildx?tab=readme-ov-file#installing
+> for instructions on how to install and upgrade the plugin.
-This can be achieved by settings build as described in
-[this blog post](https://www.docker.com/blog/speed-up-building-with-docker-buildx-and-graviton2-ec2/) and
-adding it to docker buildx `airflow_cache` builder.
+If you plan to build a number of images and have two hardware (AMD linux and ARM Mac for example available in
+the same network, you need t set up a hardware builder that will build them in parallel using both hardware
+options. This usually can be done with those two commands - providing that you have HOST:PORT of the
+remote hardware available to securely connect to.
-This usually can be done with those two commands:
+The benefit of this method is that the images can be built and pushed in a single step - there is no need
+to build and push them separately and perform the additional manifest merge step.
```bash
-docker buildx create --name airflow_cache # your local builder
+docker buildx create --name airflow_cache --driver docker-container unix:///var/run/docker.sock # your local builder (you might want to use
docker buildx create --name airflow_cache --append HOST:PORT # your remote builder
```
One of the ways to have HOST:PORT is to login to the remote machine via SSH and forward the port to
the docker engine running on the remote machine.
+Note that you need relatively new `buildx plugin` installed (0.23 is used at the moment of writing this
+in order to use `docker-container` driver - which is necessary to build multi-platform images).
+Images build with this driver are built in a separate container - and unless you use `--load` or `--push` options
+of building the image, they are not available in the local docker daemon.
+
When everything is fine you should see both local and remote builder configured and reporting status:
```bash
@@ -120,3 +130,64 @@ breeze release-management release-prod-images --airflow-version "${VERSION}" --s
This will wipe Breeze cache and docker-context-files in order to make sure the build is "clean". It
also performs image verification after pushing the images.
+
+
+## Building images separately on different hardware and merging them into multi-platform image
+
+You can also build your multi-platform images separately on different hardware and then push digest-only
+images that can later be merged with a manifest into a single multi-platform image. This is what
+we are using in the CI pipeline where we build our images on separate hardware instances and merge the images
+into a multi-platform image as a separate step.
+
+On each hardware you build you must set up the same `airflow_cache` docker-container builder as in the
+previous step - but you do not need to append the remote builder to the local one.
+
+```bash
+docker buildx create --name airflow_cache --driver docker-container unix:///var/run/docker.sock
+```
+
+You prepare the images separately by specifying the folder where digest-metadata of the images should be stored
+
+Preparing AMD image (on AMD hardware):
+
+```shell script
+breeze release-management release-prod-images --airflow-version "${VERSION}" --limit-platform linux/amd64 --metadata-folder dist
+```
+
+Preparing ARM image (on ARM hardware):
+
+```shell script
+breeze release-management release-prod-images --airflow-version "${VERSION}" --limit-platform linux/arm64 --metadata-folder dist
+```
+
+These commands will push the digest-only image to the registry and store the metadata in the specified folder
+with the `metadata-AIRFLOW_VERSION-linux_amd64-PYTHON_VERSION.json` and
+`metadata-AIRFLOW_VERSION-linux_arm64-PYTHON_VERSION.json` names (AIRFLOW_VERSION/PYTHON_VERSION replaced
+with appropriate versions).
+You can then merge the images into a single multi-platform image with:
+
+```bash
+docker release-management merge-prod-images --airflow-version "${VERSION}" --metadata-folder dist
+```
+
+The same can be repeated for slim images by adding `--slim-images` option to the command, the manifests are
+named `metadata-AIRFLOW_VERSION-slim-linux_amd64-PYTHON_VERSION.json` and
+`metadata-AIRFLOW_VERSION-slim-linux_arm64-PYTHON_VERSION.json`.
+
+Preparing AMD image (on AMD hardware):
+
+```shell script
+breeze release-management release-prod-images --slim-images --airflow-version "${VERSION}" --limit-platform linux/amd64 --metadata-folder dist
+```
+
+Preparing ARM image (on ARM hardware):
+
+```shell script
+breeze release-management release-prod-images --slim-images --airflow-version "${VERSION}" --limit-platform linux/arm64 --metadata-folder dist
+```
+
+Merging the images:
+
+```bash
+docker release-management merge-prod-images --slim-images --airflow-version "${VERSION}" --metadata-folder dist
+```
diff --git a/dev/breeze/doc/09_release_management_tasks.rst b/dev/breeze/doc/09_release_management_tasks.rst
index 75ad444c9a7..5575cc9ea71 100644
--- a/dev/breeze/doc/09_release_management_tasks.rst
+++ b/dev/breeze/doc/09_release_management_tasks.rst
@@ -163,7 +163,9 @@ Releasing Production images
"""""""""""""""""""""""""""
The **Production image** can be released by release managers who have permissions to push the image. This
-happens only when there is an RC candidate or final version of Airflow released.
+happens only when there is an RC candidate or final version of Airflow released. Normally it happens in CI,
+via ("Release PROD images" workflow that uses appropriate breeze commands, but you can also do it locally,
+providing that you are release manager and you have write access to the DockerHub registry.
You release "regular" and "slim" images as separate steps.
@@ -171,16 +173,53 @@ Releasing "regular" images:
.. code-block:: bash
- breeze release-management release-prod-images --airflow-version 2.4.0
+ breeze release-management release-prod-images --airflow-version 3.0.0
Or "slim" images:
.. code-block:: bash
- breeze release-management release-prod-images --airflow-version 2.4.0 --slim-images
+ breeze release-management release-prod-images --airflow-version 3.0.0 --slim-images
By default when you are releasing the "final" image, we also tag image with "latest" tags but this
-step can be skipped if you pass the ``--skip-latest`` flag.
+step can be skipped if you pass the ``--skip-latest`` flag (last python version is always tagged
+as non-versioned image). For example:
+
+* airflow-3.0.0-python3.12 is tagged as airflow-3.0.0
+
+If `airflow-3.0.0' is the latest version:
+
+* airflow-3.0.0-python3.12 is tagged as airflow-latest
+* airflow-3.0.0-python3.12 is tagged as airflow-latest-python3.12
+* airflow-3.0.0-python3.11 is tagged as airflow-latest-python3.11
+
+and so on.
+
+This command by default uses either emulation or you need to have driver configured to be able to build
+a multi-platform image on your local machine - but this might be long (for emulation) or a bit complex to
+setup (to have multi-hardware support in your buildx driver). The steps to do so are described in the
+`MANUALLY_BUILDING_IMAGES.md <../../MANUALLY_BUILDING_IMAGES.md>`__ document.
+
+However you can also use the ``--metadata-folder`` flag to specify the folder with metadata files where
+information about the image is stored and build the images separately on each hardware. The images are pushed
+to the registry as ``digest-only`` images without tags and digest information is stored in the locally generated
+metadata files. You can then transfer the metadata files generated on different hardware to a single machine
+and run ``breeze merge-production-images`` command below to merge and publish such
+multi-platform image. Again - details of this process are described in the same
+`MANUALLY_BUILDING_IMAGES.md <../../MANUALLY_BUILDING_IMAGES.md>`__ document. In case ``--metadata-folder``,
+tagging is skipped and it is only performed when you merge the images.
+
+Releasing "regular" images:
+
+.. code-block:: bash
+
+ breeze release-management release-prod-images --airflow-version 3.0.0 --metadata-folder dist
+
+Or "slim" images:
+
+.. code-block:: bash
+
+ breeze release-management release-prod-images --airflow-version 3.0.0 --slim-images --metadata-folder dist
These are all of the available flags for the ``release-prod-images`` command:
@@ -189,6 +228,41 @@ These are all of the available flags for the ``release-prod-images`` command:
:width: 100%
:alt: Breeze release management release prod images
+Merging Production images
+"""""""""""""""""""""""""
+
+As described in the previous step, when you are building images separately - on separate hardware (ARM separately
+and AMD separately), you push images as ``digest-only`` images without tags and digest information is stored
+in the locally generated metadata files. You can then transfer the metadata files generated on different
+hardware to a single machine and run ``breeze merge-production-images`` command below to merge and publish
+such multi-platform image.
+
+You merge "regular" and "slim" images as separate steps.
+
+Merging "regular" images (after getting all the metadata files in the metadata folder):
+
+.. code-block:: bash
+
+ breeze release-management merge-prod-images --airflow-version 3.0.0 --metadata-folder dist
+
+Or "slim" images:
+
+.. code-block:: bash
+
+ breeze release-management release-prod-images --airflow-version 2.4.0 --slim-images dist
+
+By default when you are releasing the "final" image, we also tag image with "latest" tags but this
+step can be skipped if you pass the ``--skip-latest`` flag.
+
+These are all of the available flags for the ``merge-prod-images`` command:
+
+.. image:: ./images/output_release-management_merge-prod-images.svg
+ :target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/doc/images/output_release-management_merge-prod-images.svg
+ :width: 100%
+ :alt: Breeze release management merge prod images
+
+The images are also aliased in dockerhub as appropriate.
+
Adding git tags for providers
"""""""""""""""""""""""""""""
diff --git a/dev/breeze/doc/images/output-commands.svg b/dev/breeze/doc/images/output-commands.svg
index b40f3cbf9b6..85fd2d8fb12 100644
--- a/dev/breeze/doc/images/output-commands.svg
+++ b/dev/breeze/doc/images/output-commands.svg
@@ -323,9 +323,9 @@
</text><text class="breeze-help-r5" x="0" y="215.2" textLength="12.2" clip-path="url(#breeze-help-line-8)">│</text><text class="breeze-help-r4" x="24.4" y="215.2" textLength="158.6" clip-path="url(#breeze-help-line-8)">--integration</text><text class="breeze-help-r1" x="768.6" y="215.2" textLength="671" clip-path="url(#breeze-help-line-8)">Core Integrations to enable when running (can be more  </text><text class="breeze-help-r5" x="1451.8 [...]
</text><text class="breeze-help-r5" x="0" y="239.6" textLength="12.2" clip-path="url(#breeze-help-line-9)">│</text><text class="breeze-help-r1" x="768.6" y="239.6" textLength="671" clip-path="url(#breeze-help-line-9)">than one).                                           & [...]
</text><text class="breeze-help-r5" x="0" y="264" textLength="12.2" clip-path="url(#breeze-help-line-10)">│</text><text class="breeze-help-r7" x="768.6" y="264" textLength="671" clip-path="url(#breeze-help-line-10)">(all | all-testable | cassandra | celery | drill |     </text><text class="breeze-help-r5" x="1451.8" y="264" textLength="12.2" clip-path="url(#breeze-help-line-10)">│</text><text class="breeze-help-r1" x=" [...]
-</text><text class="breeze-help-r5" x="0" y="288.4" textLength="12.2" clip-path="url(#breeze-help-line-11)">│</text><text class="breeze-help-r7" x="768.6" y="288.4" textLength="671" clip-path="url(#breeze-help-line-11)">gremlin | kafka | kerberos | keycloak | mongo | mssql |</text><text class="breeze-help-r5" x="1451.8" y="288.4" textLength="12.2" clip-path="url(#breeze-help-line-11)">│</text><text class="breeze-help-r1" x="1464" y=" [...]
-</text><text class="breeze-help-r5" x="0" y="312.8" textLength="12.2" clip-path="url(#breeze-help-line-12)">│</text><text class="breeze-help-r7" x="768.6" y="312.8" textLength="671" clip-path="url(#breeze-help-line-12)">openlineage | otel | pinot | qdrant | redis | statsd | </text><text class="breeze-help-r5" x="1451.8" y="312.8" textLength="12.2" clip-path="url(#breeze-help-line-12)">│</text><text class="breeze-help-r1" x="1464 [...]
-</text><text class="breeze-help-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-help-line-13)">│</text><text class="breeze-help-r7" x="768.6" y="337.2" textLength="671" clip-path="url(#breeze-help-line-13)">trino | ydb)                                          [...]
+</text><text class="breeze-help-r5" x="0" y="288.4" textLength="12.2" clip-path="url(#breeze-help-line-11)">│</text><text class="breeze-help-r7" x="768.6" y="288.4" textLength="671" clip-path="url(#breeze-help-line-11)">gremlin | kafka | kafka | kerberos | keycloak | mongo |</text><text class="breeze-help-r5" x="1451.8" y="288.4" textLength="12.2" clip-path="url(#breeze-help-line-11)">│</text><text class="breeze-help-r1" x="1464" y=" [...]
+</text><text class="breeze-help-r5" x="0" y="312.8" textLength="12.2" clip-path="url(#breeze-help-line-12)">│</text><text class="breeze-help-r7" x="768.6" y="312.8" textLength="671" clip-path="url(#breeze-help-line-12)">mssql | openlineage | otel | pinot | qdrant | redis |  </text><text class="breeze-help-r5" x="1451.8" y="312.8" textLength="12.2" clip-path="url(#breeze-help-line-12)">│</text><text class="breeze-help-r1" x= [...]
+</text><text class="breeze-help-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-help-line-13)">│</text><text class="breeze-help-r7" x="768.6" y="337.2" textLength="671" clip-path="url(#breeze-help-line-13)">statsd | trino | ydb)                                  </text><text class="breeze- [...]
</text><text class="breeze-help-r5" x="0" y="361.6" textLength="12.2" clip-path="url(#breeze-help-line-14)">│</text><text class="breeze-help-r4" x="24.4" y="361.6" textLength="317.2" clip-path="url(#breeze-help-line-14)">--standalone-dag-processor</text><text class="breeze-help-r1" x="341.6" y="361.6" textLength="12.2" clip-path="url(#breeze-help-line-14)">/</text><text class="breeze-help-r4" x="353.8" y="361.6" textLength="341.6" clip-path="url(#breeze-help-line-14)">--no-standalone-dag [...]
</text><text class="breeze-help-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-help-line-15)">│</text><text class="breeze-help-r4" x="24.4" y="386" textLength="12.2" clip-path="url(#breeze-help-line-15)">r</text><text class="breeze-help-r1" x="768.6" y="386" textLength="671" clip-path="url(#breeze-help-line-15)">(required for Airflow 3).                      [...]
</text><text class="breeze-help-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-help-line-16)">│</text><text class="breeze-help-r5" x="768.6" y="410.4" textLength="671" clip-path="url(#breeze-help-line-16)">[default: standalone-dag-processor]                    </text><text class="breeze-help-r5" x="1451.8" y="410.4" textLength="12.2" clip-path="url(#breeze-help-line-16)" [...]
diff --git a/dev/breeze/doc/images/output_release-management.svg b/dev/breeze/doc/images/output_release-management.svg
index 9884c14bd41..96b0ef7e1b0 100644
--- a/dev/breeze/doc/images/output_release-management.svg
+++ b/dev/breeze/doc/images/output_release-management.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 1123.6" xmlns="http://www.w3.org/2000/svg">
+<svg class="rich-terminal" viewBox="0 0 1482 1172.3999999999999" xmlns="http://www.w3.org/2000/svg">
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -42,7 +42,7 @@
<defs>
<clipPath id="breeze-release-management-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="1072.6" />
+ <rect x="0" y="0" width="1463.0" height="1121.3999999999999" />
</clipPath>
<clipPath id="breeze-release-management-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -173,9 +173,15 @@
<clipPath id="breeze-release-management-line-42">
<rect x="0" y="1026.3" width="1464" height="24.65"/>
</clipPath>
+<clipPath id="breeze-release-management-line-43">
+ <rect x="0" y="1050.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-line-44">
+ <rect x="0" y="1075.1" width="1464" height="24.65"/>
+ </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="1121.6" rx="8"/><text class="breeze-release-management-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="1170.4" rx="8"/><text class="breeze-release-management-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -226,9 +232,11 @@
</text><text class="breeze-release-management-r5" x="0" y="947.2" textLength="12.2" clip-path="url(#breeze-release-management-line-38)">│</text><text class="breeze-release-management-r4" x="24.4" y="947.2" textLength="292.8" clip-path="url(#breeze-release-management-line-38)">publish-docs-to-s3      </text><text class="breeze-release-management-r1" x="341.6" y="947.2" textLength="1098" clip-path="url(#breeze-release-management-line-38)">Publishes docs&# [...]
</text><text class="breeze-release-management-r5" x="0" y="971.6" textLength="1464" clip-path="url(#breeze-release-management-line-39)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-r1" x="1464" y="971.6" textLength="12.2" clip-path="url(#breeze-release-management-line-39)">
</text><text class="breeze-release-management-r5" x="0" y="996" textLength="24.4" clip-path="url(#breeze-release-management-line-40)">╭─</text><text class="breeze-release-management-r5" x="24.4" y="996" textLength="122" clip-path="url(#breeze-release-management-line-40)"> Commands </text><text class="breeze-release-management-r5" x="146.4" y="996" textLength="1293.2" clip-path="url(#breeze-release-management-line-40)">──────────────────────────────────────────────────────────── [...]
-</text><text class="breeze-release-management-r5" x="0" y="1020.4" textLength="12.2" clip-path="url(#breeze-release-management-line-41)">│</text><text class="breeze-release-management-r4" x="24.4" y="1020.4" textLength="549" clip-path="url(#breeze-release-management-line-41)">prepare-airflow-ctl-distributions            </text><text class="breeze-release-management-r1" x="597.8" y="1020.4" textLength="841.8" clip-path="url(#bree [...]
-</text><text class="breeze-release-management-r5" x="0" y="1044.8" textLength="12.2" clip-path="url(#breeze-release-management-line-42)">│</text><text class="breeze-release-management-r4" x="24.4" y="1044.8" textLength="549" clip-path="url(#breeze-release-management-line-42)">prepare-task-sdk-distributions               </text><text class="breeze-release-management-r1" x="597.8" y="1044.8" textLength="841.8" clip- [...]
-</text><text class="breeze-release-management-r5" x="0" y="1069.2" textLength="1464" clip-path="url(#breeze-release-management-line-43)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-r1" x="1464" y="1069.2" textLength="12.2" clip-path="url(#breeze-release-management-line-43)">
+</text><text class="breeze-release-management-r5" x="0" y="1020.4" textLength="12.2" clip-path="url(#breeze-release-management-line-41)">│</text><text class="breeze-release-management-r4" x="24.4" y="1020.4" textLength="402.6" clip-path="url(#breeze-release-management-line-41)">merge-prod-images                </text><text class="breeze-release-management-r1" x="451.4" y="1020.4" textLength="988.2" clip-path= [...]
+</text><text class="breeze-release-management-r5" x="0" y="1044.8" textLength="12.2" clip-path="url(#breeze-release-management-line-42)">│</text><text class="breeze-release-management-r1" x="451.4" y="1044.8" textLength="988.2" clip-path="url(#breeze-release-management-line-42)">permissions).                                 & [...]
+</text><text class="breeze-release-management-r5" x="0" y="1069.2" textLength="12.2" clip-path="url(#breeze-release-management-line-43)">│</text><text class="breeze-release-management-r4" x="24.4" y="1069.2" textLength="402.6" clip-path="url(#breeze-release-management-line-43)">prepare-airflow-ctl-distributions</text><text class="breeze-release-management-r1" x="451.4" y="1069.2" textLength="988.2" clip-path="url(#breeze-release-management-line-43)">Prepare sdist/whl distributi [...]
+</text><text class="breeze-release-management-r5" x="0" y="1093.6" textLength="12.2" clip-path="url(#breeze-release-management-line-44)">│</text><text class="breeze-release-management-r4" x="24.4" y="1093.6" textLength="402.6" clip-path="url(#breeze-release-management-line-44)">prepare-task-sdk-distributions   </text><text class="breeze-release-management-r1" x="451.4" y="1093.6" textLength="988.2" clip-path="url(#breeze-release-management-line-44)">Prepare sdist/whl& [...]
+</text><text class="breeze-release-management-r5" x="0" y="1118" textLength="1464" clip-path="url(#breeze-release-management-line-45)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-r1" x="1464" y="1118" textLength="12.2" clip-path="url(#breeze-release-management-line-45)">
</text>
</g>
</g>
diff --git a/dev/breeze/doc/images/output_release-management.txt b/dev/breeze/doc/images/output_release-management.txt
index 7f5fa81212c..78baffd63e9 100644
--- a/dev/breeze/doc/images/output_release-management.txt
+++ b/dev/breeze/doc/images/output_release-management.txt
@@ -1 +1 @@
-ef56ff9053bf03113e052c304f832bc5
+ff286e72725eee4f8f720f4b95e14839
diff --git a/dev/breeze/doc/images/output_release-management_merge-prod-images.svg b/dev/breeze/doc/images/output_release-management_merge-prod-images.svg
new file mode 100644
index 00000000000..745e3032550
--- /dev/null
+++ b/dev/breeze/doc/images/output_release-management_merge-prod-images.svg
@@ -0,0 +1,166 @@
+<svg class="rich-terminal" viewBox="0 0 1482 684.4" xmlns="http://www.w3.org/2000/svg">
+ <!-- Generated with Rich https://www.textualize.io -->
+ <style>
+
+ @font-face {
+ font-family: "Fira Code";
+ src: local("FiraCode-Regular"),
+ url("https://cdnjs.cloudflare.com/ajax/libs/firacode/6.2.0/woff2/FiraCode-Regular.woff2") format("woff2"),
+ url("https://cdnjs.cloudflare.com/ajax/libs/firacode/6.2.0/woff/FiraCode-Regular.woff") format("woff");
+ font-style: normal;
+ font-weight: 400;
+ }
+ @font-face {
+ font-family: "Fira Code";
+ src: local("FiraCode-Bold"),
+ url("https://cdnjs.cloudflare.com/ajax/libs/firacode/6.2.0/woff2/FiraCode-Bold.woff2") format("woff2"),
+ url("https://cdnjs.cloudflare.com/ajax/libs/firacode/6.2.0/woff/FiraCode-Bold.woff") format("woff");
+ font-style: bold;
+ font-weight: 700;
+ }
+
+ .breeze-release-management-merge-prod-images-matrix {
+ font-family: Fira Code, monospace;
+ font-size: 20px;
+ line-height: 24.4px;
+ font-variant-east-asian: full-width;
+ }
+
+ .breeze-release-management-merge-prod-images-title {
+ font-size: 18px;
+ font-weight: bold;
+ font-family: arial;
+ }
+
+ .breeze-release-management-merge-prod-images-r1 { fill: #c5c8c6 }
+.breeze-release-management-merge-prod-images-r2 { fill: #d0b344 }
+.breeze-release-management-merge-prod-images-r3 { fill: #c5c8c6;font-weight: bold }
+.breeze-release-management-merge-prod-images-r4 { fill: #68a0b3;font-weight: bold }
+.breeze-release-management-merge-prod-images-r5 { fill: #868887 }
+.breeze-release-management-merge-prod-images-r6 { fill: #cc555a }
+.breeze-release-management-merge-prod-images-r7 { fill: #8d7b39 }
+.breeze-release-management-merge-prod-images-r8 { fill: #8a4346 }
+.breeze-release-management-merge-prod-images-r9 { fill: #98a84b;font-weight: bold }
+ </style>
+
+ <defs>
+ <clipPath id="breeze-release-management-merge-prod-images-clip-terminal">
+ <rect x="0" y="0" width="1463.0" height="633.4" />
+ </clipPath>
+ <clipPath id="breeze-release-management-merge-prod-images-line-0">
+ <rect x="0" y="1.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-1">
+ <rect x="0" y="25.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-2">
+ <rect x="0" y="50.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-3">
+ <rect x="0" y="74.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-4">
+ <rect x="0" y="99.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-5">
+ <rect x="0" y="123.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-6">
+ <rect x="0" y="147.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-7">
+ <rect x="0" y="172.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-8">
+ <rect x="0" y="196.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-9">
+ <rect x="0" y="221.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-10">
+ <rect x="0" y="245.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-11">
+ <rect x="0" y="269.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-12">
+ <rect x="0" y="294.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-13">
+ <rect x="0" y="318.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-14">
+ <rect x="0" y="343.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-15">
+ <rect x="0" y="367.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-16">
+ <rect x="0" y="391.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-17">
+ <rect x="0" y="416.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-18">
+ <rect x="0" y="440.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-19">
+ <rect x="0" y="465.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-20">
+ <rect x="0" y="489.5" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-21">
+ <rect x="0" y="513.9" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-22">
+ <rect x="0" y="538.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-23">
+ <rect x="0" y="562.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-merge-prod-images-line-24">
+ <rect x="0" y="587.1" width="1464" height="24.65"/>
+ </clipPath>
+ </defs>
+
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="682.4" rx="8"/><text class="breeze-release-management-merge-prod-images-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management merge-prod-images</text>
+ <g transform="translate(26,22)">
+ <circle cx="0" cy="0" r="7" fill="#ff5f57"/>
+ <circle cx="22" cy="0" r="7" fill="#febc2e"/>
+ <circle cx="44" cy="0" r="7" fill="#28c840"/>
+ </g>
+
+ <g transform="translate(9, 41)" clip-path="url(#breeze-release-management-merge-prod-images-clip-terminal)">
+
+ <g class="breeze-release-management-merge-prod-images-matrix">
+ <text class="breeze-release-management-merge-prod-images-r1" x="1464" y="20" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-0)">
+</text><text class="breeze-release-management-merge-prod-images-r2" x="12.2" y="44.4" textLength="73.2" clip-path="url(#breeze-release-management-merge-prod-images-line-1)">Usage:</text><text class="breeze-release-management-merge-prod-images-r3" x="97.6" y="44.4" textLength="524.6" clip-path="url(#breeze-release-management-merge-prod-images-line-1)">breeze release-management merge-prod-images</text><text class="breeze-release-management-merge-prod-images-r1" x="634.4" y="44.4" [...]
+</text><text class="breeze-release-management-merge-prod-images-r1" x="1464" y="68.8" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-2)">
+</text><text class="breeze-release-management-merge-prod-images-r1" x="12.2" y="93.2" textLength="1085.8" clip-path="url(#breeze-release-management-merge-prod-images-line-3)">Merge production images in DockerHub based on digest files (needs DockerHub permissions).</text><text class="breeze-release-management-merge-prod-images-r1" x="1464" y="93.2" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-3)">
+</text><text class="breeze-release-management-merge-prod-images-r1" x="1464" y="117.6" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-4)">
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="142" textLength="24.4" clip-path="url(#breeze-release-management-merge-prod-images-line-5)">╭─</text><text class="breeze-release-management-merge-prod-images-r5" x="24.4" y="142" textLength="292.8" clip-path="url(#breeze-release-management-merge-prod-images-line-5)"> Merge PROD IMAGE flags </text><text class="breeze-release-management-merge-prod-images-r5" x="317.2" y="142" textLength="1 [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="166.4" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-6)">│</text><text class="breeze-release-management-merge-prod-images-r6" x="24.4" y="166.4" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-6)">*</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="166.4" textLength="207.4" clip-path="url(#breeze-release-management [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="190.8" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-7)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="190.8" textLength="195.2" clip-path="url(#breeze-release-management-merge-prod-images-line-7)">--dockerhub-repo</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="190.8" textLength="427" clip-path="url(#breeze-rel [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="215.2" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-8)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="215.2" textLength="170.8" clip-path="url(#breeze-release-management-merge-prod-images-line-8)">--limit-python</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="215.2" textLength="1122.4" clip-path="url(#breeze-re [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="239.6" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-9)">│</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="239.6" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-9)">available python versions)                   [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="264" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-10)">│</text><text class="breeze-release-management-merge-prod-images-r7" x="317.2" y="264" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-10)">(3.9 | 3.10 | 3.11 | 3.12)                [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="288.4" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-11)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="288.4" textLength="158.6" clip-path="url(#breeze-release-management-merge-prod-images-line-11)">--skip-latest</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="288.4" textLength="1122.4" clip-path="url(#breeze-r [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="312.8" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-12)">│</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="312.8" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-12)">should only be used if you release image for previous branches. Automatically set [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-13)">│</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="337.2" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-13)">rc/alpha/beta images are built.                 [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="361.6" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-14)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="361.6" textLength="158.6" clip-path="url(#breeze-release-management-merge-prod-images-line-14)">--slim-images</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="361.6" textLength="719.8" clip-path="url(#breeze-re [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-15)">│</text><text class="breeze-release-management-merge-prod-images-r6" x="24.4" y="386" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-15)">*</text><text class="breeze-release-management-merge-prod-images-r4" x="61" y="386" textLength="207.4" clip-path="url(#breeze-release-management-mer [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-16)">│</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="410.4" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-16)">registryonly by digests not by the tag because we are going to merge several [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="434.8" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-17)">│</text><text class="breeze-release-management-merge-prod-images-r1" x="317.2" y="434.8" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-17)">multi-platform one.                    [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="459.2" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-18)">│</text><text class="breeze-release-management-merge-prod-images-r7" x="317.2" y="459.2" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-18)">(DIRECTORY)                      [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-19)">│</text><text class="breeze-release-management-merge-prod-images-r8" x="317.2" y="483.6" textLength="1122.4" clip-path="url(#breeze-release-management-merge-prod-images-line-19)">[required]                       [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="508" textLength="1464" clip-path="url(#breeze-release-management-merge-prod-images-line-20)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-merge-prod-images-r1" x="1464" y="508" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-20)">
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="532.4" textLength="24.4" clip-path="url(#breeze-release-management-merge-prod-images-line-21)">╭─</text><text class="breeze-release-management-merge-prod-images-r5" x="24.4" y="532.4" textLength="195.2" clip-path="url(#breeze-release-management-merge-prod-images-line-21)"> Common options </text><text class="breeze-release-management-merge-prod-images-r5" x="219.6" y="532.4" textLength="1220" clip- [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="556.8" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-22)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="24.4" y="556.8" textLength="109.8" clip-path="url(#breeze-release-management-merge-prod-images-line-22)">--dry-run</text><text class="breeze-release-management-merge-prod-images-r9" x="158.6" y="556.8" textLength="24.4" clip-path="url(#breeze-relea [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-23)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="24.4" y="581.2" textLength="109.8" clip-path="url(#breeze-release-management-merge-prod-images-line-23)">--verbose</text><text class="breeze-release-management-merge-prod-images-r9" x="158.6" y="581.2" textLength="24.4" clip-path="url(#breeze-relea [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="605.6" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-24)">│</text><text class="breeze-release-management-merge-prod-images-r4" x="24.4" y="605.6" textLength="73.2" clip-path="url(#breeze-release-management-merge-prod-images-line-24)">--help</text><text class="breeze-release-management-merge-prod-images-r9" x="158.6" y="605.6" textLength="24.4" clip-path="url(#breeze-release-m [...]
+</text><text class="breeze-release-management-merge-prod-images-r5" x="0" y="630" textLength="1464" clip-path="url(#breeze-release-management-merge-prod-images-line-25)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-merge-prod-images-r1" x="1464" y="630" textLength="12.2" clip-path="url(#breeze-release-management-merge-prod-images-line-25)">
+</text>
+ </g>
+ </g>
+</svg>
diff --git a/dev/breeze/doc/images/output_release-management_merge-prod-images.txt b/dev/breeze/doc/images/output_release-management_merge-prod-images.txt
new file mode 100644
index 00000000000..efcdfdc9fc5
--- /dev/null
+++ b/dev/breeze/doc/images/output_release-management_merge-prod-images.txt
@@ -0,0 +1 @@
+f071fe40fe6d1cab05a6e31d44e47094
diff --git a/dev/breeze/doc/images/output_release-management_release-prod-images.svg b/dev/breeze/doc/images/output_release-management_release-prod-images.svg
index 962469926bb..439b7c7db7c 100644
--- a/dev/breeze/doc/images/output_release-management_release-prod-images.svg
+++ b/dev/breeze/doc/images/output_release-management_release-prod-images.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 733.1999999999999" xmlns="http://www.w3.org/2000/svg">
+<svg class="rich-terminal" viewBox="0 0 1482 830.8" xmlns="http://www.w3.org/2000/svg">
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -45,7 +45,7 @@
<defs>
<clipPath id="breeze-release-management-release-prod-images-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="682.1999999999999" />
+ <rect x="0" y="0" width="1463.0" height="779.8" />
</clipPath>
<clipPath id="breeze-release-management-release-prod-images-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -128,9 +128,21 @@
<clipPath id="breeze-release-management-release-prod-images-line-26">
<rect x="0" y="635.9" width="1464" height="24.65"/>
</clipPath>
+<clipPath id="breeze-release-management-release-prod-images-line-27">
+ <rect x="0" y="660.3" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-release-prod-images-line-28">
+ <rect x="0" y="684.7" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-release-prod-images-line-29">
+ <rect x="0" y="709.1" width="1464" height="24.65"/>
+ </clipPath>
+<clipPath id="breeze-release-management-release-prod-images-line-30">
+ <rect x="0" y="733.5" width="1464" height="24.65"/>
+ </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="731.2" rx="8"/><text class="breeze-release-management-release-prod-images-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management release-prod-images</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="828.8" rx="8"/><text class="breeze-release-management-release-prod-images-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management release-prod-images</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -162,12 +174,16 @@
</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-19)">│</text><text class="breeze-release-management-release-prod-images-r4" x="61" y="483.6" textLength="256.2" clip-path="url(#breeze-release-management-release-prod-images-line-19)">--include-pre-release</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="483.6" textLength="1073.6" clip-pat [...]
</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="508" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-20)">│</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="508" textLength="1073.6" clip-path="url(#breeze-release-management-release-prod-images-line-20)">we want to build an RC image with RC provider versions.      [...]
</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="532.4" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-21)">│</text><text class="breeze-release-management-release-prod-images-r4" x="61" y="532.4" textLength="158.6" clip-path="url(#breeze-release-management-release-prod-images-line-21)">--slim-images</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="532.4" textLength="719.8" clip-path="url(#b [...]
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="556.8" textLength="1464" clip-path="url(#breeze-release-management-release-prod-images-line-22)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-release-prod-images-r1" x="1464" y="556.8" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-22)">
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="581.2" textLength="24.4" clip-path="url(#breeze-release-management-release-prod-images-line-23)">╭─</text><text class="breeze-release-management-release-prod-images-r5" x="24.4" y="581.2" textLength="195.2" clip-path="url(#breeze-release-management-release-prod-images-line-23)"> Common options </text><text class="breeze-release-management-release-prod-images-r5" x="219.6" y="581.2" textLength="1 [...]
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="605.6" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-24)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="605.6" textLength="109.8" clip-path="url(#breeze-release-management-release-prod-images-line-24)">--dry-run</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="605.6" textLength="24.4" clip-path="url(#br [...]
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="630" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-25)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="630" textLength="109.8" clip-path="url(#breeze-release-management-release-prod-images-line-25)">--verbose</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="630" textLength="24.4" clip-path="url(#breeze-r [...]
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-26)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="654.4" textLength="73.2" clip-path="url(#breeze-release-management-release-prod-images-line-26)">--help</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="654.4" textLength="24.4" clip-path="url(#breeze [...]
-</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="678.8" textLength="1464" clip-path="url(#breeze-release-management-release-prod-images-line-27)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-release-prod-images-r1" x="1464" y="678.8" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-27)">
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="556.8" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-22)">│</text><text class="breeze-release-management-release-prod-images-r4" x="61" y="556.8" textLength="207.4" clip-path="url(#breeze-release-management-release-prod-images-line-22)">--metadata-folder</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="556.8" textLength="1073.6" clip-path="u [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-23)">│</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="581.2" textLength="1073.6" clip-path="url(#breeze-release-management-release-prod-images-line-23)">to registryonly by digests not by the tag because we are going to merg [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="605.6" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-24)">│</text><text class="breeze-release-management-release-prod-images-r1" x="366" y="605.6" textLength="1073.6" clip-path="url(#breeze-release-management-release-prod-images-line-24)">in multi-platform one.                 & [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="630" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-25)">│</text><text class="breeze-release-management-release-prod-images-r7" x="366" y="630" textLength="1073.6" clip-path="url(#breeze-release-management-release-prod-images-line-25)">(DIRECTORY)                     &# [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="654.4" textLength="1464" clip-path="url(#breeze-release-management-release-prod-images-line-26)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-release-prod-images-r1" x="1464" y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-26)">
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="678.8" textLength="24.4" clip-path="url(#breeze-release-management-release-prod-images-line-27)">╭─</text><text class="breeze-release-management-release-prod-images-r5" x="24.4" y="678.8" textLength="195.2" clip-path="url(#breeze-release-management-release-prod-images-line-27)"> Common options </text><text class="breeze-release-management-release-prod-images-r5" x="219.6" y="678.8" textLength="1 [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="703.2" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-28)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="703.2" textLength="109.8" clip-path="url(#breeze-release-management-release-prod-images-line-28)">--dry-run</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="703.2" textLength="24.4" clip-path="url(#br [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="727.6" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-29)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="727.6" textLength="109.8" clip-path="url(#breeze-release-management-release-prod-images-line-29)">--verbose</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="727.6" textLength="24.4" clip-path="url(#br [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="752" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-30)">│</text><text class="breeze-release-management-release-prod-images-r4" x="24.4" y="752" textLength="73.2" clip-path="url(#breeze-release-management-release-prod-images-line-30)">--help</text><text class="breeze-release-management-release-prod-images-r9" x="158.6" y="752" textLength="24.4" clip-path="url(#breeze-relea [...]
+</text><text class="breeze-release-management-release-prod-images-r5" x="0" y="776.4" textLength="1464" clip-path="url(#breeze-release-management-release-prod-images-line-31)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-release-prod-images-r1" x="1464" y="776.4" textLength="12.2" clip-path="url(#breeze-release-management-release-prod-images-line-31)">
</text>
</g>
</g>
diff --git a/dev/breeze/doc/images/output_release-management_release-prod-images.txt b/dev/breeze/doc/images/output_release-management_release-prod-images.txt
index 5ac2588c6af..0a597511ff8 100644
--- a/dev/breeze/doc/images/output_release-management_release-prod-images.txt
+++ b/dev/breeze/doc/images/output_release-management_release-prod-images.txt
@@ -1 +1 @@
-72d4ba0df1e411d9b834bd12a28c5b9d
+618872339adc234a9b32391870cac002
diff --git a/dev/breeze/doc/images/output_setup_check-all-params-in-groups.svg b/dev/breeze/doc/images/output_setup_check-all-params-in-groups.svg
index cbe7ad00356..4189d0302ae 100644
--- a/dev/breeze/doc/images/output_setup_check-all-params-in-groups.svg
+++ b/dev/breeze/doc/images/output_setup_check-all-params-in-groups.svg
@@ -199,7 +199,7 @@
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-16)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="410.4" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-16)">release-management:create-minor-branch | release-management:generate-constraints |          &# [...]
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="434.8" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-17)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="434.8" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-17)">release-management:generate-issue-content-core | release-management:generate-issue-content-helm-chart |</text><text class="breeze-setup-check-al [...]
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="459.2" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-18)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="459.2" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-18)">release-management:generate-issue-content-providers | release-management:generate-providers-metadata | </text><text class="breeze-setup-che [...]
-</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-19)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="483.6" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-19)">release-management:install-provider-distributions |                 & [...]
+</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-19)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="483.6" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-19)">release-management:install-provider-distributions | release-management:merge-prod-images |          [...]
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="508" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-20)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="508" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-20)">release-management:prepare-airflow-ctl-distributions | release-management:prepare-airflow-distributions</text><text class="breeze-setup-check-all-params- [...]
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="532.4" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-21)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="532.4" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-21)">| release-management:prepare-airflow-tarball | release-management:prepare-helm-chart-package |        [...]
</text><text class="breeze-setup-check-all-params-in-groups-r5" x="0" y="556.8" textLength="12.2" clip-path="url(#breeze-setup-check-all-params-in-groups-line-22)">│</text><text class="breeze-setup-check-all-params-in-groups-r6" x="183" y="556.8" textLength="1256.6" clip-path="url(#breeze-setup-check-all-params-in-groups-line-22)">release-management:prepare-helm-chart-tarball | release-management:prepare-provider-distributions |    </text><text class="b [...]
diff --git a/dev/breeze/doc/images/output_setup_check-all-params-in-groups.txt b/dev/breeze/doc/images/output_setup_check-all-params-in-groups.txt
index e0894a25bf0..71aebe7d03a 100644
--- a/dev/breeze/doc/images/output_setup_check-all-params-in-groups.txt
+++ b/dev/breeze/doc/images/output_setup_check-all-params-in-groups.txt
@@ -1 +1 @@
-0383b44d2790735f0cd5d0e9f64a55d9
+82638dbc2154f0543f9fff2bc3be875b
diff --git a/dev/breeze/doc/images/output_setup_regenerate-command-images.svg b/dev/breeze/doc/images/output_setup_regenerate-command-images.svg
index eef3815b573..4126ce9c5af 100644
--- a/dev/breeze/doc/images/output_setup_regenerate-command-images.svg
+++ b/dev/breeze/doc/images/output_setup_regenerate-command-images.svg
@@ -216,7 +216,7 @@
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="459.2" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-18)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="459.2" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-18)">release-management:generate-issue-content-core |                  &# [...]
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-19)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="483.6" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-19)">release-management:generate-issue-content-helm-chart |                 &# [...]
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="508" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-20)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="508" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-20)">release-management:generate-issue-content-providers | release-management:generate-providers-metadata</text><text class="breeze-setup-regenerate-command-image [...]
-</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="532.4" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-21)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="532.4" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-21)">| release-management:install-provider-distributions |                 [...]
+</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="532.4" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-21)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="532.4" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-21)">| release-management:install-provider-distributions | release-management:merge-prod-images |        </t [...]
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="556.8" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-22)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="556.8" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-22)">release-management:prepare-airflow-ctl-distributions |                 &# [...]
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="581.2" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-23)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="581.2" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-23)">release-management:prepare-airflow-distributions | release-management:prepare-airflow-tarball |     </text><text class="bre [...]
</text><text class="breeze-setup-regenerate-command-images-r5" x="0" y="605.6" textLength="12.2" clip-path="url(#breeze-setup-regenerate-command-images-line-24)">│</text><text class="breeze-setup-regenerate-command-images-r6" x="219.6" y="605.6" textLength="1220" clip-path="url(#breeze-setup-regenerate-command-images-line-24)">release-management:prepare-helm-chart-package | release-management:prepare-helm-chart-tarball |     </text><text class="bre [...]
diff --git a/dev/breeze/doc/images/output_setup_regenerate-command-images.txt b/dev/breeze/doc/images/output_setup_regenerate-command-images.txt
index ee7aa48f162..7a06cdbe2c0 100644
--- a/dev/breeze/doc/images/output_setup_regenerate-command-images.txt
+++ b/dev/breeze/doc/images/output_setup_regenerate-command-images.txt
@@ -1 +1 @@
-372f99a5ba9a16b23a502fc28afe696a
+efc183801305553e0a21da91c5cff91c
diff --git a/dev/breeze/doc/images/output_shell.svg b/dev/breeze/doc/images/output_shell.svg
index c33e37ad242..fcc898ac3af 100644
--- a/dev/breeze/doc/images/output_shell.svg
+++ b/dev/breeze/doc/images/output_shell.svg
@@ -547,9 +547,9 @@
</text><text class="breeze-shell-r5" x="0" y="264" textLength="12.2" clip-path="url(#breeze-shell-line-10)">│</text><text class="breeze-shell-r4" x="24.4" y="264" textLength="158.6" clip-path="url(#breeze-shell-line-10)">--integration</text><text class="breeze-shell-r1" x="768.6" y="264" textLength="671" clip-path="url(#breeze-shell-line-10)">Core Integrations to enable when running (can be more  </text><text class="breeze-shell-r5" x="14 [...]
</text><text class="breeze-shell-r5" x="0" y="288.4" textLength="12.2" clip-path="url(#breeze-shell-line-11)">│</text><text class="breeze-shell-r1" x="768.6" y="288.4" textLength="671" clip-path="url(#breeze-shell-line-11)">than one).                                          & [...]
</text><text class="breeze-shell-r5" x="0" y="312.8" textLength="12.2" clip-path="url(#breeze-shell-line-12)">│</text><text class="breeze-shell-r7" x="768.6" y="312.8" textLength="671" clip-path="url(#breeze-shell-line-12)">(all | all-testable | cassandra | celery | drill |     </text><text class="breeze-shell-r5" x="1451.8" y="312.8" textLength="12.2" clip-path="url(#breeze-shell-line-12)">│</text><text class="breeze- [...]
-</text><text class="breeze-shell-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-shell-line-13)">│</text><text class="breeze-shell-r7" x="768.6" y="337.2" textLength="671" clip-path="url(#breeze-shell-line-13)">gremlin | kafka | kerberos | keycloak | mongo | mssql |</text><text class="breeze-shell-r5" x="1451.8" y="337.2" textLength="12.2" clip-path="url(#breeze-shell-line-13)">│</text><text class="breeze-shell-r1" x="14 [...]
-</text><text class="breeze-shell-r5" x="0" y="361.6" textLength="12.2" clip-path="url(#breeze-shell-line-14)">│</text><text class="breeze-shell-r7" x="768.6" y="361.6" textLength="671" clip-path="url(#breeze-shell-line-14)">openlineage | otel | pinot | qdrant | redis | statsd | </text><text class="breeze-shell-r5" x="1451.8" y="361.6" textLength="12.2" clip-path="url(#breeze-shell-line-14)">│</text><text class="breeze-shell-r1" [...]
-</text><text class="breeze-shell-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-shell-line-15)">│</text><text class="breeze-shell-r7" x="768.6" y="386" textLength="671" clip-path="url(#breeze-shell-line-15)">trino | ydb)                                          [...]
+</text><text class="breeze-shell-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-shell-line-13)">│</text><text class="breeze-shell-r7" x="768.6" y="337.2" textLength="671" clip-path="url(#breeze-shell-line-13)">gremlin | kafka | kafka | kerberos | keycloak | mongo |</text><text class="breeze-shell-r5" x="1451.8" y="337.2" textLength="12.2" clip-path="url(#breeze-shell-line-13)">│</text><text class="breeze-shell-r1" x="14 [...]
+</text><text class="breeze-shell-r5" x="0" y="361.6" textLength="12.2" clip-path="url(#breeze-shell-line-14)">│</text><text class="breeze-shell-r7" x="768.6" y="361.6" textLength="671" clip-path="url(#breeze-shell-line-14)">mssql | openlineage | otel | pinot | qdrant | redis |  </text><text class="breeze-shell-r5" x="1451.8" y="361.6" textLength="12.2" clip-path="url(#breeze-shell-line-14)">│</text><text class="breeze-shell [...]
+</text><text class="breeze-shell-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-shell-line-15)">│</text><text class="breeze-shell-r7" x="768.6" y="386" textLength="671" clip-path="url(#breeze-shell-line-15)">statsd | trino | ydb)                                  </text><text class="breeze- [...]
</text><text class="breeze-shell-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-shell-line-16)">│</text><text class="breeze-shell-r4" x="24.4" y="410.4" textLength="231.8" clip-path="url(#breeze-shell-line-16)">--load-example-dags</text><text class="breeze-shell-r6" x="719.8" y="410.4" textLength="24.4" clip-path="url(#breeze-shell-line-16)">-e</text><text class="breeze-shell-r1" x="768.6" y="410.4" textLength="671" clip-path="url(#breeze-shell-line-16)">Enable configu [...]
</text><text class="breeze-shell-r5" x="0" y="434.8" textLength="12.2" clip-path="url(#breeze-shell-line-17)">│</text><text class="breeze-shell-r1" x="768.6" y="434.8" textLength="671" clip-path="url(#breeze-shell-line-17)">Airflow.                                           &# [...]
</text><text class="breeze-shell-r5" x="0" y="459.2" textLength="12.2" clip-path="url(#breeze-shell-line-18)">│</text><text class="breeze-shell-r4" x="24.4" y="459.2" textLength="317.2" clip-path="url(#breeze-shell-line-18)">--load-default-connections</text><text class="breeze-shell-r6" x="719.8" y="459.2" textLength="24.4" clip-path="url(#breeze-shell-line-18)">-c</text><text class="breeze-shell-r1" x="768.6" y="459.2" textLength="671" clip-path="url(#breeze-shell-line-18)">Enable  [...]
diff --git a/dev/breeze/doc/images/output_shell.txt b/dev/breeze/doc/images/output_shell.txt
index 4bc4dbed2a1..9e7bc4f0802 100644
--- a/dev/breeze/doc/images/output_shell.txt
+++ b/dev/breeze/doc/images/output_shell.txt
@@ -1 +1 @@
-664f71ecf40d322c88efa458a7e1a564
+bc65c025cfd332a6ea907ea7e97e5007
diff --git a/dev/breeze/doc/images/output_start-airflow.svg b/dev/breeze/doc/images/output_start-airflow.svg
index 68db4109240..a42c365fbd6 100644
--- a/dev/breeze/doc/images/output_start-airflow.svg
+++ b/dev/breeze/doc/images/output_start-airflow.svg
@@ -441,9 +441,9 @@
</text><text class="breeze-start-airflow-r5" x="0" y="312.8" textLength="12.2" clip-path="url(#breeze-start-airflow-line-12)">│</text><text class="breeze-start-airflow-r4" x="24.4" y="312.8" textLength="158.6" clip-path="url(#breeze-start-airflow-line-12)">--integration</text><text class="breeze-start-airflow-r1" x="768.6" y="312.8" textLength="671" clip-path="url(#breeze-start-airflow-line-12)">Core Integrations to enable when running (can be more [...]
</text><text class="breeze-start-airflow-r5" x="0" y="337.2" textLength="12.2" clip-path="url(#breeze-start-airflow-line-13)">│</text><text class="breeze-start-airflow-r1" x="768.6" y="337.2" textLength="671" clip-path="url(#breeze-start-airflow-line-13)">than one).                                      [...]
</text><text class="breeze-start-airflow-r5" x="0" y="361.6" textLength="12.2" clip-path="url(#breeze-start-airflow-line-14)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="361.6" textLength="671" clip-path="url(#breeze-start-airflow-line-14)">(all | all-testable | cassandra | celery | drill |     </text><text class="breeze-start-airflow-r5" x="1451.8" y="361.6" textLength="12.2" clip-path="url(#breeze-star [...]
-</text><text class="breeze-start-airflow-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-start-airflow-line-15)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="386" textLength="671" clip-path="url(#breeze-start-airflow-line-15)">gremlin | kafka | kerberos | keycloak | mongo | mssql |</text><text class="breeze-start-airflow-r5" x="1451.8" y="386" textLength="12.2" clip-path="url(#breeze-start-airflow-line-15)">│ [...]
-</text><text class="breeze-start-airflow-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-start-airflow-line-16)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="410.4" textLength="671" clip-path="url(#breeze-start-airflow-line-16)">openlineage | otel | pinot | qdrant | redis | statsd | </text><text class="breeze-start-airflow-r5" x="1451.8" y="410.4" textLength="12.2" clip-path="url(#breeze-start-airflow- [...]
-</text><text class="breeze-start-airflow-r5" x="0" y="434.8" textLength="12.2" clip-path="url(#breeze-start-airflow-line-17)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="434.8" textLength="671" clip-path="url(#breeze-start-airflow-line-17)">trino | ydb)                                    [...]
+</text><text class="breeze-start-airflow-r5" x="0" y="386" textLength="12.2" clip-path="url(#breeze-start-airflow-line-15)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="386" textLength="671" clip-path="url(#breeze-start-airflow-line-15)">gremlin | kafka | kafka | kerberos | keycloak | mongo |</text><text class="breeze-start-airflow-r5" x="1451.8" y="386" textLength="12.2" clip-path="url(#breeze-start-airflow-line-15)">│ [...]
+</text><text class="breeze-start-airflow-r5" x="0" y="410.4" textLength="12.2" clip-path="url(#breeze-start-airflow-line-16)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="410.4" textLength="671" clip-path="url(#breeze-start-airflow-line-16)">mssql | openlineage | otel | pinot | qdrant | redis |  </text><text class="breeze-start-airflow-r5" x="1451.8" y="410.4" textLength="12.2" clip-path="url(#breeze-start-air [...]
+</text><text class="breeze-start-airflow-r5" x="0" y="434.8" textLength="12.2" clip-path="url(#breeze-start-airflow-line-17)">│</text><text class="breeze-start-airflow-r7" x="768.6" y="434.8" textLength="671" clip-path="url(#breeze-start-airflow-line-17)">statsd | trino | ydb)                                 [...]
</text><text class="breeze-start-airflow-r5" x="0" y="459.2" textLength="12.2" clip-path="url(#breeze-start-airflow-line-18)">│</text><text class="breeze-start-airflow-r4" x="24.4" y="459.2" textLength="317.2" clip-path="url(#breeze-start-airflow-line-18)">--standalone-dag-processor</text><text class="breeze-start-airflow-r1" x="341.6" y="459.2" textLength="12.2" clip-path="url(#breeze-start-airflow-line-18)">/</text><text class="breeze-start-airflow-r4" x="353.8" y="459.2" textLength="3 [...]
</text><text class="breeze-start-airflow-r5" x="0" y="483.6" textLength="12.2" clip-path="url(#breeze-start-airflow-line-19)">│</text><text class="breeze-start-airflow-r4" x="24.4" y="483.6" textLength="12.2" clip-path="url(#breeze-start-airflow-line-19)">r</text><text class="breeze-start-airflow-r1" x="768.6" y="483.6" textLength="671" clip-path="url(#breeze-start-airflow-line-19)">(required for Airflow 3).            [...]
</text><text class="breeze-start-airflow-r5" x="0" y="508" textLength="12.2" clip-path="url(#breeze-start-airflow-line-20)">│</text><text class="breeze-start-airflow-r5" x="768.6" y="508" textLength="671" clip-path="url(#breeze-start-airflow-line-20)">[default: standalone-dag-processor]                    </text><text class="breeze-start-airflow-r5" x="1451.8" y="508" textLength="12.2 [...]
diff --git a/dev/breeze/doc/images/output_start-airflow.txt b/dev/breeze/doc/images/output_start-airflow.txt
index 12d99f7aa94..f43effbdbba 100644
--- a/dev/breeze/doc/images/output_start-airflow.txt
+++ b/dev/breeze/doc/images/output_start-airflow.txt
@@ -1 +1 @@
-bae200412140579f690c72361f82f838
+6d7d96665eecb42f4e73ae4b20d2c444
diff --git a/dev/breeze/doc/images/output_testing_providers-integration-tests.svg b/dev/breeze/doc/images/output_testing_providers-integration-tests.svg
index 71bce41db52..36e83ecb36e 100644
--- a/dev/breeze/doc/images/output_testing_providers-integration-tests.svg
+++ b/dev/breeze/doc/images/output_testing_providers-integration-tests.svg
@@ -237,8 +237,8 @@
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="752" textLength="1464" clip-path="url(#breeze-testing-providers-integration-tests-line-30)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-testing-providers-integration-tests-r1" x="1464" y="752" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-30)">
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="776.4" textLength="24.4" clip-path="url(#breeze-testing-providers-integration-tests-line-31)">╭─</text><text class="breeze-testing-providers-integration-tests-r5" x="24.4" y="776.4" textLength="231.8" clip-path="url(#breeze-testing-providers-integration-tests-line-31)"> Integration tests </text><text class="breeze-testing-providers-integration-tests-r5" x="256.2" y="776.4" textLength="1183.4" clip- [...]
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="800.8" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-32)">│</text><text class="breeze-testing-providers-integration-tests-r4" x="24.4" y="800.8" textLength="158.6" clip-path="url(#breeze-testing-providers-integration-tests-line-32)">--integration</text><text class="breeze-testing-providers-integration-tests-r1" x="231.8" y="800.8" textLength="1207.8" clip-path="url(#breeze-test [...]
-</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-33)">│</text><text class="breeze-testing-providers-integration-tests-r6" x="231.8" y="825.2" textLength="1207.8" clip-path="url(#breeze-testing-providers-integration-tests-line-33)">(all | all-testable | cassandra | celery | drill | gremlin | kafka | mongo& [...]
-</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="849.6" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-34)">│</text><text class="breeze-testing-providers-integration-tests-r6" x="231.8" y="849.6" textLength="1207.8" clip-path="url(#breeze-testing-providers-integration-tests-line-34)">pinot | qdrant | redis | trino | ydb)           &# [...]
+</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-33)">│</text><text class="breeze-testing-providers-integration-tests-r6" x="231.8" y="825.2" textLength="1207.8" clip-path="url(#breeze-testing-providers-integration-tests-line-33)">(all | all-testable | cassandra | celery | drill | gremlin | kafka | kafka& [...]
+</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="849.6" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-34)">│</text><text class="breeze-testing-providers-integration-tests-r6" x="231.8" y="849.6" textLength="1207.8" clip-path="url(#breeze-testing-providers-integration-tests-line-34)">openlineage | pinot | qdrant | redis | trino | ydb)       &# [...]
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="874" textLength="1464" clip-path="url(#breeze-testing-providers-integration-tests-line-35)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-testing-providers-integration-tests-r1" x="1464" y="874" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-35)">
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="898.4" textLength="24.4" clip-path="url(#breeze-testing-providers-integration-tests-line-36)">╭─</text><text class="breeze-testing-providers-integration-tests-r5" x="24.4" y="898.4" textLength="402.6" clip-path="url(#breeze-testing-providers-integration-tests-line-36)"> Advanced flag for tests command </text><text class="breeze-testing-providers-integration-tests-r5" x="427" y="898.4 [...]
</text><text class="breeze-testing-providers-integration-tests-r5" x="0" y="922.8" textLength="12.2" clip-path="url(#breeze-testing-providers-integration-tests-line-37)">│</text><text class="breeze-testing-providers-integration-tests-r4" x="24.4" y="922.8" textLength="231.8" clip-path="url(#breeze-testing-providers-integration-tests-line-37)">--github-repository</text><text class="breeze-testing-providers-integration-tests-r7" x="366" y="922.8" textLength="24.4" clip-path="url(#breeze-te [...]
diff --git a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt
index 1b6995e16c3..c098846ffa3 100644
--- a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt
+++ b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt
@@ -1 +1 @@
-48aafaf7c9282ae70307e168ffefb0ab
+945ff1e5a8c4a746f61138ba9e94c9ba
diff --git a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
index 40863c1f3e0..da0d2bc3ada 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
@@ -116,6 +116,9 @@ from airflow_breeze.utils.console import MessageType, Output, get_console
from airflow_breeze.utils.custom_param_types import BetterChoice, NotVerifiedBetterChoice
from airflow_breeze.utils.debug_pyproject_toml import debug_pyproject_tomls
from airflow_breeze.utils.docker_command_utils import (
+ check_airflow_cache_builder_configured,
+ check_docker_buildx_plugin,
+ check_regctl_installed,
check_remote_ghcr_io_commands,
execute_command_in_shell,
fix_ownership_using_docker,
@@ -1948,6 +1951,66 @@ def clean_old_provider_artifacts(
run_command(command, check=False)
+def alias_images(
+ airflow_version: str,
+ dockerhub_repo: str,
+ python_versions: list[str],
+ image_prefix: str,
+ skip_latest: bool = False,
+):
+ get_console().print("[info]Waiting for a few seconds for the new images to refresh.[/]")
+ time.sleep(10)
+ get_console().print("[info]Aliasing images with links to the newly created images.[/]")
+ for python in python_versions:
+ # Always alias the last python version to point to the non-python version
+ if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
+ get_console().print(
+ f"[info]Aliasing the {image_prefix}{airflow_version}-python{python} "
+ f"version with {image_prefix}{airflow_version}[/]"
+ )
+ alias_image(
+ f"{dockerhub_repo}:{image_prefix}{airflow_version}-python{python}",
+ f"{dockerhub_repo}:{image_prefix}{airflow_version}",
+ )
+ if not skip_latest:
+ # if we are taging latest images, we also need to alias the non-version images
+ get_console().print(
+ f"[info]Aliasing {image_prefix}{airflow_version}-python{python} "
+ f"version with {image_prefix}latest-python{python}[/]"
+ )
+ alias_image(
+ f"{dockerhub_repo}:{image_prefix}{airflow_version}-python{python}",
+ f"{dockerhub_repo}:{image_prefix}latest-python{python}",
+ )
+ if image_prefix == "":
+ alias_image(
+ f"{dockerhub_repo}:{airflow_version}-python{python}",
+ f"{dockerhub_repo}:latest-python{python}",
+ )
+ if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
+ alias_image(
+ f"{dockerhub_repo}:{image_prefix}{airflow_version}",
+ f"{dockerhub_repo}:{image_prefix}latest",
+ )
+
+
+def check_skip_latest(airflow_version, skip_latest):
+ if is_pre_release(airflow_version):
+ get_console().print(
+ f"[warning]Skipping latest image tagging as this is a pre-release version: {airflow_version}"
+ )
+ skip_latest = True
+ else:
+ if skip_latest:
+ get_console().print("[info]Skipping latest image tagging as user requested it.[/]")
+ else:
+ get_console().print(
+ "[info]Also tagging the images with latest tags as this is release version.[/]"
+ )
+ check_docker_buildx_plugin()
+ return skip_latest
+
+
@release_management.command(
name="release-prod-images", help="Release production images to DockerHub (needs DockerHub permissions)."
)
@@ -1991,6 +2054,12 @@ def clean_old_provider_artifacts(
is_flag=True,
help="Whether to prepare slim images instead of the regular ones.",
)
+@click.option(
+ "--metadata-folder",
+ type=click.Path(dir_okay=True, file_okay=False, writable=True, path_type=Path),
+ help="Folder to write the build metadata to. When this option is specified the image is pushed to registry"
+ "only by digests not by the tag because we are going to merge several images in multi-platform one.",
+)
@option_verbose
def release_prod_images(
airflow_version: str,
@@ -2001,144 +2070,188 @@ def release_prod_images(
commit_sha: str | None,
skip_latest: bool,
include_pre_release: bool,
+ metadata_folder: Path | None,
):
perform_environment_checks()
check_remote_ghcr_io_commands()
- if is_pre_release(airflow_version):
- get_console().print(
- f"[warning]Skipping latest image tagging as this is a pre-release version: {airflow_version}"
- )
- skip_latest = True
- else:
- if skip_latest:
- get_console().print("[info]Skipping latest image tagging as user requested it.[/]")
- else:
- get_console().print(
- "[info]Also tagging the images with latest tags as this is release version.[/]"
- )
- result_docker_buildx = run_command(
- ["docker", "buildx", "version"],
- check=False,
- )
- if result_docker_buildx.returncode != 0:
- get_console().print("[error]Docker buildx plugin must be installed to release the images[/]")
- get_console().print()
- get_console().print("See https://docs.docker.com/buildx/working-with-buildx/ for installation info.")
- sys.exit(1)
- result_inspect_builder = run_command(["docker", "buildx", "inspect", "airflow_cache"], check=False)
- if result_inspect_builder.returncode != 0:
- get_console().print("[error]Airflow Cache builder must be configured to release the images[/]")
- get_console().print()
- get_console().print(
- "See https://github.com/apache/airflow/blob/main/dev/MANUALLY_BUILDING_IMAGES.md"
- " for instructions on setting it up."
- )
- sys.exit(1)
- result_regctl = run_command(["regctl", "version"], check=False)
- if result_regctl.returncode != 0:
- get_console().print("[error]Regctl must be installed and on PATH to release the images[/]")
- get_console().print()
- get_console().print(
- "See https://github.com/regclient/regclient/blob/main/docs/regctl.md for installation info."
- )
- sys.exit(1)
+ skip_latest = check_skip_latest(airflow_version, skip_latest)
+ check_airflow_cache_builder_configured()
from packaging.version import Version
include_pre_release = include_pre_release or Version(airflow_version).is_prerelease
if include_pre_release:
get_console().print("[warning]Including pre-releases when considering dependencies.[/]")
+
+ if metadata_folder:
+ if limit_platform == MULTI_PLATFORM:
+ get_console().print(
+ "[error]Cannot use metadata folder with multi-platform image. Use "
+ "--limit-platform to limit it to a single platform only[/]"
+ )
+ sys.exit(1)
+ get_console().print(
+ f"[info]Will push images to registry only by digests and store metadata files in "
+ f"the {metadata_folder}[/]"
+ )
+ if not metadata_folder.exists():
+ metadata_folder.mkdir(parents=True)
+ else:
+ check_regctl_installed()
+
python_versions = CURRENT_PYTHON_MAJOR_MINOR_VERSIONS if limit_python is None else [limit_python]
+ if slim_images:
+ image_prefix = "slim-"
+ image_type = "slim"
+ else:
+ image_prefix = ""
+ image_type = "regular"
for python in python_versions:
+ build_args = {
+ "AIRFLOW_CONSTRAINTS": "constraints-no-providers",
+ "PYTHON_BASE_IMAGE": f"python:{python}-slim-bookworm",
+ "AIRFLOW_VERSION": airflow_version,
+ "INCLUDE_PRE_RELEASE": "true" if include_pre_release else "false",
+ "INSTALL_DISTRIBUTIONS_FROM_CONTEXT": "false",
+ "DOCKER_CONTEXT_FILES": "./docker-context-files",
+ }
+ if commit_sha:
+ build_args["COMMIT_SHA"] = commit_sha
if slim_images:
- slim_build_args = {
- "AIRFLOW_EXTRAS": "",
- "AIRFLOW_CONSTRAINTS": "constraints-no-providers",
- "PYTHON_BASE_IMAGE": f"python:{python}-slim-bookworm",
- "AIRFLOW_VERSION": airflow_version,
- "INCLUDE_PRE_RELEASE": "true" if include_pre_release else "false",
- "INSTALL_DISTRIBUTIONS_FROM_CONTEXT": "false",
- "DOCKER_CONTEXT_FILES": "./docker-context-files",
- }
- if commit_sha:
- slim_build_args["COMMIT_SHA"] = commit_sha
- get_console().print(f"[info]Building slim {airflow_version} image for Python {python}[/]")
- python_build_args = deepcopy(slim_build_args)
- slim_image_name = f"{dockerhub_repo}:slim-{airflow_version}-python{python}"
- docker_buildx_command = [
- "docker",
- "buildx",
- "build",
- "--builder",
- "airflow_cache",
- *convert_build_args_dict_to_array_of_args(build_args=python_build_args),
- "--platform",
- limit_platform,
- ".",
+ build_args["AIRFLOW_EXTRAS"] = ""
+ get_console().print(f"[info]Building {image_type} {airflow_version} image for Python {python}[/]")
+ python_build_args = deepcopy(build_args)
+ image_name = f"{dockerhub_repo}:{image_prefix}{airflow_version}-python{python}"
+ docker_buildx_command = [
+ "docker",
+ "buildx",
+ "build",
+ "--builder",
+ "airflow_cache",
+ *convert_build_args_dict_to_array_of_args(build_args=python_build_args),
+ "--platform",
+ limit_platform,
+ "--push",
+ ]
+ metadata_file: Path | None = None
+ if metadata_folder:
+ metadata_file = (
+ metadata_folder
+ / f"metadata-{airflow_version}-{image_prefix}{limit_platform.replace('/', '_')}-{python}.json"
+ )
+ docker_buildx_command += [
"-t",
- slim_image_name,
- "--push",
+ dockerhub_repo,
+ "--metadata-file",
+ str(metadata_file),
+ "--output",
+ "type=image,push-by-digest=true,name-canonical=true",
]
- run_command(docker_buildx_command)
- if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
- alias_image(
- slim_image_name,
- f"{dockerhub_repo}:slim-{airflow_version}",
- )
else:
- get_console().print(f"[info]Building regular {airflow_version} image for Python {python}[/]")
- image_name = f"{dockerhub_repo}:{airflow_version}-python{python}"
- regular_build_args = {
- "PYTHON_BASE_IMAGE": f"python:{python}-slim-bookworm",
- "AIRFLOW_VERSION": airflow_version,
- "INCLUDE_PRE_RELEASE": "true" if include_pre_release else "false",
- "DOCKER_CONTEXT_FILES": "./docker-context-files",
- }
- if commit_sha:
- regular_build_args["COMMIT_SHA"] = commit_sha
- docker_buildx_command = [
- "docker",
- "buildx",
- "build",
- "--builder",
- "airflow_cache",
- *convert_build_args_dict_to_array_of_args(build_args=regular_build_args),
- "--platform",
- limit_platform,
- ".",
+ docker_buildx_command += [
"-t",
image_name,
- "--push",
]
- run_command(docker_buildx_command)
- if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
- alias_image(image_name, f"{dockerhub_repo}:{airflow_version}")
- # in case of re-tagging the images might need few seconds to refresh multi-platform images in DockerHub
- time.sleep(10)
- if not skip_latest:
- get_console().print("[info]Replacing latest images with links to the newly created images.[/]")
- for python in python_versions:
- if slim_images:
- alias_image(
- f"{dockerhub_repo}:slim-{airflow_version}-python{python}",
- f"{dockerhub_repo}:slim-latest-python{python}",
- )
- else:
- alias_image(
- f"{dockerhub_repo}:{airflow_version}-python{python}",
- f"{dockerhub_repo}:latest-python{python}",
- )
- if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
- # only tag latest "default" image when we build the latest allowed python version
- if slim_images:
- alias_image(
- f"{dockerhub_repo}:slim-{airflow_version}",
- f"{dockerhub_repo}:slim-latest",
- )
- else:
- alias_image(
- f"{dockerhub_repo}:{airflow_version}",
- f"{dockerhub_repo}:latest",
- )
+ docker_buildx_command += ["."]
+ run_command(docker_buildx_command)
+ if metadata_file:
+ get_console().print(f"[green]Metadata file stored in {metadata_file}")
+ if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1] and not metadata_file:
+ get_console().print(
+ f"[info]Aliasing the latest {python} version to {image_prefix}{airflow_version}[/]"
+ )
+ alias_image(
+ image_name,
+ f"{dockerhub_repo}:{image_prefix}{airflow_version}",
+ )
+ if not metadata_folder:
+ alias_images(airflow_version, dockerhub_repo, python_versions, image_prefix, skip_latest)
+
+
+@release_management.command(
+ name="merge-prod-images",
+ help="Merge production images in DockerHub based on digest files (needs DockerHub permissions).",
+)
+@click.option("--airflow-version", required=True, help="Airflow version to release (2.3.0, 2.3.0rc1 etc.)")
+@option_dry_run
+@click.option(
+ "--dockerhub-repo",
+ default=APACHE_AIRFLOW_GITHUB_REPOSITORY,
+ show_default=True,
+ help="DockerHub repository for the images",
+)
+@click.option(
+ "--limit-python",
+ type=BetterChoice(CURRENT_PYTHON_MAJOR_MINOR_VERSIONS),
+ help="Specific python to build images for (if not specified - the images are built for all"
+ " available python versions)",
+)
+@click.option(
+ "--skip-latest",
+ is_flag=True,
+ help="Whether to skip publishing the latest images (so that 'latest' images are not updated). "
+ "This should only be used if you release image for previous branches. Automatically set when "
+ "rc/alpha/beta images are built.",
+)
+@click.option(
+ "--slim-images",
+ is_flag=True,
+ help="Whether to prepare slim images instead of the regular ones.",
+)
+@click.option(
+ "--metadata-folder",
+ type=click.Path(exists=True, dir_okay=True, file_okay=False, writable=True, path_type=Path),
+ help="Folder to write the build metadata to. When this option is specified the image is pushed to registry"
+ "only by digests not by the tag because we are going to merge several images in multi-platform one.",
+ required=True,
+)
+@option_verbose
+def merge_prod_images(
+ airflow_version: str,
+ dockerhub_repo: str,
+ slim_images: bool,
+ limit_python: str | None,
+ skip_latest: bool,
+ metadata_folder: Path,
+):
+ perform_environment_checks()
+ check_remote_ghcr_io_commands()
+ check_docker_buildx_plugin()
+ check_regctl_installed()
+ skip_latest = check_skip_latest(airflow_version, skip_latest)
+ python_versions = CURRENT_PYTHON_MAJOR_MINOR_VERSIONS if limit_python is None else [limit_python]
+ if slim_images:
+ image_prefix = "slim-"
+ else:
+ image_prefix = ""
+ for python in python_versions:
+ metadata_files = metadata_folder.glob(f"metadata-{airflow_version}-{image_prefix}*-{python}.json")
+ image_name = f"{dockerhub_repo}:{image_prefix}{airflow_version}-python{python}"
+ import json
+
+ metadata_array = [json.loads(file.read_text()) for file in metadata_files]
+ digests_to_merge = [
+ dockerhub_repo + "@" + metadata_content["containerimage.digest"]
+ for metadata_content in metadata_array
+ ]
+ get_console().print(f"[info]Merging {image_name} file from digests found in {metadata_files}[/]")
+ get_console().print(f"[info]Digests to merge: {digests_to_merge}[/]")
+ imagetool_command = [
+ "docker",
+ "buildx",
+ "imagetools",
+ "create",
+ *digests_to_merge,
+ "-t",
+ image_name,
+ ]
+ run_command(imagetool_command)
+ alias_images(
+ airflow_version,
+ dockerhub_repo,
+ python_versions,
+ image_prefix=image_prefix,
+ skip_latest=skip_latest,
+ )
def is_package_in_dist(dist_files: list[str], package: str) -> bool:
diff --git a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
index 0e4c0219199..032edfd170b 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
@@ -308,6 +308,20 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str | list[str]]]] = {
"--skip-latest",
"--include-pre-release",
"--slim-images",
+ "--metadata-folder",
+ ],
+ }
+ ],
+ "breeze release-management merge-prod-images": [
+ {
+ "name": "Merge PROD IMAGE flags",
+ "options": [
+ "--airflow-version",
+ "--dockerhub-repo",
+ "--limit-python",
+ "--skip-latest",
+ "--slim-images",
+ "--metadata-folder",
],
}
],
diff --git a/dev/breeze/src/airflow_breeze/global_constants.py b/dev/breeze/src/airflow_breeze/global_constants.py
index d0c4e2e6fa6..ef8f2917e87 100644
--- a/dev/breeze/src/airflow_breeze/global_constants.py
+++ b/dev/breeze/src/airflow_breeze/global_constants.py
@@ -71,6 +71,7 @@ TESTABLE_PROVIDERS_INTEGRATIONS = [
"drill",
"gremlin",
"kafka",
+ "kafka",
"mongo",
"mssql",
"pinot",
diff --git a/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py b/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py
index 990c5c70f87..80772a6ec84 100644
--- a/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py
+++ b/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py
@@ -904,3 +904,56 @@ def is_docker_rootless() -> bool:
# we ignore if docker is missing
pass
return False
+
+
+def check_airflow_cache_builder_configured():
+ result_inspect_builder = run_command(["docker", "buildx", "inspect", "airflow_cache"], check=False)
+ if result_inspect_builder.returncode != 0:
+ get_console().print(
+ "[error]Airflow Cache builder must be configured to "
+ "build multi-platform images with multiple builders[/]"
+ )
+ get_console().print()
+ get_console().print(
+ "See https://github.com/apache/airflow/blob/main/dev/MANUALLY_BUILDING_IMAGES.md"
+ " for instructions on setting it up."
+ )
+ sys.exit(1)
+
+
+def check_regctl_installed():
+ result_regctl = run_command(["regctl", "version"], check=False)
+ if result_regctl.returncode != 0:
+ get_console().print("[error]Regctl must be installed and on PATH to release the images[/]")
+ get_console().print()
+ get_console().print(
+ "See https://github.com/regclient/regclient/blob/main/docs/regctl.md for installation info."
+ )
+ sys.exit(1)
+
+
+def check_docker_buildx_plugin():
+ result_docker_buildx = run_command(
+ ["docker", "buildx", "version"],
+ check=False,
+ text=True,
+ capture_output=True,
+ )
+ if result_docker_buildx.returncode != 0:
+ get_console().print("[error]Docker buildx plugin must be installed to release the images[/]")
+ get_console().print()
+ get_console().print("See https://docs.docker.com/buildx/working-with-buildx/ for installation info.")
+ sys.exit(1)
+ from packaging.version import Version
+
+ version = result_docker_buildx.stdout.splitlines()[0].split(" ")[1].lstrip("v")
+ packaging_version = Version(version)
+ if packaging_version < Version("0.13.0"):
+ get_console().print("[error]Docker buildx plugin must be at least 0.13.0 to release the images[/]")
+ get_console().print()
+ get_console().print(
+ "See https://github.com/docker/buildx?tab=readme-ov-file#installing for installation info."
+ )
+ sys.exit(1)
+ else:
+ get_console().print(f"[success]Docker buildx plugin is installed and in good version: {version}[/]")