You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2022/04/01 12:41:25 UTC
[airflow] branch fix-pushing-cache-for-prod-image created (now 23c4a23)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch fix-pushing-cache-for-prod-image
in repository https://gitbox.apache.org/repos/asf/airflow.git.
at 23c4a23 Cleanup of build-image scripts
This branch includes the following new commits:
new 23c4a23 Cleanup of build-image scripts
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[airflow] 01/01: Cleanup of build-image scripts
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch fix-pushing-cache-for-prod-image
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 23c4a23cb5dac318665b2c58e5e3eff880f60fc7
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri Apr 1 14:33:22 2022 +0200
Cleanup of build-image scripts
This PR re-arranges tags int GitHub actions to be more explicit in
the pull/build/push steps on which tag of the image is currently
pulled/pushed/built.
Previously the actual configuration of those steps was determined
by combining "top", "job" and "step" variables, but that gave
little insight on what combinations of evn variables was used in
the step eventually. After this change, in all the pull/push/build
steps you can see what environment is used for that build.
Also, since we often build and push and then pull the same image in
the same workflow, the PUSH tag was sometimes used in PULL
variable as input. We change it now by introducing single
`IMAGE_TAG_FOR_THE_BUILD` which will be used as PUSH and PULL
tag as needed.
---
.github/workflows/build-images.yml | 45 ++++---
.github/workflows/ci.yml | 156 +++++++++++++------------
scripts/ci/images/ci_build_prod_image_on_ci.sh | 1 -
scripts/ci/images/ci_pull_prod_image_on_ci.sh | 6 -
scripts/ci/tools/build_dockerhub.sh | 59 ----------
5 files changed, 112 insertions(+), 155 deletions(-)
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
index 1d41133..67ab36d 100644
--- a/.github/workflows/build-images.yml
+++ b/.github/workflows/build-images.yml
@@ -39,10 +39,8 @@ env:
secrets.CONSTRAINTS_GITHUB_REPOSITORY || 'apache/airflow' }}
# This token is WRITE one - pull_request_target type of events always have the WRITE token
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- INSTALL_PROVIDERS_FROM_SOURCES: "true"
AIRFLOW_LOGIN_TO_GITHUB_REGISTRY: "true"
- GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ github.event.pull_request.head.sha || github.sha }}
+ IMAGE_TAG_FOR_THE_BUILD: "${{ github.event.pull_request.head.sha || github.sha }}"
concurrency:
group: build-${{ github.event.pull_request.number || github.ref }}
@@ -171,8 +169,6 @@ jobs:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn)[0] }}
BACKEND: sqlite
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
- DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
outputs: ${{toJSON(needs.build-info.outputs) }}
steps:
- uses: actions/checkout@v2
@@ -182,7 +178,7 @@ jobs:
submodules: recursive
- name: "Retrieve DEFAULTS from the _initialization.sh"
# We cannot "source" the script here because that would be a security problem (we cannot run
- # any code that comes from the sources coming from the PR. Therefore we extract the
+ # any code that comes from the sources coming from the PR. Therefore, we extract the
# DEFAULT_BRANCH and DEFAULT_CONSTRAINTS_BRANCH and DEBIAN_VERSION via custom grep/awk/sed commands
id: defaults
run: |
@@ -228,15 +224,24 @@ jobs:
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Build CI image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ - name: "Build CI image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: Breeze2 build-ci-image
- - name: "Push CI image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
+ UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
+ DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
+ - name: "Push CI image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_push_ci_images.sh
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
- name: >
Push empty CI images to finish waiting jobs:
- ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
if: failure() || cancelled()
run: ./scripts/ci/images/ci_push_empty_ci_images.sh
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
- name: "Candidates for pip resolver backtrack triggers: ${{ matrix.python-version }}"
if: failure() || cancelled()
run: airflow-find-newer-dependencies --max-age 1 --python "${{ matrix.python-version }}"
@@ -259,11 +264,8 @@ jobs:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn)[0] }}
BACKEND: sqlite
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
VERSION_SUFFIX_FOR_PYPI: ".dev0"
- INSTALL_PROVIDERS_FROM_SOURCES: >
- ${{ needs.build-info.outputs.defaultBranch == 'main' && 'true' || 'false' }}
steps:
- uses: actions/checkout@v2
with:
@@ -318,16 +320,25 @@ jobs:
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image for PROD ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ - name: "Pull CI image for PROD build: ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
env:
- GITHUB_REGISTRY_PULL_IMAGE_TAG: ${{ github.event.pull_request.head.sha || github.sha }}
- - name: "Build PROD image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
+ - name: "Build PROD image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_build_prod_image_on_ci.sh
- - name: "Push PROD image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
+ UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
+ DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
+ - name: "Push PROD image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_push_production_images.sh
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
- name: >
Push empty PROD images to finish waiting jobs:
- ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
if: failure() || cancelled()
run: ./scripts/ci/images/ci_push_empty_prod_images.sh
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3ec3dab..93c575e 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1,4 +1,5 @@
# Licensed to the Apache Software Foundation (ASF) under one
+# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
@@ -34,7 +35,6 @@ env:
DEBIAN_VERSION: "bullseye"
DB_RESET: "true"
VERBOSE: "true"
- DOCKER_CACHE: "pulled"
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_USERNAME: ${{ github.actor }}
# You can override CONSTRAINTS_GITHUB_REPOSITORY by setting secret in your repo but by default the
@@ -44,9 +44,9 @@ env:
secrets.CONSTRAINTS_GITHUB_REPOSITORY || 'apache/airflow' }}
# In builds from forks, this token is read-only. For scheduler/direct push it is WRITE one
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- INSTALL_PROVIDERS_FROM_SOURCES: "true"
AIRFLOW_LOGIN_TO_GITHUB_REGISTRY: "true"
ENABLE_TEST_COVERAGE: "${{ github.event_name == 'push' }}"
+ IMAGE_TAG_FOR_THE_BUILD: "${{ github.event.pull_request.head.sha || github.sha }}"
concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
@@ -242,13 +242,12 @@ jobs:
echo "::set-output name=inWorkflowBuild::false"
echo "::set-output name=buildJobDescription::Skip Build (pull_request_target)"
fi
- - name: Determine if this is merge run (or schedule)
+ - name: Determine if this is merge run
id: set-merge-run
run: echo "::set-output name=merge-run::true"
- # Only in Apache Airflow repo, when either schedule or push to main or any of v2*test branches
+ # Only in Apache Airflow repo, when there is a merge run to main or any of v2*test branches
if: |
- github.repository == 'apache/airflow' &&
- (github.event_name == 'push' || github.event_name == 'schedule') &&
+ github.repository == 'apache/airflow' && github.event_name == 'push' &&
(
github.ref_name == 'main' ||
startsWith(github.ref_name, 'v2') && endsWith(github.ref_name, 'test')
@@ -269,11 +268,6 @@ jobs:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn)[0] }}
BACKEND: sqlite
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
- DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ github.event.pull_request.head.sha || github.sha }}
- GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
outputs: ${{toJSON(needs.build-info.outputs) }}
steps:
- uses: actions/checkout@v2
@@ -309,12 +303,19 @@ jobs:
- name: "Free space"
run: airflow-freespace
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - name: "Build CI image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ - name: "Build CI image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: Breeze2 build-ci-image
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - name: "Push CI image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
+ UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
+ DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
+ - name: "Push CI image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_push_ci_images.sh
if: needs.build-info.outputs.inWorkflowBuild == 'true'
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
- name: "Candidates for pip resolver backtrack triggers: ${{ matrix.python-version }}"
if: failure() || cancelled()
run: airflow-find-newer-dependencies --max-age 1 --python "${{ matrix.python-version }}"
@@ -334,14 +335,8 @@ jobs:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn)[0] }}
BACKEND: sqlite
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
VERSION_SUFFIX_FOR_PYPI: ".dev0"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ github.event.pull_request.head.sha || github.sha }}
- GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
- INSTALL_PROVIDERS_FROM_SOURCES: >
- ${{ needs.build-info.outputs.defaultBranch == 'main' && 'true' || 'false' }}
steps:
- uses: actions/checkout@v2
with:
@@ -376,17 +371,24 @@ jobs:
- name: "Free space"
run: airflow-freespace
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - name: "Pull CI image for PROD ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ - name: "Pull CI image for PROD build: ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
env:
- GITHUB_REGISTRY_PULL_IMAGE_TAG: ${{ github.event.pull_request.head.sha || github.sha }}
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - name: "Build PROD image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ - name: "Build PROD image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_build_prod_image_on_ci.sh
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - name: "Push PROD image ${{ matrix.python-version }}:${{ env.GITHUB_REGISTRY_PUSH_IMAGE_TAG }}"
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
+ UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
+ DOCKER_CACHE: ${{ needs.build-info.outputs.cacheDirective }}
+ - name: "Push PROD image ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_push_production_images.sh
if: needs.build-info.outputs.inWorkflowBuild == 'true'
+ env:
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
run-new-breeze-tests:
timeout-minutes: 10
@@ -528,7 +530,7 @@ jobs:
- name: "Test docker-compose quick start"
run: ./scripts/ci/images/ci_run_docker_compose_quick_start_test.sh
- ci-images:
+ wait-for-ci-images:
timeout-minutes: 120
name: "Wait for CI images"
runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
@@ -537,8 +539,6 @@ jobs:
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: sqlite
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
@@ -560,15 +560,17 @@ jobs:
key: ${{ runner.os }}-docker-venv-${{ hashFiles('scripts/ci/images/ci_run_docker_tests.py') }}
- name: >
Wait for CI images
- ${{ needs.build-info.outputs.pythonVersions }}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}
+ ${{ needs.build-info.outputs.pythonVersions }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}
id: wait-for-images
- # We wait for the images to be available either from the build-images workflow
+ # We wait for the images to be available either from the "build-images.yml" workflow or
+ # from the build-ci-image above.
# We are utilising single job to wait for all images because this job merely waits
# for the images to be available and run tests with the images.
run: ./scripts/ci/images/ci_wait_for_and_verify_all_ci_images.sh
env:
CURRENT_PYTHON_MAJOR_MINOR_VERSIONS_AS_STRING: >
${{needs.build-info.outputs.pythonVersionsListAsString}}
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
VERIFY_IMAGE: "true"
PR_LABELS: "${{ needs.build-info.outputs.pullRequestLabels }}"
@@ -581,7 +583,6 @@ jobs:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
MOUNT_SELECTED_LOCAL_SOURCES: "true"
PYTHON_MAJOR_MINOR_VERSION: ${{needs.build-info.outputs.defaultPythonVersion}}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.basic-checks-only == 'false'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -597,8 +598,10 @@ jobs:
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Get Python version"
run: "echo \"::set-output name=host-python-version::$(python -c
'import platform; print(platform.python_version())')\""
@@ -618,7 +621,6 @@ pre-commit-local-installation-${{steps.host-python-version.outputs.host-python-v
key: "pre-commit-${{steps.host-python-version.outputs.host-python-version}}-\
${{ hashFiles('.pre-commit-config.yaml') }}"
restore-keys: pre-commit-${{steps.host-python-version.outputs.host-python-version}}
-
- name: "Cache eslint"
uses: actions/cache@v2
with:
@@ -706,8 +708,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Fetch inventory versions"
run: ./scripts/ci/docs/ci_docs_prepare.sh
- uses: actions/cache@v2
@@ -747,7 +751,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
VERSION_SUFFIX_FOR_PYPI: ".dev0"
NON_INTERACTIVE: "true"
GENERATE_PROVIDERS_ISSUE: "true"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.image-build == 'true' && needs.build-info.outputs.default-branch == 'main'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -764,8 +767,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Prepare provider documentation"
run: ./scripts/ci/provider_packages/ci_prepare_provider_documentation.sh
- name: "Prepare provider packages: wheel"
@@ -802,7 +807,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
VERSION_SUFFIX_FOR_PYPI: ".dev0"
NON_INTERACTIVE: "true"
GENERATE_PROVIDERS_ISSUE: "true"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.image-build == 'true' && needs.build-info.outputs.default-branch == 'main'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -819,8 +823,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Prepare provider packages: sdist"
run: ./scripts/ci/provider_packages/ci_prepare_provider_packages.sh
env:
@@ -853,7 +859,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
BACKEND: ""
DB_RESET: "false"
PYTHON_MAJOR_MINOR_VERSION: ${{needs.build-info.outputs.defaultPythonVersion}}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: >
needs.build-info.outputs.needs-helm-tests == 'true' &&
(github.repository == 'apache/airflow' || github.event_name != 'schedule') &&
@@ -872,8 +877,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Tests: Helm"
run: ./scripts/ci/testing/ci_run_airflow_testing.sh
env:
@@ -917,11 +924,9 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: postgres
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
POSTGRES_VERSION: ${{ matrix.postgres-version }}
TEST_TYPES: "${{needs.build-info.outputs.testTypes}}"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.run-tests == 'true'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -937,8 +942,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Test downgrade"
run: ./scripts/ci/testing/run_downgrade_test.sh
- name: "Test Offline SQL generation"
@@ -985,11 +992,9 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: mysql
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
MYSQL_VERSION: ${{ matrix.mysql-version }}
TEST_TYPES: "${{needs.build-info.outputs.testTypes}}"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.run-tests == 'true'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -1005,8 +1010,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Test downgrade"
run: ./scripts/ci/testing/run_downgrade_test.sh
- name: "Test Offline SQL generation"
@@ -1052,11 +1059,9 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: mssql
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
MSSQL_VERSION: ${{ matrix.mssql-version }}
TEST_TYPES: "${{needs.build-info.outputs.testTypes}}"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.run-tests == 'true'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -1072,8 +1077,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Test downgrade"
run: ./scripts/ci/testing/run_downgrade_test.sh
- name: "Tests: ${{needs.build-info.outputs.testTypes}}"
@@ -1116,10 +1123,8 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: sqlite
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
TEST_TYPES: "${{needs.build-info.outputs.testTypes}}"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.run-tests == 'true'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -1135,8 +1140,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Test downgrade"
run: ./scripts/ci/testing/run_downgrade_test.sh
- name: "Tests: ${{needs.build-info.outputs.testTypes}}"
@@ -1179,7 +1186,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
TEST_TYPES: "Quarantined"
NUM_RUNS: 10
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.run-tests == 'true'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -1207,8 +1213,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ - name: "Pull CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Tests: Quarantined"
run: ./scripts/ci/testing/ci_run_quarantined_tests.sh
env:
@@ -1275,7 +1283,7 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
with:
directory: "./coverage-files"
- prod-images:
+ wait-for-prod-images:
timeout-minutes: 120
name: "Wait for PROD images"
runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
@@ -1285,8 +1293,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
BACKEND: sqlite
PYTHON_MAJOR_MINOR_VERSION: ${{ needs.build-info.outputs.defaultPythonVersion }}
- UPGRADE_TO_NEWER_DEPENDENCIES: ${{ needs.build-info.outputs.upgradeToNewerDependencies }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
@@ -1308,16 +1314,17 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
key: ${{ runner.os }}-docker-venv-${{ hashFiles('scripts/ci/images/ci_run_docker_tests.py') }}
- name: >
Wait for PROD images
- ${{ needs.build-info.outputs.pythonVersions }}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}
- # We wait for the images to be available either from "build-images.yml' run as pull_request_target.
+ ${{ needs.build-info.outputs.pythonVersions }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}
+ # We wait for the images to be available either from "build-images.yml' run as pull_request_target
+ # or from build-prod-image above.
# We are utilising single job to wait for all images because this job merely waits
# For the images to be available and test them.
- #
id: wait-for-images
run: ./scripts/ci/images/ci_wait_for_and_verify_all_prod_images.sh
env:
CURRENT_PYTHON_MAJOR_MINOR_VERSIONS_AS_STRING: >
${{needs.build-info.outputs.pythonVersionsListAsString}}
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
VERIFY_IMAGE: "true"
PR_LABELS: "${{ needs.build-info.outputs.pullRequestLabels }}"
@@ -1343,7 +1350,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
${{needs.build-info.outputs.pythonVersionsListAsString}}
CURRENT_KUBERNETES_VERSIONS_AS_STRING: >
${{needs.build-info.outputs.kubernetesVersionsListAsString}}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: >
( needs.build-info.outputs.run-kubernetes-tests == 'true' ||
needs.build-info.outputs.needs-helm-tests == 'true' ) &&
@@ -1362,9 +1368,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Get all PROD images"
+ - name: "Get all PROD images: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_wait_for_and_verify_all_prod_images.sh
env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
VERIFY_IMAGE: "false"
- name: "Cache bin folder with tools for kubernetes testing"
uses: actions/cache@v2
@@ -1405,7 +1412,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
${{needs.build-info.outputs.pythonVersionsListAsString}}
CURRENT_KUBERNETES_VERSIONS_AS_STRING: >
${{needs.build-info.outputs.kubernetesVersionsListAsString}}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: >
needs.build-info.outputs.run-kubernetes-tests == 'true' &&
needs.build-info.outputs.default-branch == 'main'
@@ -1423,9 +1429,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: python -m pip install --editable ./dev/breeze/
- name: "Free space"
run: airflow-freespace
- - name: "Get all PROD images"
+ - name: "Get all PROD images: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
run: ./scripts/ci/images/ci_wait_for_and_verify_all_prod_images.sh
env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
VERIFY_IMAGE: "false"
- name: "Cache virtualenv for kubernetes testing"
uses: actions/cache@v2
@@ -1478,7 +1485,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
CURRENT_PYTHON_MAJOR_MINOR_VERSIONS_AS_STRING: ${{needs.build-info.outputs.pythonVersionsListAsString}}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ github.event.pull_request.head.sha || github.sha }}"
if: needs.build-info.outputs.upgradeToNewerDependencies != 'false'
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
@@ -1497,9 +1503,10 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
run: airflow-freespace
- name: >
Wait for CI images
- ${{ needs.build-info.outputs.pythonVersions }}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}
+ ${{ needs.build-info.outputs.pythonVersions }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}
run: ./scripts/ci/images/ci_wait_for_and_verify_all_ci_images.sh
env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
VERIFY_IMAGE: "false"
- name: "Generate constraints with PyPI providers"
run: ./scripts/ci/constraints/ci_generate_all_constraints.sh
@@ -1559,11 +1566,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
env:
RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- # Rebuild images before push using the latest constraints (just pushed) without
- # eager upgrade. Do not wait for images, but rebuild them
- UPGRADE_TO_NEWER_DEPENDENCIES: "false"
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_PUSH_IMAGE_TAG: "latest"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
@@ -1583,14 +1585,24 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- name: "Build CI image cache and push ${{env.PYTHON_MAJOR_MINOR_VERSION}}"
run: ./scripts/ci/images/ci_build_ci_image_on_ci.sh
env:
- PREPARE_BUILDX_CACHE: "true"
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: "latest"
+ UPGRADE_TO_NEWER_DEPENDENCIES: "false"
PLATFORM: "linux/amd64,linux/arm64"
+ PREPARE_BUILDX_CACHE: "true"
+ - name: "Pull CI image for PROD build: ${{ matrix.python-version }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
+ run: ./scripts/ci/images/ci_pull_ci_image_on_ci.sh
+ env:
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
- name: "Build PROD image cache and push ${{env.PYTHON_MAJOR_MINOR_VERSION}}"
run: ./scripts/ci/images/ci_build_prod_image_on_ci.sh
env:
- VERSION_SUFFIX_FOR_PYPI: ".dev0"
- PREPARE_BUILDX_CACHE: "true"
+ GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
+ GITHUB_REGISTRY_PUSH_IMAGE_TAG: "latest"
+ UPGRADE_TO_NEWER_DEPENDENCIES: "false"
PLATFORM: "linux/amd64,linux/arm64"
+ PREPARE_BUILDX_CACHE: "true"
+ VERSION_SUFFIX_FOR_PYPI: ".dev0"
- name: "Stop ARM instance"
run: ./scripts/ci/images/ci_stop_arm_instance.sh
if: always()
diff --git a/scripts/ci/images/ci_build_prod_image_on_ci.sh b/scripts/ci/images/ci_build_prod_image_on_ci.sh
index 5e30610..444c333 100755
--- a/scripts/ci/images/ci_build_prod_image_on_ci.sh
+++ b/scripts/ci/images/ci_build_prod_image_on_ci.sh
@@ -20,7 +20,6 @@ export INSTALL_FROM_PYPI="false"
export INSTALL_PROVIDERS_FROM_SOURCES="false"
export INSTALL_FROM_DOCKER_CONTEXT_FILES="true"
export AIRFLOW_PRE_CACHED_PIP_PACKAGES="false"
-export DOCKER_CACHE="pulled"
export VERBOSE="true"
# shellcheck source=scripts/ci/libraries/_script_init.sh
diff --git a/scripts/ci/images/ci_pull_prod_image_on_ci.sh b/scripts/ci/images/ci_pull_prod_image_on_ci.sh
index 997f3c3..7b6f9cb 100755
--- a/scripts/ci/images/ci_pull_prod_image_on_ci.sh
+++ b/scripts/ci/images/ci_pull_prod_image_on_ci.sh
@@ -16,14 +16,8 @@
# specific language governing permissions and limitations
# under the License.
-export INSTALL_FROM_PYPI="false"
-export INSTALL_PROVIDERS_FROM_SOURCES="false"
-export INSTALL_FROM_DOCKER_CONTEXT_FILES="true"
-export AIRFLOW_PRE_CACHED_PIP_PACKAGES="false"
-export DOCKER_CACHE="pulled"
export VERBOSE="true"
-
# shellcheck source=scripts/ci/libraries/_script_init.sh
. "$( dirname "${BASH_SOURCE[0]}" )/../libraries/_script_init.sh"
diff --git a/scripts/ci/tools/build_dockerhub.sh b/scripts/ci/tools/build_dockerhub.sh
deleted file mode 100755
index 82c1fef..0000000
--- a/scripts/ci/tools/build_dockerhub.sh
+++ /dev/null
@@ -1,59 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# shellcheck disable=SC2030,SC2031
-: "${PYTHON_MAJOR_MINOR_VERSION:?"ERROR: PYTHON_MAJOR_MINOR_VERSION not set !!!!"}"
-: "${INSTALL_AIRFLOW_VERSION:?"ERROR: INSTALL_AIRFLOW_VERSION not set !!!!"}"
-export FORCE_ANSWER_TO_QUESTIONS="yes"
-export VERBOSE="true"
-# This is an image built from the "release" tag (either RC or final one).
-# In this case all packages are taken from PyPI rather than from locally built sources
-export INSTALL_FROM_PYPI="true"
-export INSTALL_FROM_DOCKER_CONTEXT_FILES="false"
-export INSTALL_PROVIDERS_FROM_SOURCES="false"
-export AIRFLOW_PRE_CACHED_PIP_PACKAGES="false"
-export DOCKER_CACHE="local"
-export DOCKER_TAG=${INSTALL_AIRFLOW_VERSION}-python${PYTHON_MAJOR_MINOR_VERSION}
-export AIRFLOW_CONSTRAINTS_REFERENCE="constraints-${INSTALL_AIRFLOW_VERSION}"
-export AIRFLOW_CONSTRAINTS="constraints"
-# shellcheck source=scripts/ci/libraries/_script_init.sh
-. "$( dirname "${BASH_SOURCE[0]}" )/../libraries/_script_init.sh"
-echo
-echo "Building and pushing ${INSTALL_AIRFLOW_VERSION} Airflow PROD image for ${PYTHON_MAJOR_MINOR_VERSION}"
-echo
-build_images::clean_build_cache
-build_images::prepare_prod_build
-build_images::build_prod_images
-verify_image::verify_prod_image "${AIRFLOW_PROD_IMAGE}"
-
-export RELEASE_IMAGE="apache/airflow:${INSTALL_AIRFLOW_VERSION}-python${PYTHON_MAJOR_MINOR_VERSION}"
-echo
-echo "Pushing airflow PROD image as ${RELEASE_IMAGE}"
-echo
-# Re-tag the image to be published in "apache/airflow"
-docker tag "${AIRFLOW_PROD_IMAGE}" "${RELEASE_IMAGE}"
-docker push "${RELEASE_IMAGE}"
-if [[ ${PYTHON_MAJOR_MINOR_VERSION} == "${DEFAULT_PYTHON_MAJOR_MINOR_VERSION}" ]]; then
- export DEFAULT_VERSION_IMAGE="apache/airflow:${INSTALL_AIRFLOW_VERSION}"
- echo
- echo "Pushing default airflow image as ${DEFAULT_VERSION_IMAGE}"
- echo
- # In case of default Python version we also push ":version" tag
- docker tag "${RELEASE_IMAGE}" "${DEFAULT_VERSION_IMAGE}"
- docker push "${DEFAULT_VERSION_IMAGE}"
-fi