[airflow] 02/02: Forces rebuilding the image for cache pushing

[ka...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a commit to branch v2-1-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit dff4950fe1153f9873f40086de8f4923ab05789f
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Aug 16 16:07:47 2021 +0200

    Forces rebuilding the image for cache pushing
    
    Fixes bug in pushing latest image to cache on "push/schedule".
    
    When the build is successful and passes all tests vi either
    `push' or 'schedule' events, we attempt to rebuild the image
    with latest constraints just pushed and push it as a fresh
    cache for Github Registry. This keeps the time to build image
    small without manually refreshing the cache, it also automatically
    checks if there is a new "python" base image available so that
    we can use it in the new cache.
    
    There was a bug that the image has not been FORCE_PULLED and
    rebuilt in this case - just latest images were used.
    
    This had so far no negative effects because due to test
    instability, latest main images pretty much never succeeded in
    all tests, so the images in `main` were refreshed manually
    periodically anyway. However for v2-1-test the scope of tests
    run is far smaller now (no Helm tests, no Provider tests)
    and they succeed mostly when they should.
    
    Also PROD image was built without ".dev0" suffix which
    also failed.
    
    This PR fixes it so that the images are built properly and pushed.
    
    (cherry picked from commit dac71ef45fdbeba585ddb5b4211b31ef8643371f)
---
 .github/workflows/ci.yml | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0d65286..f8bc251 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1145,10 +1145,18 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
     env:
       RUNS_ON: ${{ fromJson(needs.build-info.outputs.runsOn) }}
       PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
+      # Rebuild images before push using the latest constraints (just pushed) without
+      # eager upgrade. Do not wait for images, but rebuild them, but always check if
+      # there is a new Python base image to pull and rebuild. This way, when latest python
+      # is not pushed, we will re-use the last cache to start from and when there is
+      # a new python image, we will rebuild it from scratch (same as during the "build-images.ci")
       GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
       GITHUB_REGISTRY_PUSH_IMAGE_TAG: "latest"
       PUSH_PYTHON_BASE_IMAGE: "true"
+      FORCE_PULL_IMAGES: "true"
       CHECK_IF_BASE_PYTHON_IMAGE_UPDATED: "true"
+      GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
+      UPGRADE_TO_NEWER_DEPENDENCIES: "false"
     steps:
       - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
         uses: actions/checkout@v2
@@ -1164,6 +1172,8 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
         run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
       - name: "Prepare PROD image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:latest"
         run: ./scripts/ci/images/ci_prepare_prod_image_on_ci.sh
+        env:
+          VERSION_SUFFIX_FOR_PYPI: ".dev0"
       - name: "Push CI image ${{ env.PYTHON_MAJOR_MINOR_VERSION }}:latest"
         run: ./scripts/ci/images/ci_push_ci_images.sh
       - name: "Push PROD images ${{ env.PYTHON_MAJOR_MINOR_VERSION }}:latest"