You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/19 14:53:49 UTC
[GitHub] [airflow] ashb opened a new pull request #15944: Streamline Build Images workflow using new GitHub Actions features
ashb opened a new pull request #15944:
URL: https://github.com/apache/airflow/pull/15944
Use `pull_request_target` event for building images, and `concurrency` to
automatically cancel old jobs for PRs.
This means that:
- GitHub will automatically cancel old jobs for us, so we don't have to
handle that ourselves (removes most of the use of the
cancel-workflow-action)
New pushes show this for a little while
And then the old job shows this as the reason
- GitHub displays these checks directly on the PR, but it is still run
in the context of our repo, meaning it has access write to our
repo/access to secrets etc.
- Since it shows up directly on the PR checks, we don't need to create the
check in the "CI" workflow to show the status of the Image Build.
- We also don't need to post the comment saying _why_ it failed, as the
Build Image status will show up directly there
- Since `pull_request_target` has information about the PR in the
`github.event` context, we don't need the complex mechanism to find
the "other" PR, we can do a fairly simple API request and filter by
the commit SHA to find and cancel to CI workflow job. (This removes
the final use of the cancel-workflow-action)
One change I had to make here what tag we use for Docker images we build
and push up. Previously we used the "source run ID" (i.e. the id of the
CI run) but with pull_request_target we don't have that anymore. We
could use the same API mechanism we do to cancel to find the target job,
but the only requirement here is for an ID that both jobs know -- the
SHA of the PR branch fills that need
Extra side benefits of this:
- The sidebar of commits in main branch aren't "polluted" with Build
Images for PRs like they were previously.
Still do do:
- [ ] Update breeze to pull the new image tag format when given a numeric id (use API to look up sha that a build is for)
- [ ] Delete old build-image-workflow-run workflow file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#discussion_r636050674
##########
File path: .github/workflows/build-images-workflow-run.yml
##########
@@ -1,561 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
----
-name: "Build Images"
-on: # yamllint disable-line rule:truthy
- workflow_run:
- workflows: ["CI Build"]
- types: ['requested']
-env:
- MOUNT_SELECTED_LOCAL_SOURCES: "false"
- FORCE_ANSWER_TO_QUESTIONS: "yes"
- FORCE_PULL_IMAGES: "false"
- CHECK_IMAGE_FOR_REBUILD: "true"
- SKIP_CHECK_REMOTE_IMAGE: "true"
- DB_RESET: "true"
- VERBOSE: "true"
- USE_GITHUB_REGISTRY: "true"
- GITHUB_REPOSITORY: ${{ github.repository }}
- GITHUB_USERNAME: ${{ github.actor }}
- # You can override CONSTRAINTS_GITHUB_REPOSITORY by setting secret in your repo but by default the
- # Airflow one is going to be used
- CONSTRAINTS_GITHUB_REPOSITORY: >-
- ${{ secrets.CONSTRAINTS_GITHUB_REPOSITORY != '' &&
- secrets.CONSTRAINTS_GITHUB_REPOSITORY || 'apache/airflow' }}
- # This token is WRITE one - workflow_run type of events always have the WRITE token
- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- # This token should not be empty in workflow_run type of event.
- CONTAINER_REGISTRY_TOKEN: ${{ secrets.PAT_CR }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
- BUILD_IMAGES: ${{ secrets.AIRFLOW_GITHUB_REGISTRY_WAIT_FOR_IMAGE != 'false' }}
- INSTALL_PROVIDERS_FROM_SOURCES: "true"
- GITHUB_REGISTRY: ${{ secrets.OVERRIDE_GITHUB_REGISTRY }}
-
-jobs:
-
- cancel-workflow-runs:
- timeout-minutes: 10
- name: "Cancel workflow runs"
- runs-on: ${{ github.repository == 'apache/airflow' && 'self-hosted' || 'ubuntu-20.04' }}
- outputs:
- sourceHeadRepo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- sourceHeadBranch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}
- sourceHeadSha: ${{ steps.source-run-info.outputs.sourceHeadSha }}
- mergeCommitSha: ${{ steps.source-run-info.outputs.mergeCommitSha }}
- targetCommitSha: ${{ steps.source-run-info.outputs.targetCommitSha }}
- pullRequestNumber: ${{ steps.source-run-info.outputs.pullRequestNumber }}
- pullRequestLabels: ${{ steps.source-run-info.outputs.pullRequestLabels }}
- targetBranch: ${{ steps.source-run-info.outputs.targetBranch }}
- sourceEvent: ${{ steps.source-run-info.outputs.sourceEvent }}
- cacheDirective: ${{ steps.cache-directive.outputs.docker-cache }}
- buildImages: ${{ steps.build-images.outputs.buildImages }}
- runsOn: ${{ github.repository == 'apache/airflow' && '["self-hosted"]' || '["ubuntu-20.04"]' }}
- steps:
- - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
- uses: actions/checkout@v2
- with:
- persist-credentials: false
- submodules: recursive
- - name: "Get information about the original trigger of the run"
- uses: ./.github/actions/get-workflow-origin
- id: source-run-info
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Cancel duplicated 'CI Build' runs"
- uses: ./.github/actions/cancel-workflow-runs
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: allDuplicates
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Output BUILD_IMAGES"
- id: build-images
- run: |
- # Workaround - jobs cannot access env variable in "ifs"
- # https://github.community/t/how-to-set-and-access-a-workflow-variable/17335/16
- echo "::set-output name=buildImages::${BUILD_IMAGES}"
- - name: "Cancel duplicated 'Build Image' runs"
- # We find duplicates of our own "Build Image" runs - due to a missing feature
- # in GitHub Actions, we have to use Job names to match Event/Repo/Branch matching
- # trick ¯\_(ツ)_/¯. We name the build-info job appropriately
- # and then we try to find and cancel all the jobs with the same Event + Repo + Branch as the
- # current Event/Repo/Branch combination.
- uses: ./.github/actions/cancel-workflow-runs
- with:
- cancelMode: namedJobs
- token: ${{ secrets.GITHUB_TOKEN }}
- notifyPRCancel: true
- jobNameRegexps: >
- [".*Event: ${{ steps.source-run-info.outputs.sourceEvent }}
- Repo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- Branch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}.*"]
- if: env.BUILD_IMAGES == 'true'
- - name: "Cancel all 'CI Build' runs where some jobs failed"
- # We find any of the "CI Build" workflow runs, where any of the important jobs
- # failed. The important jobs are selected by the regexp array below.
- # We also produce list of canceled "CI Build' runs as output, so that we
- # can cancel all the matching "Build Images" workflow runs in the two following steps.
- # Yeah. Adding to the complexity ¯\_(ツ)_/¯.
- uses: ./.github/actions/cancel-workflow-runs
- id: cancel-failed
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: failedJobs
- sourceRunId: ${{ github.event.workflow_run.id }}
- notifyPRCancel: true
- jobNameRegexps: >
- ["^Pylint$", "^Static checks", "^Build docs$", "^Spell check docs$",
- "^Provider packages", "^Checks: Helm tests$", "^Test OpenAPI*"]
Review comment:
It's an optimisation only, and I am not sure if thats' so much needed after we sped up most of the builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#discussion_r636051783
##########
File path: .github/workflows/build-images-workflow-run.yml
##########
@@ -1,561 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
----
-name: "Build Images"
-on: # yamllint disable-line rule:truthy
- workflow_run:
- workflows: ["CI Build"]
- types: ['requested']
-env:
- MOUNT_SELECTED_LOCAL_SOURCES: "false"
- FORCE_ANSWER_TO_QUESTIONS: "yes"
- FORCE_PULL_IMAGES: "false"
- CHECK_IMAGE_FOR_REBUILD: "true"
- SKIP_CHECK_REMOTE_IMAGE: "true"
- DB_RESET: "true"
- VERBOSE: "true"
- USE_GITHUB_REGISTRY: "true"
- GITHUB_REPOSITORY: ${{ github.repository }}
- GITHUB_USERNAME: ${{ github.actor }}
- # You can override CONSTRAINTS_GITHUB_REPOSITORY by setting secret in your repo but by default the
- # Airflow one is going to be used
- CONSTRAINTS_GITHUB_REPOSITORY: >-
- ${{ secrets.CONSTRAINTS_GITHUB_REPOSITORY != '' &&
- secrets.CONSTRAINTS_GITHUB_REPOSITORY || 'apache/airflow' }}
- # This token is WRITE one - workflow_run type of events always have the WRITE token
- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- # This token should not be empty in workflow_run type of event.
- CONTAINER_REGISTRY_TOKEN: ${{ secrets.PAT_CR }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
- BUILD_IMAGES: ${{ secrets.AIRFLOW_GITHUB_REGISTRY_WAIT_FOR_IMAGE != 'false' }}
- INSTALL_PROVIDERS_FROM_SOURCES: "true"
- GITHUB_REGISTRY: ${{ secrets.OVERRIDE_GITHUB_REGISTRY }}
-
-jobs:
-
- cancel-workflow-runs:
- timeout-minutes: 10
- name: "Cancel workflow runs"
- runs-on: ${{ github.repository == 'apache/airflow' && 'self-hosted' || 'ubuntu-20.04' }}
- outputs:
- sourceHeadRepo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- sourceHeadBranch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}
- sourceHeadSha: ${{ steps.source-run-info.outputs.sourceHeadSha }}
- mergeCommitSha: ${{ steps.source-run-info.outputs.mergeCommitSha }}
- targetCommitSha: ${{ steps.source-run-info.outputs.targetCommitSha }}
- pullRequestNumber: ${{ steps.source-run-info.outputs.pullRequestNumber }}
- pullRequestLabels: ${{ steps.source-run-info.outputs.pullRequestLabels }}
- targetBranch: ${{ steps.source-run-info.outputs.targetBranch }}
- sourceEvent: ${{ steps.source-run-info.outputs.sourceEvent }}
- cacheDirective: ${{ steps.cache-directive.outputs.docker-cache }}
- buildImages: ${{ steps.build-images.outputs.buildImages }}
- runsOn: ${{ github.repository == 'apache/airflow' && '["self-hosted"]' || '["ubuntu-20.04"]' }}
- steps:
- - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
- uses: actions/checkout@v2
- with:
- persist-credentials: false
- submodules: recursive
- - name: "Get information about the original trigger of the run"
- uses: ./.github/actions/get-workflow-origin
- id: source-run-info
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Cancel duplicated 'CI Build' runs"
- uses: ./.github/actions/cancel-workflow-runs
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: allDuplicates
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Output BUILD_IMAGES"
- id: build-images
- run: |
- # Workaround - jobs cannot access env variable in "ifs"
- # https://github.community/t/how-to-set-and-access-a-workflow-variable/17335/16
- echo "::set-output name=buildImages::${BUILD_IMAGES}"
- - name: "Cancel duplicated 'Build Image' runs"
- # We find duplicates of our own "Build Image" runs - due to a missing feature
- # in GitHub Actions, we have to use Job names to match Event/Repo/Branch matching
- # trick ¯\_(ツ)_/¯. We name the build-info job appropriately
- # and then we try to find and cancel all the jobs with the same Event + Repo + Branch as the
- # current Event/Repo/Branch combination.
- uses: ./.github/actions/cancel-workflow-runs
- with:
- cancelMode: namedJobs
- token: ${{ secrets.GITHUB_TOKEN }}
- notifyPRCancel: true
- jobNameRegexps: >
- [".*Event: ${{ steps.source-run-info.outputs.sourceEvent }}
- Repo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- Branch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}.*"]
- if: env.BUILD_IMAGES == 'true'
- - name: "Cancel all 'CI Build' runs where some jobs failed"
- # We find any of the "CI Build" workflow runs, where any of the important jobs
- # failed. The important jobs are selected by the regexp array below.
- # We also produce list of canceled "CI Build' runs as output, so that we
- # can cancel all the matching "Build Images" workflow runs in the two following steps.
- # Yeah. Adding to the complexity ¯\_(ツ)_/¯.
- uses: ./.github/actions/cancel-workflow-runs
- id: cancel-failed
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: failedJobs
- sourceRunId: ${{ github.event.workflow_run.id }}
- notifyPRCancel: true
- jobNameRegexps: >
- ["^Pylint$", "^Static checks", "^Build docs$", "^Spell check docs$",
- "^Provider packages", "^Checks: Helm tests$", "^Test OpenAPI*"]
Review comment:
Selective checks already take care about most savings we have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-847925754
🤞🏻
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-844229319
> Update breeze to pull the new image tag format when given a numeric id (use API to look up sha that a build is for)
Ths might be not needed. We just need to print SHA in Breeze in instructions and we can use SHA directly. No need to use build ID, maybe just rename the parameter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-847801068
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-844183876
I've been testing this out on my fork here https://github.com/ashb/airflow/pull/9 -- that is a PR opened from a different user.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-844229319
> Update breeze to pull the new image tag format when given a numeric id (use API to look up sha that a build is for)
Ths might be not needed. We just need to print SHA in Breeze in instructions and we can use SHA directly. No need to use build ID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#discussion_r636026437
##########
File path: .github/workflows/build-images-workflow-run.yml
##########
@@ -1,561 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
----
-name: "Build Images"
-on: # yamllint disable-line rule:truthy
- workflow_run:
- workflows: ["CI Build"]
- types: ['requested']
-env:
- MOUNT_SELECTED_LOCAL_SOURCES: "false"
- FORCE_ANSWER_TO_QUESTIONS: "yes"
- FORCE_PULL_IMAGES: "false"
- CHECK_IMAGE_FOR_REBUILD: "true"
- SKIP_CHECK_REMOTE_IMAGE: "true"
- DB_RESET: "true"
- VERBOSE: "true"
- USE_GITHUB_REGISTRY: "true"
- GITHUB_REPOSITORY: ${{ github.repository }}
- GITHUB_USERNAME: ${{ github.actor }}
- # You can override CONSTRAINTS_GITHUB_REPOSITORY by setting secret in your repo but by default the
- # Airflow one is going to be used
- CONSTRAINTS_GITHUB_REPOSITORY: >-
- ${{ secrets.CONSTRAINTS_GITHUB_REPOSITORY != '' &&
- secrets.CONSTRAINTS_GITHUB_REPOSITORY || 'apache/airflow' }}
- # This token is WRITE one - workflow_run type of events always have the WRITE token
- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- # This token should not be empty in workflow_run type of event.
- CONTAINER_REGISTRY_TOKEN: ${{ secrets.PAT_CR }}
- GITHUB_REGISTRY_PULL_IMAGE_TAG: "latest"
- GITHUB_REGISTRY_WAIT_FOR_IMAGE: "false"
- BUILD_IMAGES: ${{ secrets.AIRFLOW_GITHUB_REGISTRY_WAIT_FOR_IMAGE != 'false' }}
- INSTALL_PROVIDERS_FROM_SOURCES: "true"
- GITHUB_REGISTRY: ${{ secrets.OVERRIDE_GITHUB_REGISTRY }}
-
-jobs:
-
- cancel-workflow-runs:
- timeout-minutes: 10
- name: "Cancel workflow runs"
- runs-on: ${{ github.repository == 'apache/airflow' && 'self-hosted' || 'ubuntu-20.04' }}
- outputs:
- sourceHeadRepo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- sourceHeadBranch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}
- sourceHeadSha: ${{ steps.source-run-info.outputs.sourceHeadSha }}
- mergeCommitSha: ${{ steps.source-run-info.outputs.mergeCommitSha }}
- targetCommitSha: ${{ steps.source-run-info.outputs.targetCommitSha }}
- pullRequestNumber: ${{ steps.source-run-info.outputs.pullRequestNumber }}
- pullRequestLabels: ${{ steps.source-run-info.outputs.pullRequestLabels }}
- targetBranch: ${{ steps.source-run-info.outputs.targetBranch }}
- sourceEvent: ${{ steps.source-run-info.outputs.sourceEvent }}
- cacheDirective: ${{ steps.cache-directive.outputs.docker-cache }}
- buildImages: ${{ steps.build-images.outputs.buildImages }}
- runsOn: ${{ github.repository == 'apache/airflow' && '["self-hosted"]' || '["ubuntu-20.04"]' }}
- steps:
- - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
- uses: actions/checkout@v2
- with:
- persist-credentials: false
- submodules: recursive
- - name: "Get information about the original trigger of the run"
- uses: ./.github/actions/get-workflow-origin
- id: source-run-info
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Cancel duplicated 'CI Build' runs"
- uses: ./.github/actions/cancel-workflow-runs
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: allDuplicates
- sourceRunId: ${{ github.event.workflow_run.id }}
- - name: "Output BUILD_IMAGES"
- id: build-images
- run: |
- # Workaround - jobs cannot access env variable in "ifs"
- # https://github.community/t/how-to-set-and-access-a-workflow-variable/17335/16
- echo "::set-output name=buildImages::${BUILD_IMAGES}"
- - name: "Cancel duplicated 'Build Image' runs"
- # We find duplicates of our own "Build Image" runs - due to a missing feature
- # in GitHub Actions, we have to use Job names to match Event/Repo/Branch matching
- # trick ¯\_(ツ)_/¯. We name the build-info job appropriately
- # and then we try to find and cancel all the jobs with the same Event + Repo + Branch as the
- # current Event/Repo/Branch combination.
- uses: ./.github/actions/cancel-workflow-runs
- with:
- cancelMode: namedJobs
- token: ${{ secrets.GITHUB_TOKEN }}
- notifyPRCancel: true
- jobNameRegexps: >
- [".*Event: ${{ steps.source-run-info.outputs.sourceEvent }}
- Repo: ${{ steps.source-run-info.outputs.sourceHeadRepo }}
- Branch: ${{ steps.source-run-info.outputs.sourceHeadBranch }}.*"]
- if: env.BUILD_IMAGES == 'true'
- - name: "Cancel all 'CI Build' runs where some jobs failed"
- # We find any of the "CI Build" workflow runs, where any of the important jobs
- # failed. The important jobs are selected by the regexp array below.
- # We also produce list of canceled "CI Build' runs as output, so that we
- # can cancel all the matching "Build Images" workflow runs in the two following steps.
- # Yeah. Adding to the complexity ¯\_(ツ)_/¯.
- uses: ./.github/actions/cancel-workflow-runs
- id: cancel-failed
- with:
- token: ${{ secrets.GITHUB_TOKEN }}
- cancelMode: failedJobs
- sourceRunId: ${{ github.event.workflow_run.id }}
- notifyPRCancel: true
- jobNameRegexps: >
- ["^Pylint$", "^Static checks", "^Build docs$", "^Spell check docs$",
- "^Provider packages", "^Checks: Helm tests$", "^Test OpenAPI*"]
Review comment:
Oh, this step is still probably needed, meaning we can't remove the cancel-workflow_runs action entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-847801425
I think no rebase is neeed. It should **just work** . The name of images did not change, and this is the only interface between the workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #15944:
URL: https://github.com/apache/airflow/pull/15944
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-847774158
@potiuk Cool, can you give this a ✅ then please?
And then once merged lets see if we need every open PR to rebase to pick it up or not :/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-844229319
> Update breeze to pull the new image tag format when given a numeric id (use API to look up sha that a build is for)
Ths might be not needed. We just need to print SHA in output of tests in instructions and we can use SHA directly. No need to use build ID, maybe just rename the parameter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15944: Streamline Build Images workflow using new GitHub Actions features
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15944:
URL: https://github.com/apache/airflow/pull/15944#issuecomment-847748807
Happy to merge this one even without the "fast-fail" for the names of the jobs. I have already put the "deprecation notice" in my action : https://github.com/potiuk/cancel-workflow-runs#readme after the `concurrency` feature added by GitHub Actions..
One less thing to maintain :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org