You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/18 22:33:42 UTC
[GitHub] [airflow] potiuk opened a new pull request #20939: Fix cancelling of Pull Request builds when image build fails
potiuk opened a new pull request #20939:
URL: https://github.com/apache/airflow/pull/20939
When image build fails, the pull request that triggered it should
be cancelled. The #15944 introduced rewrite of the GitHub actions
code but by mistake it also introduced a failure in cancelling
the PR workflow by missing pipeline to jq.
In most cases it did not matter, but it cause "wait for images"
in PRs to run far longer than they should be.
This PR restores cancelling feature.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20939:
URL: https://github.com/apache/airflow/pull/20939#issuecomment-1015895088
Hey @ashb - I will test the cancelling after the change but I believe it might explain some "far too long" runnig "wait for image" jobs in case we had breaking "build image" changes in main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #20939:
URL: https://github.com/apache/airflow/pull/20939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #20939:
URL: https://github.com/apache/airflow/pull/20939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20939:
URL: https://github.com/apache/airflow/pull/20939#issuecomment-1016234538
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
malthe commented on pull request #20939:
URL: https://github.com/apache/airflow/pull/20939#issuecomment-1016182628
I propose a change to use `--jq` which is used elsewhere in the script rather than piping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20939:
URL: https://github.com/apache/airflow/pull/20939#issuecomment-1016214503
> I propose a change to use `--jq` which is used elsewhere in the script rather than piping.
Thanks @malthe - we are in the process of rewriting all the scripts to python (including those small snippets) as part of #12282 so let's have minimal changes here.
I tested tha in https://github.com/potiuk/airflow/runs/4865079794?check_suite_focus=true and piping works as expected:
```
if [[ "push" == 'pull_request_target' ]]; then
event_filter="event=pull_request&"
else
branch="${GITHUB_REF#refs/heads/}"
event_filter=""
fi
for cancel_url in $(
gh api "/repos/$GITHUB_REPOSITORY/actions/runs?${event_filter}branch=${branch}" | \
jq -r '
.workflow_runs[] |
select(.head_sha == $ENV.GITHUB_REGISTRY_PUSH_IMAGE_TAG and .status != "completed") |
.cancel_url
' \
); do
# One of these URls will be _this_ workflow, so lets exclude that!
[[ $cancel_url == */$thisRun/* ]] && continue
echo "Cancelling $cancel_url"
gh api -X POST --silent "$cancel_url"
done
shell: /usr/bin/bash -e {0}
env:
MOUNT_SELECTED_LOCAL_SOURCES: false
FORCE_ANSWER_TO_QUESTIONS: yes
CHECK_IMAGE_FOR_REBUILD: true
SKIP_CHECK_REMOTE_IMAGE: true
DB_RESET: true
VERBOSE: true
GITHUB_REPOSITORY: potiuk/airflow
GITHUB_USERNAME: potiuk
CONSTRAINTS_GITHUB_REPOSITORY: apache/airflow
GITHUB_TOKEN: ***
GITHUB_REGISTRY_PULL_IMAGE_TAG: latest
GITHUB_REGISTRY_WAIT_FOR_IMAGE: false
INSTALL_PROVIDERS_FROM_SOURCES: true
AIRFLOW_LOGIN_TO_GITHUB_REGISTRY: true
GITHUB_REGISTRY_PUSH_IMAGE_TAG: 2050e628f8e5fd18b8b13b5c52dcbf3daad003bc
branch:
thisRun: 1717050085
Cancelling https://api.github.com/repos/potiuk/airflow/actions/runs/1717050086/cancel
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20939: Fix cancelling of Pull Request builds when image build fails
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20939:
URL: https://github.com/apache/airflow/pull/20939#issuecomment-1016214503
> I propose a change to use `--jq` which is used elsewhere in the script rather than piping.
Thanks @malthe - we are in the process of rewriting all the scripts to python (including those small snippets) as part of #12282 so let's have minimal changes here.
I tested thaahttps://github.com/potiuk/airflow/runs/4865079794?check_suite_focus=true and piping works as expected:
```
if [[ "push" == 'pull_request_target' ]]; then
event_filter="event=pull_request&"
else
branch="${GITHUB_REF#refs/heads/}"
event_filter=""
fi
for cancel_url in $(
gh api "/repos/$GITHUB_REPOSITORY/actions/runs?${event_filter}branch=${branch}" | \
jq -r '
.workflow_runs[] |
select(.head_sha == $ENV.GITHUB_REGISTRY_PUSH_IMAGE_TAG and .status != "completed") |
.cancel_url
' \
); do
# One of these URls will be _this_ workflow, so lets exclude that!
[[ $cancel_url == */$thisRun/* ]] && continue
echo "Cancelling $cancel_url"
gh api -X POST --silent "$cancel_url"
done
shell: /usr/bin/bash -e {0}
env:
MOUNT_SELECTED_LOCAL_SOURCES: false
FORCE_ANSWER_TO_QUESTIONS: yes
CHECK_IMAGE_FOR_REBUILD: true
SKIP_CHECK_REMOTE_IMAGE: true
DB_RESET: true
VERBOSE: true
GITHUB_REPOSITORY: potiuk/airflow
GITHUB_USERNAME: potiuk
CONSTRAINTS_GITHUB_REPOSITORY: apache/airflow
GITHUB_TOKEN: ***
GITHUB_REGISTRY_PULL_IMAGE_TAG: latest
GITHUB_REGISTRY_WAIT_FOR_IMAGE: false
INSTALL_PROVIDERS_FROM_SOURCES: true
AIRFLOW_LOGIN_TO_GITHUB_REGISTRY: true
GITHUB_REGISTRY_PUSH_IMAGE_TAG: 2050e628f8e5fd18b8b13b5c52dcbf3daad003bc
branch:
thisRun: 1717050085
Cancelling https://api.github.com/repos/potiuk/airflow/actions/runs/1717050086/cancel
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org