You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/09 16:00:17 UTC
[GitHub] [spark] mik-laj opened a new pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
mik-laj opened a new pull request #31104:
URL: https://github.com/apache/spark/pull/31104
Similar to: https://github.com/apache/spark/pull/31098 https://github.com/apache/calcite/pull/2318 (solution suggestted by @vlsi - https://github.com/apache/pulsar/issues/9154#issuecomment-756984731)
### What changes were proposed in this pull request?
This PR aims to reduce the GitHub Action usage by cancelling the previous build.
### Why are the changes needed?
In most case, the last commit is meaningful.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554448573
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
The n1hility action I based my action initially on should be deprecated as it has no good support for that. I actually created a ticket to either depreacate it it merge my changes: https://github.com/n1hility/cancel-previous-runs/issues/7 (I am happy my action to disapear and merge it back) but I have not heard from the author despite nagging several times
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757327093
@dongjoon-hyun Can I ask for review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554501220
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
I think you can merge these changes as a temporary workaround and see how the Apache policy updates change. I think you can merge these changes as a temporary workaround and try to contact INFRA about clarifying these rules. Especially since it does not cause the security reasons that led to the introduction of the new INFRA policy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554448680
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
I pushed a new change.s I passed token and used workflow_run event.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757406111
Thanks @mik-laj for addressing my comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757419037
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758318463
I enabled this only for the jobs triggered by PRs but there seems a bug in the plugin. If the PR triggers the job cancelation, it cancels all the workflows in the master branch too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757329603
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502921
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
@potiuk recommends using commit hashes to your actions. See: https://github.com/apache/airflow/pull/13327/files
This way it is even in case of an attack on his Github account, the Apache infrastructure will be safe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554497201
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Seems we're working around the new Apache policy vs GitHub Actions to use this plugin. Should we allow this via INFRA ticket? I have seen the original repo was asked here [INFRA-21287](https://issues.apache.org/jira/browse/INFRA-21287).
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
In that case, are we able to use it by explicit tag or hash via `potiuk/cancel-workflow-runs@hash`, instead of manually fetching the repository?
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Thanks for clarification, @mik-laj.
I think maybe it's a better approach to just allow this plugin by the standard approach via filing an INFRA ticket and allowing it if this fork will be maintained by syncing with the upstream because the newer changes will likely be reviewed properly as it seems like the fork owner @potiuk is a committer and PMC in Apache Airflow.
If we plan to make a PR to allow the diff in the original repo (https://github.com/n1hility/cancel-previous-runs), the temporary workaround might be fine but we will have to make sure either allowing `n1hility/cancel-previous-runs` or `potiuk/cancel-workflow-runs` in the end.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Awesome!
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
If this is supposed to be maintained separately, we can even switch this to the proper release or using `master` branch since @potiuk seems a ASF committer / PMC in Airflow. However, using hash seems okay too for now.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
If this is supposed to be maintained separately, we can even switch this to the proper release or using `master` branch since @potiuk seems a ASF committer / PMC in Airflow - all newer changes will likely be reviewed properly. However, using hash seems okay too for now.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
👌
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757408218
**[Test build #133883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133883/testReport)** for PR 31104 at commit [`a30c6c3`](https://github.com/apache/spark/commit/a30c6c354f5770b04c3a845680546bb2f18853a2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757414638
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38470/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757419039
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758490308
The reason why it is so 'aggressive' is that when you have queues, it's very likely that your "cancel runs' are also queued, so we want to make sure that we cancel everything that we know is a duplicate, whenever any of the 'cancel' runs gets executed.
The plugin can be easily extended to exclude certain branches from this cancelling if you think this is something you need, but I'd like to ask first - what do you loose if those master builds are cancelled if there is another master build already scheduled to run ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554500912
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache INFRA has no possibility of limiting the use of fluent references(v2, master, etc) and the rules for using actions are just being established. The new policy was only introduced a week or two ago due to one security incident, and now the INFRA team is watching how users use custom actions and what actions they use. We may try to add these actions to the whitelist, but there is little chance of that yet. For now, INFRA is a bit heavy with discussions about other Github Action limitations. See: https://lists.apache.org/thread.html/r5303eec41cc1dfc51c15dbe44770e37369330f9644ef09813f649120%40%3Cbuilds.apache.org%3E This change is intended to help us fix/mitigate this issue.
In the case of Apache Airflow, we decided to fork this action to apache/ (See: https://github.com/apache/airflow-cancel-workflow-runs), but now we consider migrations to use submodules (See: https://github.com/apache/airflow/issues/13513).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757332023
I might be slightly late, however, https://github.com/potiuk/cancel-workflow-runs seems to be a more up to date action, and the documentation is way better there.
A bonus point is that @potiuk is PMC for Apache Airflow ( https://people.apache.org/phonebook.html?uid=potiuk )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758637717
Actually the feature is already there - you can skip certain event types from being cancelled (in this case skipping the 'push' event will skip cancelling any of the runs that were originated by the "push" event (i.e. all master merges). I will make a PR that should enable it now and feel free to merge it if this is what you really want.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554445165
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This token is not needed. ;-) Each action gets a token by default in a input named "token". The action/checkout works the same way. See: https://github.com/actions/checkout/blob/c952173edf28a2bd22e1a4926590c1ac39630461/src/input-helper.ts#L109
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554444888
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Token is needed here: https://github.com/apache/spark/pull/31104/checks?check_run_id=1673964946#step:4:15
See how the token was declared: https://github.com/apache/spark/pull/31098/files#diff-48c0ee97c53013d18d6bbae44648f7fab9af2e0bf5b0dc1ca761e18ec5c478f2R94
Could you please add it to see if the action would be able to cancel your previous runs for this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758487194
> I enabled this only for the jobs triggered by PRs but there seems a bug in the plugin. If the PR triggers the job cancelation, it cancels all the workflows in the master branch too.
Not all - but 'all but the latest one'. This is by design.
This is not a bug. The plugin works 'aggressively' in `cancelAllDuplicates` mode - whenever 'cancelAllDuplicates' is run, it will cancel all 'duplicated' runs (all runs that match the same tuple: ('branch', 'oirigin repo', 'event') - regardles which branch it is.
So for every PR and every branch (including `master`) it will only leave the 'latest build` running. All other will be cancelled. This is by design. The most recent build from master should not be cancelled.
The assumption is that if you merged several merges to master, you really want to run tests on the latest version of this - all the previous merges are not 'valid' any more - they are superseded by the latest build.
In case of Airflow it saves hours of needless builds.. Is this a problem for Spark ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757399074
@mik-laj, mind mentioning the diff from the fork (https://github.com/potiuk/cancel-workflow-runs) and the original (https://github.com/n1hility/cancel-previous-runs) in the PR description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757407286
Let me leave it to @dongjoon-hyun to take another look. I am he are pretty active in the maintenance of GitHub Actions, and I would like to make sure having his thought too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757424864
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502672
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
If this is supposed to be maintained separately, we can even switch this to the proper release or using `master` branch since @potiuk seems a ASF committer / PMC in Airflow - all newer changes will likely be reviewed properly. However, using hash seems okay too for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757332023
I might be slightly late, however, https://github.com/potiuk/cancel-workflow-runs seems to be a more up to date action, and the documentation is way better there.
A bonus point is that @potiuk is PMC for Apache Airflow ( https://people.apache.org/phonebook.html?uid=potiuk )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554448357
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Yes. It must be a workflow_run action. Ant the 'cancel-workflow-run' action of mine has all the support for that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757407626
Ohh. I reverted one change by mistake, but already reverted. Now we use the action from the potiuk / repository and the file has the correct name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757329603
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502964
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
👌
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757415330
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38472/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554448573
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
The n1hility action I based my action initially on should be deprecated as it has no good support for that. I actually created a ticket to either deprecate it or merge my changes: https://github.com/n1hility/cancel-previous-runs/issues/7 (I am happy my action to disapear and merge it back) but I have not heard from the author despite nagging several times
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554501817
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
It looks like this action has been whitelisted. I will update this PR.
https://issues.apache.org/jira/browse/INFRA-21287
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757329603
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757407165
**[Test build #133881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133881/testReport)** for PR 31104 at commit [`1792fc9`](https://github.com/apache/spark/commit/1792fc9fb72b786a8abcdb6c08652955137df4c1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758683871
> I see, the new approach sounds much better. Thanks for working on this.
You are welcome! Happy to help!
Also the action has quite more sophisticated ways of cancelling the builds. I think at Apache Airflow we are saving at least 30% (in some cases even more) of the build time by pretty aggressive cancelling.
For example when any of our jobs fails at the "static check" failure we also cancel the workflow aggressively as well.
This is important because we have 5-10 minutes long 'static checks' and 40-50 minutes tests. but we want to start tests as soon as possible rather than waiting for static checks to complete. So we want to start them in parallel, but then if any of the static checks fails, we want to cancel all other tests because we know the change will have to be re-pushed anyway.
Unfortunately GA only allow such 'fail-fast' in matrix-jobs, so I had to add this option to the cancel action.
This way we save even more of the precious build time for the whole ASF community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554445165
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This token is not needed. ;-) Each action gets a token by default in a variable named "Token". The action/checkout works the same way. See: https://github.com/actions/checkout/blob/c952173edf28a2bd22e1a4926590c1ac39630461/src/input-helper.ts#L109
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758683871
> I see, the new approach sounds much better. Thanks for working on this.
You are welcome! Happy to help!
Also the action has quite more sophisticated ways of cancelling the builds. I think at Apache Airflow we are saving at least 30% (in some cases even more) of the build time by pretty aggressive cancelling.
For example when any of our jobs fails at the "static check" failure we also cancel the workflow aggressively as well.
This is important because we have 5-10 minutes long 'static checks' and 40-50 minutes tests. but we want to start tests as soon as possible rather than waiting for static checks to complete. So we want to start them in parallel, but then if any of the static checks fail, we want to cancel all other tests because we know the change will have to be re-pushed anyway.
Unfortunately GA only allow such 'fail-fast' in matrix-jobs, so I had to add this option to the cancel action.
This way we save even more of the precious build time for the whole ASF community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758653641
1) The cancelation worklfow must run in master branch in order to utlise GITHUB_TOKEN that has "write" capabilities (otherwise you have no rights to cancel workflo). This is how 'workflow_run' type of even works - it always runs using `master` code. See this stack-overflow question for example : https://stackoverflow.com/questions/63343937/how-to-use-the-github-actions-workflow-run-event#:~:text=The%20new%20workflow_run%20event%20enables,token%20as%20well%20as%20secrets.
This is due to security reasons. Otherwise any person could make a PR where they could change the workflow and do whatever they want to your repository. Workflow_run always runs wtith the workflow that is defined in your master, not in the PR. In order to test any change in the workflow_run you have to push it as master (I, for example, test such changes by pushing the PR branch as "master" to my own fork `github push -f potiuk <branch>:master` (where `potiuk` is the remote which is my own private fork).
2) Yes. The action is 'aggressive` because it is only useful if you are in the situation where your queue is "full" which means that any new action (including the cancel one) might take hours to even start executing. So if 'any' of the cancel workflows starts running - even from PRs that were created hours ago (just to remind - it's always using a master code) - this action looks at ALL builds and cancel ALL duplicate ones. That by default includes all "push" events (any merge to master is 'push', "schedule" events, "pull_request" - both coming from forks and pull requests that are done inside the repo.
The mode of cancelling 'cancelAllDuplicates` does exactly this. It does not even look at the original branch/PR that triggered it.
3) The change I am opening changes it's behaviour - it will not look for duplicates when there is a "push" or "schedule" event. Which means it will be cancelling ALL duplicates in ALL PRs opened that have more than one build in progress. This means that whenever cancel workflow has a chance to run, only the latest PR push will continue executing, others will be cancelled.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554444888
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Token is needed here: https://github.com/apache/spark/pull/31104/checks?check_run_id=1673964946#step:4:15
See how the token was declared: https://github.com/apache/spark/pull/31098/files#diff-48c0ee97c53013d18d6bbae44648f7fab9af2e0bf5b0dc1ca761e18ec5c478f2R94
Could you please add it to see if the action would be able to cancel your previous runs for this PR?
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Token is required for cancel-previous-runs: https://github.com/n1hility/cancel-previous-runs/blob/178b93a12fb2731212e48c2cc5b7e37b937fd339/action.yml#L5-L7
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
@mik-laj, please double-check the link I highlight: https://github.com/apache/spark/pull/31104/checks?check_run_id=1673964946#step:4:15
It literally says that `n1hility/cancel-previous-runs` fails and it requires authentication:
```
Run ./build/.actions/n1hility/cancel-previous-runs
with:
env:
MODULES_TO_TEST: core, unsafe, kvstore, avro, network-common, network-shuffle, repl, launcher, examples, sketch, graphx
EXCLUDED_TAGS:
INCLUDED_TAGS:
HADOOP_PROFILE: hadoop3.2
HIVE_PROFILE: hive2.3
CONDA_PREFIX: /usr/share/miniconda
GITHUB_PREV_SHA: 50d17ba02600601f8f63fcd55c2bf70865f9dc6e
GITHUB_INPUT_BRANCH:
[@octokit/rest] `const Octokit = require("@octokit/rest")` is deprecated. Use `const { Octokit } = require("@octokit/rest")` instead
Branch is patch-1, repo is spark, and owner is apache, and id is 474259497
(node:1987) UnhandledPromiseRejectionWarning: Error: Parameter token or opts.auth is required
at Function.getAuthString (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5899:19)
at Function.getOctokitOptions (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5864:29)
at new GitHub (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5848:22)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1492:25
at Generator.next (<anonymous>)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1448:71
at new Promise (<anonymous>)
at module.exports.131.__awaiter (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1444:12)
at cancelDuplicates (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1491:12)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1600:13
(node:1987) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:1987) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
```
Apparently, that does not look like a workable action.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757327093
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757403345
add to whitelist
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554445967
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This value is passed to all actions by default. See for another example: https://github.com/JamesIves/github-pages-deploy-action/pull/530
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554447297
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
I agree. I checked carefully and it looks like this action must be run as workflow_run.
> If you use forks, you should create a separate "Cancelling" workflow_run triggered workflow. The workflow_run should be responsible for all canceling actions. The examples below show the possible ways the action can be utilized.
https://github.com/potiuk/cancel-workflow-runs#usage
I am rewriting my PR now to use workflow_run. I will also ask @potiuk for a review when I'm finished.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757405898
@HyukjinKwon I renamed this file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757407165
**[Test build #133881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133881/testReport)** for PR 31104 at commit [`1792fc9`](https://github.com/apache/spark/commit/1792fc9fb72b786a8abcdb6c08652955137df4c1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758654523
The PR is here: https://github.com/apache/spark/pull/31153
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757329603
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554446356
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
@mik-laj, please double-check the link I highlight: https://github.com/apache/spark/pull/31104/checks?check_run_id=1673964946#step:4:15
It literally says that `n1hility/cancel-previous-runs` fails and it requires authentication:
```
Run ./build/.actions/n1hility/cancel-previous-runs
with:
env:
MODULES_TO_TEST: core, unsafe, kvstore, avro, network-common, network-shuffle, repl, launcher, examples, sketch, graphx
EXCLUDED_TAGS:
INCLUDED_TAGS:
HADOOP_PROFILE: hadoop3.2
HIVE_PROFILE: hive2.3
CONDA_PREFIX: /usr/share/miniconda
GITHUB_PREV_SHA: 50d17ba02600601f8f63fcd55c2bf70865f9dc6e
GITHUB_INPUT_BRANCH:
[@octokit/rest] `const Octokit = require("@octokit/rest")` is deprecated. Use `const { Octokit } = require("@octokit/rest")` instead
Branch is patch-1, repo is spark, and owner is apache, and id is 474259497
(node:1987) UnhandledPromiseRejectionWarning: Error: Parameter token or opts.auth is required
at Function.getAuthString (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5899:19)
at Function.getOctokitOptions (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5864:29)
at new GitHub (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:5848:22)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1492:25
at Generator.next (<anonymous>)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1448:71
at new Promise (<anonymous>)
at module.exports.131.__awaiter (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1444:12)
at cancelDuplicates (/home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1491:12)
at /home/runner/work/spark/spark/build/.actions/n1hility/cancel-previous-runs/dist/index.js:1600:13
(node:1987) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:1987) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
```
Apparently, that does not look like a workable action.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757407165
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757399074
@mik-laj, mind mentioning the diff from the fork (https://github.com/potiuk/cancel-workflow-runs) and the original (https://github.com/n1hility/cancel-previous-runs) in the PR description?
Also, it would be great if we can mention how it was tested at "How was this patch tested?". If the same way with the plugin was manually tested via another repo, that's fine too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554498780
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
In that case, are we able to use it by explicit tag or hash via `potiuk/cancel-workflow-runs@hash`, instead of manually fetching the repository?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758487194
This is not a bug. The plugin works 'aggressively' in `cancelAllDuplicates` mode - whenever 'cancelAllDuplicates' is run, it will cancel all 'duplicated' runs (all runs that match the same ('branch', 'oirigin repo', 'event').
So for every PR and every branch (including `master`) it will only leave the 'latest build` running. All other will be cancelled. This is by design. The most recent build from master should not be cancelled.
The assumption is that if you merged several merges to master, you really want to run tests on the latest version of this - all the previous merges are not 'valid' any more - they are superseded by the latest build.
In case of Airflow it saves hours of needless builds.. Is this a problem for Spark ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758318977
I disabled this workflow for now.
Let me know if this is fixed in the upstream.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554445165
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This token is not needed. ;-) Each action gets a token by default in a variable named "Token". The action/checkout works the same way. See: https://github.com/actions/checkout/blob/c952173edf28a2bd22e1a4926590c1ac39630461/src/input-helper.ts#L109
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This token is not needed. ;-) Each action gets a token by default in a input named "token". The action/checkout works the same way. See: https://github.com/actions/checkout/blob/c952173edf28a2bd22e1a4926590c1ac39630461/src/input-helper.ts#L109
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
This value is passed to all actions by default. See for another example: https://github.com/JamesIves/github-pages-deploy-action/pull/530
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
I agree. I checked carefully and it looks like this action must be run as workflow_run.
> If you use forks, you should create a separate "Cancelling" workflow_run triggered workflow. The workflow_run should be responsible for all canceling actions. The examples below show the possible ways the action can be utilized.
https://github.com/potiuk/cancel-workflow-runs#usage
I am rewriting my PR now to use workflow_run. I will also ask @potiuk for a review when I'm finished.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
I pushed a new change.s I passed token and used workflow_run event.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache's policy is to prohibit running code that is not reviewed, and this can happen if you use tags rather than exact hash references.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache INFRA has no possibility of limiting the use of fluent references(v2, master, etc) and the rules for using actions are just being established. The new policy was only introduced a week or two ago due to one security incident, and now the INFRA team is watching how users use custom actions and what actions they use. We may try to add these actions to the whitelist, but there is little chance of that yet. For now, INFRA is a bit heavy with discussions about other Github Action limitations. See: https://lists.apache.org/thread.html/r5303eec41cc1dfc51c15dbe44770e37369330f9644ef09813f649120%40%3Cbuilds.apache.org%3E This change is intended to help us fix/mitigate this issue.
In the case of Apache Airflow, we decided to fork this action to apache/ (See: https://github.com/apache/airflow-cancel-workflow-runs), but now we consider migrations to use submodules (See: https://github.com/apache/airflow/issues/13513).
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache INFRA has no possibility of limiting the use of fluent references(v2, master, etc) and the rules for using actions are just being established. The new policy was only introduced a week or two ago due to one security incident, and now the INFRA team is watching how users use custom actions and what actions they use. We may try to add these actions to the whitelist, but there is little chance of that yet. For now, INFRA is a bit heavy with discussions about other Github Action limitations. See: https://lists.apache.org/thread.html/r5303eec41cc1dfc51c15dbe44770e37369330f9644ef09813f649120%40%3Cbuilds.apache.org%3E This change is intended to help us fix/mitigate this issue.
In the case of Apache Airflow, we decided to fork this action to apache/ (See: https://github.com/apache/airflow-cancel-workflow-runs), but now we consider migrations to use submodules (See: https://github.com/apache/airflow/issues/13513).
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
I think you can merge these changes as a temporary workaround and see how the Apache policy updates change. I think you can merge these changes as a temporary workaround and try to contact INFRA about clarifying these rules. Especially since it does not cause the security reasons that led to the introduction of the new INFRA policy.
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
It looks like this action has been whitelisted.
https://issues.apache.org/jira/browse/INFRA-21287
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
It looks like this action has been whitelisted. I will update this PR.
https://issues.apache.org/jira/browse/INFRA-21287
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
@potiuk recommends using commit hashes to your actions. See: https://github.com/apache/airflow/pull/13327/files
This way it is even in case of an attack on his Github account, the Apache infrastructure will be safe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554497201
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Seems we're working around the new Apache policy vs GitHub Actions to use this plugin. Should we allow this via INFRA ticket? I have seen the original repo was asked here [INFRA-21287](https://issues.apache.org/jira/browse/INFRA-21287).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554501817
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
It looks like this action has been whitelisted.
https://issues.apache.org/jira/browse/INFRA-21287
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757399074
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758683871
> I see, the new approach sounds much better. Thanks for working on this.
You are welcome! Happy to help!
Also the action has quite more sophisticated ways of cancelling the builds. I think at Apache Airflow we are saving at least 30% (in some cases even more) of the build time by pretty aggressive cancelling.
For example when any of our jobs fails at the "static check" failure we also cancel the workflow aggressively as well.
This is important because we have 5-10 minutes long 'static checks' and 40-50 minutes tests. but we want to start tests as soon as possible rather than waiting for static checks to complete. So we want to start them in parallel, but then if any of the static checks fails, we want to cancel all running tests because we know the change will have to be re-pushed anyway.
Unfortunately GA only allow such 'fail-fast' in matrix-jobs, so I had to add this option to the cancel action.
This way we save even more of the precious build time for the whole ASF community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554500912
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache INFRA has no possibility of limiting the use of fluent references(v2, master, etc) and the rules for using actions are just being established. The new policy was only introduced a week or two ago due to one security incident, and now the INFRA team is watching how users use custom actions and what actions they use. We may try to add these actions to the whitelist, but there is little chance of that yet. For now, INFRA is a bit heavy with discussions about other Github Action limitations. See: https://lists.apache.org/thread.html/r5303eec41cc1dfc51c15dbe44770e37369330f9644ef09813f649120%40%3Cbuilds.apache.org%3E This change is intended to help us fix/mitigate this issue.
In the case of Apache Airflow, we decided to fork this action to apache/ (See: https://github.com/apache/airflow-cancel-workflow-runs), but now we consider migrations to use submodules (See: https://github.com/apache/airflow/issues/13513).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758487194
This is not a bug. The plugin works 'aggressively' in `cancelAllDuplicates` mode - whenever 'cancelAllDuplicates' is run, it will cancel all 'duplicated' runs (all runs that match the same tuple: ('branch', 'oirigin repo', 'event') - regardles which branch it is.
So for every PR and every branch (including `master`) it will only leave the 'latest build` running. All other will be cancelled. This is by design. The most recent build from master should not be cancelled.
The assumption is that if you merged several merges to master, you really want to run tests on the latest version of this - all the previous merges are not 'valid' any more - they are superseded by the latest build.
In case of Airflow it saves hours of needless builds.. Is this a problem for Spark ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502466
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Thanks for clarification, @mik-laj.
I think maybe it's a better approach to just allow this plugin by the standard approach via filing an INFRA ticket and allowing it if this fork will be maintained by syncing with the upstream because the newer changes will likely be reviewed properly as it seems like the fork owner @potiuk is a committer and PMC in Apache Airflow.
If we plan to make a PR to allow the diff in the original repo (https://github.com/n1hility/cancel-previous-runs), the temporary workaround might be fine but we will have to make sure either allowing `n1hility/cancel-previous-runs` or `potiuk/cancel-workflow-runs` in the end.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757403174
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502505
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Awesome!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758319978
OMG. Thank you for disabling it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #31104:
URL: https://github.com/apache/spark/pull/31104
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757399074
@mik-laj, mind mentioning the diff from the fork (https://github.com/potiuk/cancel-workflow-runs) and the original (https://github.com/n1hility/cancel-previous-runs) in the PR description?
Also, it would be great if we can mention how it was tested at "How was this patch tested?". If the same way with the plugin was manually tested via another repo, that's fine too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757422331
**[Test build #133881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133881/testReport)** for PR 31104 at commit [`1792fc9`](https://github.com/apache/spark/commit/1792fc9fb72b786a8abcdb6c08652955137df4c1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758664469
Sure, I used the same approach at https://github.com/apache/spark/blob/master/.github/workflows/test_report.yml to handle the security issue by leveraging `workflow_run` when `workflow_run` feature first came out. I am aware of that.
I see, the new approach sounds much better. Thanks for working on this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] vlsi commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
vlsi commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554445399
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Token is required for cancel-previous-runs: https://github.com/n1hility/cancel-previous-runs/blob/178b93a12fb2731212e48c2cc5b7e37b937fd339/action.yml#L5-L7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757422942
**[Test build #133883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133883/testReport)** for PR 31104 at commit [`a30c6c3`](https://github.com/apache/spark/commit/a30c6c354f5770b04c3a845680546bb2f18853a2).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757408218
**[Test build #133883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133883/testReport)** for PR 31104 at commit [`a30c6c3`](https://github.com/apache/spark/commit/a30c6c354f5770b04c3a845680546bb2f18853a2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-757424864
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mik-laj commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554498229
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,25 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ name: checkout potiuk/cancel-workflow-runs
+ with:
+ repository: potiuk/cancel-workflow-runs
+ path: ./build/.actions/potiuk/cancel-workflow-runs
+ ref: 953e057dc81d3458935a18d1184c386b0f6b5738 # @master
+ fetch-depth: 1
+ - uses: ./build/.actions/potiuk/cancel-workflow-runs
Review comment:
Apache's policy is to prohibit running code that is not reviewed, and this can happen if you use tags rather than exact hash references.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554448357
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
Yes. It must be a workflow_run action. Ant the 'cancel-workflow-run' action of mine has all the support for that.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
The n1hility action I based my action initially on should be deprecated as it has no good support for that. I actually created a ticket to either depreacate it it merge my changes: https://github.com/n1hility/cancel-previous-runs/issues/7 (I am happy my action to disapear and merge it back) but I have not heard from the author despite nagging several times
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -89,6 +89,14 @@ jobs:
# In order to fetch changed files
with:
fetch-depth: 0
+ - uses: actions/checkout@v2
+ name: checkout n1hility/cancel-previous-runs
+ with:
+ repository: n1hility/cancel-previous-runs
+ path: ./build/.actions/n1hility/cancel-previous-runs
+ ref: 953c92201f368370112ea2754545cb4468d89f12 # v2
+ fetch-depth: 1
+ - uses: ./build/.actions/n1hility/cancel-previous-runs
Review comment:
The n1hility action I based my action initially on should be deprecated as it has no good support for that. I actually created a ticket to either deprecate it or merge my changes: https://github.com/n1hility/cancel-previous-runs/issues/7 (I am happy my action to disapear and merge it back) but I have not heard from the author despite nagging several times
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31104:
URL: https://github.com/apache/spark/pull/31104#discussion_r554502672
##########
File path: .github/workflows/build_and_test_workflow_run.yml
##########
@@ -0,0 +1,18 @@
+name: Cancelling Duplicates
+on:
+ workflow_run:
+ workflows:
+ - 'Build and test'
+ types: ['requested']
+
+jobs:
+ cancel-duplicate-workflow-runs:
+ name: "Cancel duplicate workflow runs"
+ runs-on: ubuntu-latest
+ steps:
+ - uses: potiuk/cancel-workflow-runs@953e057dc81d3458935a18d1184c386b0f6b5738
Review comment:
If this is supposed to be maintained separately, we can even switch this to the proper release or using `master` branch since @potiuk seems a ASF committer / PMC in Airflow. However, using hash seems okay too for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31104: [SPARK-34053][INFRA] Cancel the previous build
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31104:
URL: https://github.com/apache/spark/pull/31104#issuecomment-758642598
@potiuk, I already tried such fix at https://github.com/apache/spark/pull/31121 but it did not work. The cancelation workflow triggered by a PR canceled workflow runs in master branch, which looks not making sense much. If a user open a PR, then it cancels the jobs in the master branch .. ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org