You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "volatilemolotov (via GitHub)" <gi...@apache.org> on 2023/08/04 16:15:25 UTC
[GitHub] [beam] volatilemolotov opened a new pull request, #27852: update all jobs with new name and phrase matrix, required for rerun
volatilemolotov opened a new pull request, #27852:
URL: https://github.com/apache/beam/pull/27852
Adds a matrix to handle job name and phrase to allow reruns.
Tested on fork, all jobs run and some failed due to dependencies that the fork cannot satisfy (eg. metrics cluster)
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285957106
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
I was suggesting early exiting the whole flow from setup-action
> We could implement gh run cancel but it would be really hard because the parrallel instances launched. In normal usage 3/4 (python example) instances would have to detect if the mismatch is because someone put "Run xxxx 3.8" or "Run xxxx".
I don't think we need to make this distinction. If the job in question is Python 3.8, we can still early exit on anything other than `Run Python_Precommit 3.8`, its ok if typos don't trigger the job correctly. We could even allow triggering all jobs together with `Run Python_Precommit` pretty easily
So the step in setup-action would be something like:
```
if: github.event.comment.body != '${{ inputs.comment_phrase }}' and github.event.comment.body != '${{ inputs.comment_phrase }} ${{ inputs.version }}' # version would now need to be passed in
run: |
echo 'comment does not match this jobs trigger phrase ${{ inputs.comment_phrase }} ${{ inputs.version }}'
gh run cancel ${{ inputs.github_current_run_id }}
gh run watch ${{ inputs.github_current_run_id }}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285943837
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Early exit `setup-action` or the whole flow?
`setup-action` is really short and not causing issues. Issue that the test runs outside the `setup-action`.
We could implement `gh run cancel` but it would be really hard because the parrallel instances launched. In normal usage 3/4 (python example) instances would have to detect if the mismatch is because someone put "Run xxxx 3.8" or "Run xxxx".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1287231833
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Run Job With Matrix 3.9' ||
+ github.event.comment.body == 'Run Job With Matrix 3.10' ||
+ github.event.comment.body == 'Run Job With Matrix 3.11')
Review Comment:
Missed it, sorry.
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27852:
URL: https://github.com/apache/beam/pull/27852#issuecomment-1666297877
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @damccorm for label build.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285880699
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Could we do that, but then early exit in setup-action if the phrase doesn't actually match since we'll have the information we need at that point? Early exiting is probably a good idea regardless
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] andreydevyatkin commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "andreydevyatkin (via GitHub)" <gi...@apache.org>.
andreydevyatkin commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1284652248
##########
.github/workflows/beam_PostCommit_Go_Dataflow_ARM.yml:
##########
@@ -55,14 +55,19 @@ jobs:
github.event_name == 'schedule' ||
github.event.comment.body == 'Run Go PostCommit Dataflow ARM'
runs-on: [self-hosted, ubuntu-20.04, main]
- name: beam_PostCommit_Go_Dataflow_ARM (Run Go PostCommit Dataflow ARM)
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: ["beam_PostCommit_Go_Dataflow_ARM"]
+ job_phrase: ["Run Go PostCommit Dataflow ARM"]
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
- comment_phrase: 'Run Go PostCommit Dataflow ARM'
+ comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} ${{ matrix.job_phrase }}
Review Comment:
The brackets are missing here. Should be:
```
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
```
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
I think it should be `Run Job With Matrix 3.9`. Applies to others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1287213208
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Run Job With Matrix 3.9' ||
+ github.event.comment.body == 'Run Job With Matrix 3.10' ||
+ github.event.comment.body == 'Run Job With Matrix 3.11')
Review Comment:
Could we update this as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1284673364
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Ok, got it. Do we gain anything (other than a bit of compute on a typo) from specifying `Run Job With Matrix 3.8` vs `Run Job With Matrix `? If not, I might vote for leaving the values out to avoid extra opportunity for mistakes when we add new versions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm merged pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #27852:
URL: https://github.com/apache/beam/pull/27852
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1286017066
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Can we just fail the job instead of `gh run cancel`? This will just be on comment triggers which we can document should be ignored, so the red signal won't really mean anything, and we could easily provide a clear error message (so something like the following)
```
if: github.event.comment.body != '${{ inputs.comment_phrase }}' and github.event.comment.body != '${{ inputs.comment_phrase }} ${{ inputs.version }}' # version would now need to be passed in
run: |
>&2 echo 'comment does not match this jobs trigger phrase ${{ inputs.comment_phrase }} ${{ inputs.version }}'
exit 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1286028789
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Since we pass the phrase to `setup-action` this should be possible if we are accepting to have red workflows (Not that big. these workflow runs are not attached to a PR). Let me figure it out and create a PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285856513
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
@damccorm I tested it and what happens if a person does `Run Job With Matrix` a number of instances of the job run with no repercussions besides the compute. If that is acceptable i can revert to `startsWith` filter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285943837
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Early exit `setup-action` or the whole flow.
`setup-action` is really short and not causing issues. Issue that the test runs outside the `setup-action`.
We could implement `gh run cancel` but it would be really hard because the parrallel instances launched. In normal usage 3/4 (python example) instances would have to detect if the mismatch is because someone put "Run xxxx 3.8" or "Run xxxx".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285988074
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Problem is that there are x number of instances which all get a different `inputs.version` due to that being passed on from `matrix`.
In that case if a dev issues `Run Python_Precommit 3.9` the `3.8` job will see that comment does not equal `phrase` nor `phrase 3.8` and will issue `gh run cancel` which will cancel the whole run, even the valid job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1285621589
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
I agree on that one but let me check what happens when we leave out the version, what happens with the jobs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1284667056
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Are we still able to use the matrix values here? E.g. could we do `${{ matrix.job_phrase }} ${{ matrix.python_version }}`? Same question applies to the non-versioned workflows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1284670947
##########
.github/workflows/README.md:
##########
@@ -19,13 +19,71 @@
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
-1) A set of specific triggers
-2) An explicit checkout step
-3) A set of GitHub token permissions
-4) Concurrency Groups
-5) Comment Triggering Support
+1) Name and phrase set via matrix for re-run to work (See below)
+2) A set of specific triggers
+3) An explicit checkout step
+4) A set of GitHub token permissions
+5) Concurrency Groups
+6) Comment Triggering Support
Each of these is described in more detail below.
+## Name and Phrase
+Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
+```
+jobs:
+ beam_job_name:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: [beam_job_name]
+ job_phrase: [Run Job Phrase]
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ github.event_name == 'workflow_dispatch' ||
+ github.event.comment.body == 'Run Job Phrase'
+ steps:
+ - uses: actions/checkout@v3
+ - name: Setup repository
+ uses: ./.github/actions/setup-action
+ with:
+ comment_phrase: ${{ matrix.job_phrase }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+
+```
+And in case when the workflow already utilizes matrix do the following:
+```
+jobs:
+ beam_job_with_matrix:
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
+ runs-on: [self-hosted, ubuntu-20.04, main]
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ job_name: ["beam_job_with_matrix"]
+ job_phrase: ["Run Job With Matrix"]
+ python_version: ['3.8','3.9','3.10','3.11']
+ if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event_name == 'schedule' ||
+ (github.event.comment.body == 'Run Job With Matrix 3.8' ||
+ github.event.comment.body == 'Job With Matrix 3.9' ||
Review Comment:
Fixed.
@damccorm Unfortunately no, in `if` you only have access to expression contexts: `github, inputs, vars, needs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] volatilemolotov commented on a diff in pull request #27852: update all jobs with new name and phrase matrix, required for rerun
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on code in PR #27852:
URL: https://github.com/apache/beam/pull/27852#discussion_r1284673002
##########
.github/workflows/beam_PostCommit_Go_Dataflow_ARM.yml:
##########
@@ -55,14 +55,19 @@ jobs:
github.event_name == 'schedule' ||
github.event.comment.body == 'Run Go PostCommit Dataflow ARM'
runs-on: [self-hosted, ubuntu-20.04, main]
- name: beam_PostCommit_Go_Dataflow_ARM (Run Go PostCommit Dataflow ARM)
+ name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
+ strategy:
+ matrix:
+ job_name: ["beam_PostCommit_Go_Dataflow_ARM"]
+ job_phrase: ["Run Go PostCommit Dataflow ARM"]
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
- comment_phrase: 'Run Go PostCommit Dataflow ARM'
+ comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ matrix.job_name }} ${{ matrix.job_phrase }}
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org