You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2023/07/26 13:54:53 UTC
[beam] 01/01: Add instructions on adding new Actions workflows
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/actionsInstructions
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 525e7f58a4fab77d7d7e94f3fb65e96a20022baf
Author: Danny McCormick <da...@google.com>
AuthorDate: Wed Jul 26 09:54:45 2023 -0400
Add instructions on adding new Actions workflows
---
.github/workflows/README.md | 83 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 82 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/README.md b/.github/workflows/README.md
index 8bb6a184ac1..636a06cfe82 100644
--- a/.github/workflows/README.md
+++ b/.github/workflows/README.md
@@ -14,6 +14,87 @@
specific language governing permissions and limitations
under the License.
-->
+
+# Guidelines for Workflows
+
+On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
+
+1) A set of specific triggers
+2) An explicit checkout step
+3) A set of GitHub token permissions
+4) Concurrency Groups
+5) Comment Triggering Support
+
+Each of these is described in more detail below.
+
+## Workflow triggers
+
+GitHub allows workflows to define a set of triggers that dictate when a job should be run. For more info, see [documentation here](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows).
+
+For the purposes of Beam, each CI workflow should define the following triggers:
+
+1) A `push` trigger
+2) A `pull_request_target` trigger
+3) An issue_comment trigger (for issue created). This is needed for comment triggering support (see section below).
+4) A scheduled trigger
+5) A workflow_dispatch trigger
+
+The `push`/`pull_request_target` triggers should only run when appropriate paths are modified. See https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Go.yml#L4 for an example (you can copy and paste this into your workflow, you just need to change the paths).
+
+## Checkout step
+
+Because we use the `pull_request_target` trigger instead of `pull_request`, we need an explicit checkout of the correct commit. This can be done as the first step of any jobs in your workflow. See https://github.com/apache/beam/blob/907c5110163b0efe52e9e12127fd013c7fc455d7/.github/workflows/beam_PreCommit_Go.yml#L46 for an example (you can copy and paste this into your workflow).
+
+## Token Permissions
+
+You should explicitly define the GitHub Actions token scopes needed by your job. For most jobs, this will be `actions: write` (needed for comment triggering support) and `read` permissions for all other options. See https://github.com/apache/beam/blob/907c5110163b0efe52e9e12127fd013c7fc455d7/.github/workflows/beam_PreCommit_Go.yml#L16 for an example (you can copy and paste this into your workflow).
+
+## Concurrency Groups
+
+Concurrency groups are a way of making sure that no more than one Actions run is running per job/group at any given time (previous ones can be cancelled). To reduce resource usage, you should define the following concurrency group:
+
+```
+concurrency:
+ group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.sha || github.head_ref || github.ref }}-${{ github.event.sender.login }}-${{ github.event.schedule }}'
+ cancel-in-progress: true
+```
+
+this defines the following groups:
+
+1) Each workflow will represent a different set of groups
+2) Within each workflow, each PR will represent a single group
+3) Within each non-PR run for a workflow, each commit will represent a set of groups
+4) Within each commit, each push event, schedule event, and manual run event will represent a set of groups
+
+## Comment Triggering Support
+
+In order to make it easier for non-committers to interact with workflows, workflows should implement comment triggering support. This requires 3 additional components beyond the ones mentioned above:
+
+1) Each job should be gated on an if condition that maps to that workflow's comment trigger (example: https://github.com/apache/beam/blob/907c5110163b0efe52e9e12127fd013c7fc455d7/.github/workflows/beam_PreCommit_Go.yml#L38)
+2) Each job should have the rerun action immediately after its checkout step. This should be gated on the comment trigger (example: https://github.com/apache/beam/blob/907c5110163b0efe52e9e12127fd013c7fc455d7/.github/workflows/beam_PreCommit_Go.yml#L49)
+3) Each job should have a descriptive name that includes the comment trigger (example: https://github.com/apache/beam/blob/ba8fc935222aeb070668fbafd588bc58e7a21289/.github/workflows/beam_PreCommit_CommunityMetrics.yml#L48)
+
+# Testing new workflows or workflow updates
+
+## Testing New Workflows
+
+New workflows are not added to the [UI on the Actions tab](https://github.com/apache/beam/actions) until they are merged into the repo's main branch (master for Beam).
+To test new workflows, we recommend the following pattern:
+
+1) Fork the Beam repo
+2) Add your proposed workflow to the main branch of your fork.
+3) Run the workflow in the [Actions tab](https://github.com/apache/beam/actions) of your fork using the `Run workflow` button
+
+## Testing Workflow Updates
+
+Once a workflow has been added to the repo, you can develop normally on a branch (using your fork or the main Beam repo if you are a committer).
+To do so:
+
+1) Make your change on a development branch.
+2) Navigate to your workflow in the [Actions tab](https://github.com/apache/beam/actions) of your repo.
+3) Click run workflow. Before clicking submit, update to run on your branch.
+
+# Workflows
Please note that jobs with matrix need to have matrix element in the comment. Example:
```Run Python PreCommit (3.8)```
| Workflow name | Matrix | Trigger Phrase | Cron Status |
@@ -32,4 +113,4 @@ Please note that jobs with matrix need to have matrix element in the comment. Ex
| [ Python PreCommit ](https://github.com/apache/beam/actions/workflows/job_PreCommit_Python.yml) | ['3.8','3.9','3.10','3.11'] | `Run Python PreCommit (matrix_element)` | [](https://github.com/apache/beam/actions/workflows/job_PreCommit_Python.yml) |
| [ PreCommit Python Integration](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Integration.yml) | ['3.8','3.11'] | `Run Python_Integration PreCommit (matrix_element)` | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Integration.yml) |
| [ RAT PreCommit ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_RAT.yml) | N/A | `Run RAT PreCommit` | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_RAT.yml) |
-| [ PreCommit SQL Java11 ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_SQL_Java11.yml) | N/A |`Run SQL_Java11 PreCommit`| [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_SQL_Java11.yml) |
\ No newline at end of file
+| [ PreCommit SQL Java11 ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_SQL_Java11.yml) | N/A |`Run SQL_Java11 PreCommit`| [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_SQL_Java11.yml) |