You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/08/19 16:49:49 UTC
[GitHub] [beam] Abacn commented on a diff in pull request #22703: [GitHub Actions] Self-hosted runners migration
Abacn commented on code in PR #22703:
URL: https://github.com/apache/beam/pull/22703#discussion_r950281339
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,73 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## Beam GitHub Actions
+Currently, we have both GitHub-hosted and self-hosted (GCP cloud) runners for running the GitHub Actions workflows. The majority of our workflows that run in Ubuntu and Windows run in self-hosted runners, except for those that runs on MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our GCP self-hosted runners.
+
+### Getting Started with self-hosted runners
+* The steps for creating your own self-hosted runners for testing your workflows in: `.github/gh-actions-self-hosted-runners/README.md`
+* Depending on your workflow's needs, it must specify the following `runs-on` tags to run in the specified operating system:
+ * Ubuntu 20.04 self-hosted runner: `[self-hosted, ubuntu-20.04]`
+ * Windows 2019 self-hosted runner: `[self-hosted,windows-server-2019]`
Review Comment:
```suggestion
* Windows Server 2019 self-hosted runner: `[self-hosted,windows-server-2019]`
```
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,73 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## Beam GitHub Actions
+Currently, we have both GitHub-hosted and self-hosted (GCP cloud) runners for running the GitHub Actions workflows. The majority of our workflows that run in Ubuntu and Windows run in self-hosted runners, except for those that runs on MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our GCP self-hosted runners.
+
+### Getting Started with self-hosted runners
+* The steps for creating your own self-hosted runners for testing your workflows in: `.github/gh-actions-self-hosted-runners/README.md`
Review Comment:
```suggestion
* Refer to [this README](.github/gh-actions-self-hosted-runners/README.md) for the steps for creating your own self-hosted runners for testing your workflows.
```
##########
.github/workflows/build_playground_backend.yml:
##########
@@ -17,16 +17,17 @@ name: Build And Deploy Playground Backend Application
on:
push:
- tags: 'v*'
+ tags: ['v*']
branches: ['master', 'release-*']
- pull_request:
+ pull_request_target:
paths: ['playground/backend/**']
branches: ['playground-staging']
workflow_dispatch:
+permissions: read-all
jobs:
build_playground_backend_docker_image:
name: Build Playground Backend App
- runs-on: ubuntu-latest
+ runs-on: [self-hosted, ubuntu-20.04]
Review Comment:
A generic question: is there a fork repository showing this migration is working?
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,73 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## Beam GitHub Actions
+Currently, we have both GitHub-hosted and self-hosted (GCP cloud) runners for running the GitHub Actions workflows. The majority of our workflows that run in Ubuntu and Windows run in self-hosted runners, except for those that runs on MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our GCP self-hosted runners.
Review Comment:
Unfortunately the usage of runner in term "GitHub Runner" conflicts with Beam Runner. May need some clarification at the beginning of the document.
##########
.github/gh-actions-self-hosted-runners/README.md:
##########
@@ -0,0 +1,106 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# GitHub Actions - Self-hosted Runners
+The current GitHub Actions workflows are being tested on multiple operating systems, such as Ubuntu, Windows and MacOS. The way to migrate these runners from GitHub to GCP is by implementing self-hosted runners, so we implemented them in both Ubuntu and Windows environments, going with Google Kubernetes Engine and Google Cloud Compute VMs instances respectively.
+
+On the other hand, we will rely on GitHub-hosted runners for MacOS builds until a straightforward implementation approach comes out.
+
+## Ubuntu
+Ubuntu Self-hosted runners are stored in Artifact Registry and implemented using Google Kubernetes Engine with the following specifications:
+
+#### Cluster
+* Cluster: [gh-actions-linux-runners](https://console.cloud.google.com/kubernetes/clusters/details/us-central1-a/gh-actions-linux-runners/details?project=apache-beam-testing)
+* Image: [linux-github-actions-runner](https://console.cloud.google.com/artifacts/docker/apache-beam-testing/us-central1/beam-github-actions/linux-github-actions-runner?project=apache-beam-testing)
+
+#### Pool
+* Number of nodes: 5
+* Cluster Autoscaler: ON
+ * Minimum number of nodes: 5
+ * Maximum number of nodes: 10
+
+#### Node
+* Machine Type: e2-custom-6-18432
+* Disk Size: 100 GB
+* CPU: 6 vCPUs
+* Memory : 18 GB
+
+#### Pod
+* Image: $LOCAL_IMAGE_NAME LOCATION-docker.pkg.dev/PROJECT-ID/REPOSITORY/IMAGE:latest
+* CPU: 2
+* Memory: 1028 Mi
+* Volumes
+ * docker-sock
+ * gcloud-key
+* Secret env variables
+ * Kubernetes Secrets
+ * github-actions-secrets
+ * gcloud-key
+
+#### AutoScaling
+* Horizontal Pod Autoscaling
+ * 5-10 nodes (From Pool Cluster Autoscaler)
+ * HorizontalPodAutoscaler
+ * Min replicas: 10
+ * Max replicas: 20
+ * CPU utilization: 70%
+* Vertical Pod Autoscaling
+ * updateMode: "Auto"
+
+
+## Windows
+Windows Virtual machines have the following specifications
+
+#### VM specifications
+* Instance Template: _TODO: Add GCP link when final resources are in place._
+* Machine Type: n2-standard-2
+* Disk Size: 70 GB
+* Disk Image: [disk-image-windows-runner](https://console.cloud.google.com/compute/imagesDetail/projects/apache-beam-testing/global/images/disk-image-windows-runner?project=apache-beam-testing)
+* CPU: 2 vCPUs
+* Memory : 8 GB
+
+#### Instance group settings
+* Region: us-west1 (multizone)
+* Scale-out metric: 70% of CPU Usage.
+* Cooldown period: 300s
+
+#### Notes:
+At first glance we considered implementing Windows runners using K8s, however this was not optimal because of the following reasons:
+
+* VS Build tools are required for certain workflows, unfortunately official images that support this dependency are huge in size, reaching 20GB easily which is not an ideal case for k8S management.
+* Windows Subsystem For Linux(WSL) is a feature that allows to execute bash scripts inside Windows which removes tech debt by avoiding writing steps in powershell, but this feature is disabled with payload removed in Windows containers.
Review Comment:
Does this still invoke win32 applications? For example, for python unit test workflow, the script (https://github.com/apache/beam/blob/7a469fd20ef198a38e1df6af081062904dd1cbbb/sdks/python/scripts/run_pytest.sh#L39) invokes pytest should still be win32 version.
##########
.github/gh-actions-self-hosted-runners/self-hosted-windows/README.md:
##########
@@ -0,0 +1,54 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# GitHub Actions - Self-hosted Windows Runners
+
+This folder contains the required resources to deploy the GitHub Actions self-hosted runners for the workflows running in Windows OS.
+
+#### How to build a new instance template for the instance group?
+
+* Create a new VM Instance using Windows 2019 Datacenter OS with at least 70GB of disk, 2VCPUs and 8 GB of RAM.
Review Comment:
```suggestion
* Create a new VM Instance using Windows 2019 Datacenter OS with at least 70GB of disk, 2vCPUs and 8 GB of RAM.
```
##########
.github/ACTIONS.md:
##########
@@ -17,10 +17,73 @@
under the License.
-->
-> **PLEASE update this file if you add new github action or change name/trigger phrase of a github action.**
+> **PLEASE update this file if you add new GitHub Action or change name/trigger phrase of a GitHub Action.**
-## Beam Github Actions
+## Beam GitHub Actions
+Currently, we have both GitHub-hosted and self-hosted (GCP cloud) runners for running the GitHub Actions workflows. The majority of our workflows that run in Ubuntu and Windows run in self-hosted runners, except for those that runs on MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our GCP self-hosted runners.
Review Comment:
```suggestion
Currently, we have both GitHub-hosted and self-hosted (GCP cloud) runners (virtual machines) for running the GitHub Actions workflows. The majority of our workflows that run in Ubuntu and Windows run in self-hosted runners, except for those that runs on MacOS and the `Monitor Self-Hosted Runners Status` workflow that monitors our GCP self-hosted runners.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org