You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2023/05/24 20:31:26 UTC
[GitHub] [beam] damccorm commented on a diff in pull request #26845: Initial terraform code for ARC based runners
damccorm commented on code in PR #26845:
URL: https://github.com/apache/beam/pull/26845#discussion_r1204719924
##########
.github/gh-actions-self-hosted-runners/arc/README.md:
##########
@@ -0,0 +1,63 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Actions Runner Contoler
+
+# About
+Check out the docs at https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md
+
+# Installing
+1. Create a file called `overrides.tfvars` with the following contents:
+```
+project_id = "PROJECT_ID" # google PROJECT_ID that you want to deploy in
+region = "gcp_region" # GCP region for the network
+zone = "europe-west3-c" # GCP zone for the nodes
+min_main_node_count = "1" # Minimal and initial node count for main pool
+max_main_node_count = "5" # Maximal node count for main pool
+environment = "environment_name" # Name of the environment. Used as a prefix like dev- stag- anything-
+ingress_domain = "fqdn" # FQDN for webhook ingress
+organization = "org" # Github Organization to use runners in
+repository = "repo" # Repository to use runners in
+github_app_id = "app_id" # Github App app_id
+github_app_installation_id = "install_id" # Github App install id
+github_app_private_key_path = "absolute_path_to_key" # Absolute filesystem path to .pem keyfile generated in the Github App
+deploy_webhook = "false" # Terraform to deploy the scaling webhook
+max_main_replicas = "2" # Max number of runner PODs . Do not confuse with Nodes
+min_main_replicas = "1" # Min number of runner PODs . Do not confuse with Nodes
+webhook_scaling = "false" # Enable webhook scaling. When disabled runner busy percentage is used
+```
+
+2. Create a google bucket for storing the terraform state. Make sure you disable public access and allow your account to access it.
+
+3. Create a Github App in your account and install it in the repo you want to provide runners for.
+All is explained in : https://github.com/actions/actions-runner-controller/blob/master/docs/authenticating-to-the-github-api.md
+
+4. In your Google Cloud Project create the secrets for
Review Comment:
These should probably be steps 1 and 2 since they are used in the overrides file
##########
.github/gh-actions-self-hosted-runners/arc/config/arc_deployment.tpl:
##########
@@ -0,0 +1,39 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: RunnerDeployment
+metadata:
+ name: main-runners
+spec:
+ template:
+ spec:
+ image: summerwind/actions-runner:v2.304.0-ubuntu-20.04-30355f7
Review Comment:
Why did we decide on this image? Looks like this is maybe an alias for actions? Might be worth dropping in a comment
##########
.github/gh-actions-self-hosted-runners/arc/README.md:
##########
@@ -0,0 +1,63 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Actions Runner Contoler
+
+# About
+Check out the docs at https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md
+
+# Installing
+1. Create a file called `overrides.tfvars` with the following contents:
+```
+project_id = "PROJECT_ID" # google PROJECT_ID that you want to deploy in
+region = "gcp_region" # GCP region for the network
+zone = "europe-west3-c" # GCP zone for the nodes
+min_main_node_count = "1" # Minimal and initial node count for main pool
+max_main_node_count = "5" # Maximal node count for main pool
+environment = "environment_name" # Name of the environment. Used as a prefix like dev- stag- anything-
Review Comment:
Nit: indentation is off
##########
.github/gh-actions-self-hosted-runners/arc/README.md:
##########
@@ -0,0 +1,63 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Actions Runner Contoler
+
+# About
+Check out the docs at https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md
+
+# Installing
+1. Create a file called `overrides.tfvars` with the following contents:
+```
+project_id = "PROJECT_ID" # google PROJECT_ID that you want to deploy in
+region = "gcp_region" # GCP region for the network
+zone = "europe-west3-c" # GCP zone for the nodes
+min_main_node_count = "1" # Minimal and initial node count for main pool
+max_main_node_count = "5" # Maximal node count for main pool
+environment = "environment_name" # Name of the environment. Used as a prefix like dev- stag- anything-
+ingress_domain = "fqdn" # FQDN for webhook ingress
+organization = "org" # Github Organization to use runners in
+repository = "repo" # Repository to use runners in
+github_app_id = "app_id" # Github App app_id
+github_app_installation_id = "install_id" # Github App install id
+github_app_private_key_path = "absolute_path_to_key" # Absolute filesystem path to .pem keyfile generated in the Github App
Review Comment:
Should these actually be plain text variables, or are they references to the secrets we create in step 4?
##########
.github/gh-actions-self-hosted-runners/arc/locals.tf:
##########
@@ -0,0 +1,39 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+
+locals {
+ subnetwork_cidr_range = "10.128.0.0/20"
+ arc_values = {
+ "githubWebhookServer.enabled" = "true"
+ "authSecret.create" = "true"
+ "authSecret.github_app_id" = data.google_secret_manager_secret_version.github_app_id.secret_data
+ "authSecret.github_app_installation_id" = data.google_secret_manager_secret_version.github_app_install_id.secret_data
+ "authSecret.github_app_private_key" = data.google_secret_manager_secret_version.github_private_key.secret_data
+ "githubWebhookServer.ingress.enabled" = "true"
+ "githubWebhookServer.ingress.hosts[0].host" = var.ingress_domain
+ "githubWebhookServer.ingress.hosts[0].paths[0].path" = "/"
+ "githubWebhookServer.ingress.hosts[0].paths[0].pathType" = "ImplementationSpecific"
+ "githubWebhookServer.service.type" = "NodePort"
+ #"githubWebhookServer.ingress.tls[0].hosts[0]" = var.ingress_domain
Review Comment:
Do we need this? Can we get rid of the comment?
##########
.github/gh-actions-self-hosted-runners/arc/config/arc_autoscaler.tpl:
##########
@@ -0,0 +1,46 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+apiVersion: actions.summerwind.dev/v1alpha1
+kind: HorizontalRunnerAutoscaler
+metadata:
+ name: main-runners
+spec:
+ # Runners in the targeted RunnerDeployment won't be scaled down
+ # for 5 minutes instead of the default 10 minutes now
+ scaleDownDelaySecondsAfterScaleOut: 300
Review Comment:
Any reason for this?
##########
.github/gh-actions-self-hosted-runners/arc/README.md:
##########
@@ -0,0 +1,63 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Actions Runner Contoler
+
+# About
+Check out the docs at https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md
+
+# Installing
+1. Create a file called `overrides.tfvars` with the following contents:
+```
+project_id = "PROJECT_ID" # google PROJECT_ID that you want to deploy in
+region = "gcp_region" # GCP region for the network
+zone = "europe-west3-c" # GCP zone for the nodes
+min_main_node_count = "1" # Minimal and initial node count for main pool
+max_main_node_count = "5" # Maximal node count for main pool
+environment = "environment_name" # Name of the environment. Used as a prefix like dev- stag- anything-
+ingress_domain = "fqdn" # FQDN for webhook ingress
+organization = "org" # Github Organization to use runners in
+repository = "repo" # Repository to use runners in
+github_app_id = "app_id" # Github App app_id
+github_app_installation_id = "install_id" # Github App install id
+github_app_private_key_path = "absolute_path_to_key" # Absolute filesystem path to .pem keyfile generated in the Github App
+deploy_webhook = "false" # Terraform to deploy the scaling webhook
+max_main_replicas = "2" # Max number of runner PODs . Do not confuse with Nodes
+min_main_replicas = "1" # Min number of runner PODs . Do not confuse with Nodes
+webhook_scaling = "false" # Enable webhook scaling. When disabled runner busy percentage is used
+```
+
+2. Create a google bucket for storing the terraform state. Make sure you disable public access and allow your account to access it.
+
+3. Create a Github App in your account and install it in the repo you want to provide runners for.
+All is explained in : https://github.com/actions/actions-runner-controller/blob/master/docs/authenticating-to-the-github-api.md
+
+4. In your Google Cloud Project create the secrets for
+- Github App ID
+- Github App Installation ID
+- Github App PEM key
+All are created in the step before
+
+5. Init terraform with:
Review Comment:
```suggestion
5. From this directory, init terraform with:
```
Nit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org