You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2020/11/15 09:37:54 UTC
[airflow] 11/19: Proposal: remove -serviceaccount suffix from KSA
names in helm chart (#10892)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c4b9dbe53aa585fc8184913412414655a3353fa2
Author: Jacob Ferriero <jf...@google.com>
AuthorDate: Tue Sep 15 03:39:39 2020 -0700
Proposal: remove -serviceaccount suffix from KSA names in helm chart (#10892)
* [WIP] remove -serviceaccount suffix in helm chart
It's quite annoying to have `-serviceaccount` in each service account name as this is a useless 15 characters that provides no additional information.
"why is this so frustrating to you Jake?"
GCP service accounts have 30 char name limit https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating
For manageability / clarity I'd like to keep KSA and GSA names exactly the same when using workload identity which maps KSA<>GSA 1:1 https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity.
(cherry picked from commit 23768f694697be1b29ec00b8deeed5777d2538e8)
---
chart/README.md | 10 ++++++++--
chart/templates/cleanup/cleanup-cronjob.yaml | 2 +-
chart/templates/cleanup/cleanup-serviceaccount.yaml | 2 +-
chart/templates/rbac/pod-cleanup-rolebinding.yaml | 2 +-
chart/templates/rbac/pod-launcher-rolebinding.yaml | 4 ++--
chart/templates/scheduler/scheduler-deployment.yaml | 4 ++--
chart/templates/scheduler/scheduler-serviceaccount.yaml | 2 +-
chart/templates/workers/worker-deployment.yaml | 2 +-
chart/templates/workers/worker-serviceaccount.yaml | 2 +-
9 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/chart/README.md b/chart/README.md
index 1ae5d6c..d56f114 100644
--- a/chart/README.md
+++ b/chart/README.md
@@ -35,6 +35,7 @@ cluster using the [Helm](https://helm.sh) package manager.
## Installing the Chart
To install this repository from source (using helm 3)
+
```bash
kubectl create namespace airflow
helm repo add stable https://charts.helm.sh/stable/
@@ -48,6 +49,7 @@ section lists the parameters that can be configured during installation.
> **Tip**: List all releases using `helm list`
## Upgrading the Chart
+
To upgrade the chart with the release name `airflow`:
```bash
@@ -90,6 +92,7 @@ helm upgrade airflow . \
```
## Mounting DAGS using Git-Sync side car without Persistence
+
This option will use an always running Git-Sync side car on every scheduler,webserver and worker pods. The Git-Sync side car containers will sync DAGs from a git repository every configured number of seconds. If you are using the KubernetesExecutor, Git-sync will run as an initContainer on your worker pods.
```bash
@@ -102,6 +105,7 @@ helm upgrade airflow . \
```
## Mounting DAGS from an externally populated PVC
+
In this approach, Airflow will read the DAGs from a PVC which has `ReadOnlyMany` or `ReadWriteMany` accessMode. You will have to ensure that the PVC is populated/updated with the required DAGs(this won't be handled by the chart). You can pass in the name of the volume claim to the chart
```bash
@@ -213,12 +217,14 @@ The following tables lists the configurable parameters of the Airflow chart and
| `webserver.resources.limits.memory` | Memory Limit of webserver | `~` |
| `webserver.resources.requests.cpu` | CPU Request of webserver | `~` |
| `webserver.resources.requests.memory` | Memory Request of webserver | `~` |
+| `webserver.service.annotations` | Annotations to be added to the webserver service | `{}` |
| `webserver.defaultUser` | Optional default airflow user information | `{}` |
-| `dags.persistence.*` | Dag persistence configuration | Please refer to `values.yaml` |
-| `dags.gitSync.*` | Git sync configuration | Please refer to `values.yaml` |
+| `dags.persistence.*` | Dag persistence configuration | Please refer to `values.yaml` |
+| `dags.gitSync.*` | Git sync configuration | Please refer to `values.yaml` |
| `multiNamespaceMode` | Whether the KubernetesExecutor can launch pods in multiple namespaces | `False` |
| `serviceAccountAnnottions.*` | Map of annotations for worker, webserver, scheduler kubernetes service accounts | {} |
+
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
```bash
diff --git a/chart/templates/cleanup/cleanup-cronjob.yaml b/chart/templates/cleanup/cleanup-cronjob.yaml
index 3ad3d4f..fd253d2 100644
--- a/chart/templates/cleanup/cleanup-cronjob.yaml
+++ b/chart/templates/cleanup/cleanup-cronjob.yaml
@@ -51,7 +51,7 @@ spec:
{{ toYaml .Values.affinity | indent 12 }}
tolerations:
{{ toYaml .Values.tolerations | indent 12 }}
- serviceAccountName: {{ .Release.Name }}-cleanup-serviceaccount
+ serviceAccountName: {{ .Release.Name }}-cleanup
{{- if or .Values.registry.secretName .Values.registry.connection }}
imagePullSecrets:
- name: {{ template "registry_secret" . }}
diff --git a/chart/templates/cleanup/cleanup-serviceaccount.yaml b/chart/templates/cleanup/cleanup-serviceaccount.yaml
index 769cbdc..6ef3aec 100644
--- a/chart/templates/cleanup/cleanup-serviceaccount.yaml
+++ b/chart/templates/cleanup/cleanup-serviceaccount.yaml
@@ -22,7 +22,7 @@
kind: ServiceAccount
apiVersion: v1
metadata:
- name: {{ .Release.Name }}-cleanup-serviceaccount
+ name: {{ .Release.Name }}-cleanup
labels:
tier: airflow
release: {{ .Release.Name }}
diff --git a/chart/templates/rbac/pod-cleanup-rolebinding.yaml b/chart/templates/rbac/pod-cleanup-rolebinding.yaml
index 4c2dd25..0d09b87 100644
--- a/chart/templates/rbac/pod-cleanup-rolebinding.yaml
+++ b/chart/templates/rbac/pod-cleanup-rolebinding.yaml
@@ -37,6 +37,6 @@ roleRef:
name: {{ .Release.Name }}-cleanup-role
subjects:
- kind: ServiceAccount
- name: {{ .Release.Name }}-cleanup-serviceaccount
+ name: {{ .Release.Name }}-cleanup
namespace: {{ .Release.Namespace }}
{{- end }}
diff --git a/chart/templates/rbac/pod-launcher-rolebinding.yaml b/chart/templates/rbac/pod-launcher-rolebinding.yaml
index f258de7..849f32d 100644
--- a/chart/templates/rbac/pod-launcher-rolebinding.yaml
+++ b/chart/templates/rbac/pod-launcher-rolebinding.yaml
@@ -51,12 +51,12 @@ roleRef:
subjects:
{{- if $grantScheduler }}
- kind: ServiceAccount
- name: {{ .Release.Name }}-scheduler-serviceaccount
+ name: {{ .Release.Name }}-scheduler
namespace: {{ .Release.Namespace }}
{{- end }}
{{- if $grantWorker }}
- kind: ServiceAccount
- name: {{ .Release.Name }}-worker-serviceaccount
+ name: {{ .Release.Name }}-worker
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}
diff --git a/chart/templates/scheduler/scheduler-deployment.yaml b/chart/templates/scheduler/scheduler-deployment.yaml
index 771dd1a..619a16c1 100644
--- a/chart/templates/scheduler/scheduler-deployment.yaml
+++ b/chart/templates/scheduler/scheduler-deployment.yaml
@@ -19,7 +19,7 @@
## Airflow Scheduler Deployment/StatefulSet
#################################
-# Are we using a local/sequenial executor?
+# Are we using a local/sequential executor?
{{- $local := or (eq .Values.executor "LocalExecutor") (eq .Values.executor "SequentialExecutor") }}
# Is persistence enabled on the _workers_?
# This is important because in $local mode, the scheduler assumes the role of the worker
@@ -81,7 +81,7 @@ spec:
{{ toYaml .Values.tolerations | indent 8 }}
restartPolicy: Always
terminationGracePeriodSeconds: 10
- serviceAccountName: {{ .Release.Name }}-scheduler-serviceaccount
+ serviceAccountName: {{ .Release.Name }}-scheduler
securityContext:
runAsUser: {{ .Values.uid }}
fsGroup: {{ .Values.gid }}
diff --git a/chart/templates/scheduler/scheduler-serviceaccount.yaml b/chart/templates/scheduler/scheduler-serviceaccount.yaml
index f12991d..2d61c76 100644
--- a/chart/templates/scheduler/scheduler-serviceaccount.yaml
+++ b/chart/templates/scheduler/scheduler-serviceaccount.yaml
@@ -22,7 +22,7 @@
kind: ServiceAccount
apiVersion: v1
metadata:
- name: {{ .Release.Name }}-scheduler-serviceaccount
+ name: {{ .Release.Name }}-scheduler
labels:
tier: airflow
release: {{ .Release.Name }}
diff --git a/chart/templates/workers/worker-deployment.yaml b/chart/templates/workers/worker-deployment.yaml
index 53998da..77d5fe2 100644
--- a/chart/templates/workers/worker-deployment.yaml
+++ b/chart/templates/workers/worker-deployment.yaml
@@ -72,7 +72,7 @@ spec:
{{ toYaml .Values.tolerations | indent 8 }}
terminationGracePeriodSeconds: {{ .Values.workers.terminationGracePeriodSeconds }}
restartPolicy: Always
- serviceAccountName: {{ .Release.Name }}-worker-serviceaccount
+ serviceAccountName: {{ .Release.Name }}-worker
securityContext:
runAsUser: {{ .Values.uid }}
fsGroup: {{ .Values.gid }}
diff --git a/chart/templates/workers/worker-serviceaccount.yaml b/chart/templates/workers/worker-serviceaccount.yaml
index f87751b..d7ed927 100644
--- a/chart/templates/workers/worker-serviceaccount.yaml
+++ b/chart/templates/workers/worker-serviceaccount.yaml
@@ -22,7 +22,7 @@
kind: ServiceAccount
apiVersion: v1
metadata:
- name: {{ .Release.Name }}-worker-serviceaccount
+ name: {{ .Release.Name }}-worker
labels:
tier: airflow
release: {{ .Release.Name }}