You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2024/08/15 22:03:13 UTC
(spark-kubernetes-operator) branch main updated: [SPARK-48398] Add Helm Chart
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 5478679 [SPARK-48398] Add Helm Chart
5478679 is described below
commit 5478679b7b08ac37a033a92c442824ab769ad95f
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Aug 15 15:03:06 2024 -0700
[SPARK-48398] Add Helm Chart
### What changes were proposed in this pull request?
This PR proposes Spark Kubernetes Operator Helm chart, which describes deployment manifest for the operator.
### Why are the changes needed?
Helm chart helps users to install, upgrade and uninstall Spark Operator in Kubernetes cluster. It provides a cloud native way of managing Operator in place. This chart helps us to complete the end-to-end flow for the operator app.
### Does this PR introduce _any_ user-facing change?
No (not yet released)
### How was this patch tested?
* CI added for chart linting
* Chart test for operator deployment RBAC validation
* We are also working on SPARK-49214 to include the test automation in workflow. Till that is resolved, this patch can be validated manually by running the following steps:
#### Start minikube
Start miniKube and make it access locally-built image
```shell
minikube start
eval $(minikube docker-env)
```
#### Build Spark Operator Locally
```bash
# Build a local container image which can be used for minikube.etc.
docker build --build-arg APP_VERSION=0.1.0 -t spark-kubernetes-operator:0.1.0 -f build-tools/docker/Dockerfile .
# Generate CRD yaml and make it available for chart deployment
./gradlew spark-operator-api:relocateGeneratedCRD
```
#### Install the Spark Operator
```bash
helm install spark-kubernetes-operator --create-namespace -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
```
#### Verify the Installation
``` bash
helm test spark-kubernetes-operator
```
Here's a sample spark app yaml
```yml
apiVersion: org.apache.spark/v1alpha1
kind: SparkApplication
metadata:
name: spark-pi
spec:
mainClass: "org.apache.spark.examples.SparkPi"
jars: "local:///opt/spark/examples/jars/spark-examples_2.13-4.0.0-preview1.jar"
sparkConf:
spark.executor.instances: "5"
spark.kubernetes.container.image: "spark:4.0.0-preview1"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
applicationTolerations:
resourceRetainPolicy: OnFailure
runtimeVersions:
scalaVersion: "2.13"
sparkVersion: "4.0.0-preview1"
```
Create a file named `spark-pi.yaml` and run
```bash
kubectl create -f spark-pi.yaml
```
The operator should be able to operate given app from start to complete.
You can delete job with the following.
```
$ kubectl delete -f spark-pi.yaml
sparkapplication.org.apache.spark "spark-pi" deleted
```
#### Uninstallation
To remove the installed resources from your cluster, reset environment to the defaults and
shutdown the cluster:
```bash
helm uninstall spark-kubernetes-operator
eval $(minikube docker-env --unset)
minikube stop
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #44 from jiangzho/helm.
Lead-authored-by: Dongjoon Hyun <do...@apache.org>
Co-authored-by: zhou-jiang <zh...@apple.com>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.github/.licenserc.yaml | 1 +
.github/workflows/build_and_test.yml | 3 +
.gitignore | 1 +
.../helm/spark-kubernetes-operator/.helmignore | 12 ++
.../helm/spark-kubernetes-operator/Chart.yaml | 21 +++
.../templates/_helpers.tpl | 146 +++++++++++++++
.../templates/app-rbac.yaml | 176 ++++++++++++++++++
.../templates/operator-rbac.yaml | 160 ++++++++++++++++
.../templates/spark-operator.yaml | 205 +++++++++++++++++++++
.../templates/tests/test-rbac.yaml | 63 +++++++
.../helm/spark-kubernetes-operator/values.yaml | 185 +++++++++++++++++++
dev/.rat-excludes | 1 +
spark-operator-api/build.gradle | 8 +
13 files changed, 982 insertions(+)
diff --git a/.github/.licenserc.yaml b/.github/.licenserc.yaml
index 7365997..5507e4d 100644
--- a/.github/.licenserc.yaml
+++ b/.github/.licenserc.yaml
@@ -18,5 +18,6 @@ header:
- gradlew
- 'build/**'
- '**/*.json'
+ - '**/.helmignore'
comment: on-failure
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index 2c12aae..fd99dfc 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -44,6 +44,9 @@ jobs:
- name: Build with Gradle
run: |
./gradlew build
+ - name: Validate helm chart linting
+ run: |
+ helm lint --strict build-tools/helm/spark-kubernetes-operator
build-image:
name: "Build Operator Image CI"
runs-on: ubuntu-latest
diff --git a/.gitignore b/.gitignore
index 5e0e9b6..b174a96 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@
.vscode
/lib/
target/
+build-tools/helm/spark-kubernetes-operator/crds
# Gradle Files #
################
diff --git a/build-tools/helm/spark-kubernetes-operator/.helmignore b/build-tools/helm/spark-kubernetes-operator/.helmignore
new file mode 100644
index 0000000..5920392
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/.helmignore
@@ -0,0 +1,12 @@
+.DS_Store
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
diff --git a/build-tools/helm/spark-kubernetes-operator/Chart.yaml b/build-tools/helm/spark-kubernetes-operator/Chart.yaml
new file mode 100644
index 0000000..deed565
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/Chart.yaml
@@ -0,0 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+apiVersion: v2
+name: spark-kubernetes-operator
+description: A Helm chart for the Apache Spark Kubernetes Operator
+type: application
+version: 0.1.0-SNAPSHOT
+appVersion: 0.1.0
+icon: https://spark.apache.org/favicon.ico
diff --git a/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl b/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl
new file mode 100644
index 0000000..e8ee901
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "spark-operator.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "spark-operator.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "spark-operator.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "spark-operator.commonLabels" -}}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+helm.sh/chart: {{ include "spark-operator.chart" . }}
+{{- end }}
+
+{{/*
+Dynamic config (hot properties) labels
+*/}}
+{{- define "spark-operator.dynamicConfigLabels" -}}
+app.kubernetes.io/name: {{ include "spark-operator.name" . }}
+app.kubernetes.io/component: "operator-dynamic-config-overrides"
+{{- include "spark-operator.commonLabels" . }}
+{{- end }}
+
+{{/*
+Bootstrap config labels
+*/}}
+{{- define "spark-operator.configLabels" -}}
+app.kubernetes.io/name: {{ include "spark-operator.name" . }}
+app.kubernetes.io/component: "operator-config"
+{{- include "spark-operator.commonLabels" . }}
+{{- end }}
+
+{{/*
+Deployment selector labels
+*/}}
+{{- define "spark-operator.deploymentSelectorLabels" -}}
+app.kubernetes.io/name: {{ include "spark-operator.name" . }}
+app.kubernetes.io/component: "operator-deployment"
+{{- end }}
+
+{{/*
+Create the path of the operator image to use
+*/}}
+{{- define "spark-operator.imagePath" -}}
+{{- if .Values.image.digest }}
+{{- .Values.image.repository }}@{{ .Values.image.digest }}
+{{- else }}
+{{- .Values.image.repository }}:{{ default .Chart.AppVersion .Values.image.tag }}
+{{- end }}
+{{- end }}
+
+{{/*
+List of Spark app namespaces. If not provied in values, use the same namespace as operator
+*/}}
+{{- define "spark-operator.appNamespacesStr" -}}
+{{- if index (.Values.appResources.namespaces) "data" }}
+{{- $ns_list := join "," .Values.appResources.namespaces.data }}
+{{- printf "%s" $ns_list }}
+{{- else }}
+{{- printf "%s" .Release.Namespace }}
+{{- end }}
+{{- end }}
+
+{{/*
+Default property overrides
+*/}}
+{{- define "spark-operator.defaultPropertyOverrides" -}}
+# Runtime resolved properties
+spark.kubernetes.operator.namespace={{ .Release.Namespace }}
+spark.kubernetes.operator.name={{- include "spark-operator.name" . }}
+spark.kubernetes.operator.dynamicConfig.enabled={{ .Values.operatorConfiguration.dynamicConfig.enable }}
+{{- if .Values.appResources.namespaces.overrideWatchedNamespaces }}
+spark.kubernetes.operator.watchedNamespaces={{ include "spark-operator.appNamespacesStr" . | trim }}
+{{- end }}
+{{- end }}
+
+{{/*
+Readiness Probe properties overrides
+*/}}
+{{- define "spark-operator.readinessProbe.failureThreshold" -}}
+{{- default 30 .Values.operatorDeployment.operatorPod.operatorContainer.probes.startupProbe.failureThreshold }}
+{{- end }}
+{{- define "spark-operator.readinessProbe.periodSeconds" -}}
+{{- default 10 .Values.operatorDeployment.operatorPod.operatorContainer.probes.startupProbe.periodSeconds }}
+{{- end }}
+
+{{/*
+Liveness Probe properties override
+*/}}
+{{- define "spark-operator.livenessProbe.initialDelaySeconds" -}}
+{{- default 30 .Values.operatorDeployment.operatorPod.operatorContainer.probes.livenessProbe.initialDelaySeconds }}
+{{- end }}
+{{- define "spark-operator.livenessProbe.periodSeconds" -}}
+{{- default 10 .Values.operatorDeployment.operatorPod.operatorContainer.probes.livenessProbe.periodSeconds }}
+{{- end }}
+
+{{/*
+Readiness Probe property overrides
+*/}}
+{{- define "spark-operator.probePort" -}}
+{{- default 19091 .Values.operatorDeployment.operatorPod.operatorContainer.probes.port }}
+{{- end }}
diff --git a/build-tools/helm/spark-kubernetes-operator/templates/app-rbac.yaml b/build-tools/helm/spark-kubernetes-operator/templates/app-rbac.yaml
new file mode 100644
index 0000000..f76d56b
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/templates/app-rbac.yaml
@@ -0,0 +1,176 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{/*
+RBAC rules used to create the app (cluster)role based on the scope
+*/}}
+{{- define "spark-operator.appRbacRules" }}
+rules:
+ - apiGroups:
+ - ""
+ resources:
+ - pods
+ - services
+ - configmaps
+ - persistentvolumeclaims
+ verbs:
+ - '*'
+{{- end }}
+
+{{/*
+RoleRef for app service account rolebindings
+*/}}
+{{- define "spark-operator.appRoleRef" }}
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+{{- if .Values.appResources.clusterRole.create }}
+ kind: ClusterRole
+ name: {{ .Values.appResources.clusterRole.name }}
+{{- else }}
+ kind: Role
+ name: {{ .Values.appResources.role.name }}
+{{- end }}
+{{- end }}
+
+{{/*
+Labels and annotations to be applied
+*/}}
+{{- define "spark-operator.appLabels" -}}
+ {{- with .Values.appResources.labels }}
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+ {{- include "spark-operator.commonLabels" . | nindent 4 }}
+{{- end }}
+
+{{- define "spark-operator.appAnnotations" -}}
+ {{- with .Values.appResources.annotations }}
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+{{- end }}
+
+{{- define "spark-operator.appLabelsAnnotations" }}
+ labels:
+ {{ template "spark-operator.appLabels" $ }}
+ annotations:
+ {{ template "spark-operator.appAnnotations" $ }}
+{{- end }}
+---
+{{- $appResources := .Values.appResources -}}
+{{- $systemNs := .Release.Namespace -}}
+{{- $operatorRbac := .Values.operatorRbac -}}
+{{- if index (.Values.appResources.namespaces) "data" }}
+{{- range $appNs := .Values.appResources.namespaces.data }}
+{{- if $appResources.namespaces.create }}
+apiVersion: v1
+kind: Namespace
+metadata:
+ name: {{ $appNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+---
+{{- end }}
+{{- if $appResources.serviceAccount.create }}
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: {{ $appResources.serviceAccount.name }}
+ namespace: {{ $appNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+---
+{{- end }}
+{{- if $appResources.role.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: {{ $appResources.role.name }}
+ namespace: {{ $appNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+{{- template "spark-operator.appRbacRules" $ }}
+---
+{{- end }}
+{{- if $appResources.roleBinding.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: {{ $appResources.roleBinding.name }}
+ namespace: {{ $appNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+{{- template "spark-operator.appRoleRef" $ }}
+subjects:
+ - kind: ServiceAccount
+ name: {{ $appResources.serviceAccount.name }}
+ namespace: {{ $appNs }}
+---
+{{- end }}
+{{- end }}
+{{- else }}
+{{- if $appResources.serviceAccount.create }}
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: {{ $appResources.serviceAccount.name }}
+ namespace: {{ $systemNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+---
+{{- end }}
+{{- if $appResources.role.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: {{ $appResources.role.name }}
+ namespace: {{ $systemNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+{{- template "spark-operator.appRbacRules" $ }}
+---
+{{- end }}
+{{- if $appResources.roleBinding.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: {{ $appResources.serviceAccount.name }}
+ namespace: {{ $systemNs }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+{{- template "spark-operator.appRoleRef" $ }}
+subjects:
+ - kind: ServiceAccount
+ name: {{ $appResources.serviceAccount.name }}
+ namespace: {{ $systemNs }}
+---
+{{- end }}
+{{- end }}
+
+{{- if $appResources.clusterRole.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: {{ $appResources.clusterRole.name }}
+{{- template "spark-operator.appLabelsAnnotations" $ }}
+{{- template "spark-operator.appRbacRules" $ }}
+---
+{{- end }}
+{{- if $appResources.sparkApplicationSentinel.create }}
+{{- range $sentinelNs := .Values.appResources.sparkApplicationSentinel.sentinelNamespaces.data }}
+apiVersion: org.apache.spark/v1alpha1
+kind: SparkApplication
+metadata:
+ name: {{ $appResources.sparkApplicationSentinel.name }}
+ namespace: {{ $sentinelNs }}
+ labels:
+ "spark.operator/sentinel": "true"
+ {{- template "spark-operator.appLabels" $ }}
+ annotations:
+ {{- template "spark-operator.appAnnotations" $ }}
+{{- end }}
+---
+{{- end }}
diff --git a/build-tools/helm/spark-kubernetes-operator/templates/operator-rbac.yaml b/build-tools/helm/spark-kubernetes-operator/templates/operator-rbac.yaml
new file mode 100644
index 0000000..f8364e6
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/templates/operator-rbac.yaml
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{/*
+RBAC rules used to create the operator (cluster)role
+*/}}
+{{- define "spark-operator.operatorRbacRules" }}
+rules:
+ - apiGroups:
+ - ""
+ resources:
+ - pods
+ - services
+ - configmaps
+ - persistentvolumeclaims
+ - persistentvolumes
+ - events
+ verbs:
+ - '*'
+ - apiGroups:
+ - "apps"
+ resources:
+ - statefulsets
+ verbs:
+ - '*'
+ - apiGroups:
+ - "org.apache.spark"
+ resources:
+ - '*'
+ verbs:
+ - '*'
+{{- end }}
+
+{{/*
+Labels and annotations to be applied on rbacResources
+*/}}
+{{- define "spark-operator.rbacLabelsAnnotations" }}
+ labels:
+ {{- with .Values.operatorRbac.labels }}
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+ {{- include "spark-operator.commonLabels" . | nindent 4 }}
+ {{- with .Values.operatorRbac.annotations }}
+ annotations:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+{{- end }}
+
+---
+# Service account and rolebindings for operator
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: {{ .Values.operatorRbac.serviceAccount.name }}
+ namespace: {{ .Release.Namespace }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+---
+{{- if .Values.operatorRbac.clusterRoleBinding.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: {{ .Values.operatorRbac.clusterRoleBinding.name }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: {{ .Values.operatorRbac.clusterRole.name }}
+subjects:
+ - kind: ServiceAccount
+ name: {{ .Values.operatorRbac.serviceAccount.name }}
+ namespace: {{ .Release.Namespace }}
+{{- end }}
+---
+{{- if .Values.operatorRbac.clusterRole.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: {{ .Values.operatorRbac.clusterRole.name }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+{{- template "spark-operator.operatorRbacRules" $ }}
+---
+{{- end }}
+{{- if .Values.operatorConfiguration.dynamicConfig.enable }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: {{ .Values.operatorRbac.configManagement.roleName }}
+ namespace: {{ .Release.Namespace }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+rules:
+ - apiGroups:
+ - ""
+ resources:
+ - configmaps
+ verbs:
+ - '*'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: {{ .Values.operatorRbac.configManagement.roleBindingName }}
+ namespace: {{ .Release.Namespace }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: {{ .Values.operatorRbac.configManagement.roleName }}
+subjects:
+ - kind: ServiceAccount
+ name: {{ .Values.operatorRbac.serviceAccount.name }}
+ namespace: {{ .Release.Namespace }}
+---
+{{- end }}
+
+{{- $appResources := .Values.appResources -}}
+{{- $systemNs := .Release.Namespace -}}
+{{- $operatorRbac := .Values.operatorRbac -}}
+{{- if index (.Values.appResources.namespaces) "data" }}
+{{- range $appNs := .Values.appResources.namespaces.data }}
+{{- if $operatorRbac.role.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: {{ $operatorRbac.role.name }}
+ namespace: {{ $appNs }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+{{- template "spark-operator.operatorRbacRules" $ }}
+---
+{{- end }}
+{{- if $operatorRbac.roleBinding.create }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: {{ $operatorRbac.roleBinding.name }}
+ namespace: {{ $appNs }}
+{{- template "spark-operator.rbacLabelsAnnotations" $ }}
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: {{ $operatorRbac.roleBinding.roleRef.kind }}
+ name: {{ $operatorRbac.roleBinding.roleRef.name }}
+subjects:
+ - kind: ServiceAccount
+ name: {{ $operatorRbac.serviceAccount.name }}
+ namespace: {{ $systemNs }}
+---
+{{- end }}
+{{- end }}
+{{- end }}
diff --git a/build-tools/helm/spark-kubernetes-operator/templates/spark-operator.yaml b/build-tools/helm/spark-kubernetes-operator/templates/spark-operator.yaml
new file mode 100644
index 0000000..5edd228
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/templates/spark-operator.yaml
@@ -0,0 +1,205 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: {{ include "spark-operator.name" . }}
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "spark-operator.deploymentSelectorLabels" . | nindent 4 }}
+ {{- include "spark-operator.commonLabels" . | nindent 4 }}
+spec:
+ replicas: {{ .Values.operatorDeployment.replicas }}
+ strategy:
+ {{- toYaml .Values.operatorDeployment.strategy | nindent 4 }}
+ selector:
+ matchLabels:
+ {{- include "spark-operator.deploymentSelectorLabels" . | nindent 6 }}
+ template:
+ metadata:
+ labels:
+ {{- include "spark-operator.deploymentSelectorLabels" . | nindent 8 }}
+ {{- if index (.Values.operatorDeployment.operatorPod) "labels" }}
+ {{- with .Values.operatorDeployment.operatorPod.labels }}
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- end }}
+ annotations:
+ kubectl.kubernetes.io/default-container: {{ .Chart.Name }}
+ {{- if index (.Values.operatorDeployment.operatorPod) "annotations" }}
+ {{- with .Values.operatorDeployment.operatorPod.annotations }}
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- end }}
+ spec:
+ {{- with .Values.operatorDeployment.operatorPod.priorityClassName }}
+ priorityClassName: {{ . }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.securityContext }}
+ securityContext:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- if .Values.operatorDeployment.operatorPod.nodeSelector }}
+ nodeSelector: {{ toYaml .Values.operatorDeployment.operatorPod.nodeSelector | nindent 8 }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.tolerations }}
+ tolerations:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.affinity }}
+ affinity:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.imagePullSecrets }}
+ imagePullSecrets:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ serviceAccountName: {{ .Values.operatorRbac.serviceAccount.name }}
+ {{- if .Values.operatorDeployment.operatorPod.topologySpreadConstraints }}
+ topologySpreadConstraints: {{ toYaml .Values.operatorDeployment.operatorPod.topologySpreadConstraints | nindent 8 }}
+ {{- end }}
+ containers:
+ - name: {{ .Chart.Name }}
+ image: {{ include "spark-operator.imagePath" . }}
+ imagePullPolicy: {{ .Values.image.pullPolicy }}
+ command: [ "./docker-entrypoint.sh", "operator" ]
+ ports:
+ - containerPort: {{ include "spark-operator.probePort" . }}
+ name: probe-port
+ env:
+ - name: OPERATOR_NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ - name: HOST_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.hostIP
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ - name: POD_NAME
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.name
+ - name: OPERATOR_NAME
+ value: {{ include "spark-operator.name" . }}
+ - name: LOG_CONFIG
+ value: -Dlog4j.configurationFile=/opt/spark-operator/conf/log4j2.properties
+ - name: OPERATOR_JAVA_OPTS
+ value: {{ .Values.operatorDeployment.operatorPod.operatorContainer.jvmArgs }}
+ {{- with .Values.operatorDeployment.operatorPod.operatorContainer.env }}
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.operatorContainer.envFrom }}
+ envFrom:
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.operatorContainer.resources }}
+ resources:
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ readinessProbe:
+ httpGet:
+ port: probe-port
+ path: /readyz
+ failureThreshold: {{ include "spark-operator.readinessProbe.failureThreshold" . }}
+ periodSeconds: {{ include "spark-operator.readinessProbe.periodSeconds" . }}
+ livenessProbe:
+ httpGet:
+ port: probe-port
+ path: /healthz
+ initialDelaySeconds: {{ include "spark-operator.livenessProbe.initialDelaySeconds" . }}
+ periodSeconds: {{ include "spark-operator.livenessProbe.periodSeconds" . }}
+ {{- with .Values.operatorDeployment.operatorPod.operatorContainer.securityContext }}
+ securityContext:
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ volumeMounts:
+ - name: spark-operator-config-volume
+ mountPath: /opt/spark-operator/conf
+ - name: logs-volume
+ mountPath: /opt/spark-operator/logs
+ {{- with .Values.operatorDeployment.operatorPod.operatorContainer.volumeMounts }}
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ {{- with .Values.operatorDeployment.operatorPod.additionalContainers }}
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- if index (.Values.operatorDeployment.operatorPod) "dnsPolicy" }}
+ dnsPolicy: {{ .Values.operatorDeployment.operatorPod.dnsPolicy | quote }}
+ {{- end }}
+ {{- if index (.Values.operatorDeployment.operatorPod) "dnsConfig" }}
+ dnsConfig:
+ {{- with .Values.operatorDeployment.operatorPod.dnsConfig }}
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- end }}
+ restartPolicy: Always
+ volumes:
+ - name: spark-operator-config-volume
+ configMap:
+ name: spark-kubernetes-operator-configuration
+ - name: logs-volume
+ emptyDir: { }
+ {{- with .Values.operatorDeployment.operatorPod.volumes }}
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: spark-kubernetes-operator-configuration
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "spark-operator.configLabels" . | nindent 4 }}
+data:
+ log4j2.properties: |+
+{{- if .Values.operatorConfiguration.append }}
+ {{- $.Files.Get "conf/log4j2.properties" | nindent 4 -}}
+{{- end }}
+{{- if index (.Values.operatorConfiguration) "log4j2.properties" }}
+ {{- index (.Values.operatorConfiguration) "log4j2.properties" | nindent 4 -}}
+{{- end }}
+ spark-operator.properties: |+
+ {{- include "spark-operator.defaultPropertyOverrides" . | nindent 4 }}
+{{- if .Values.operatorConfiguration.append }}
+ {{- $.Files.Get "conf/spark-operator.properties" | nindent 4 -}}
+{{- end }}
+{{- if index (.Values.operatorConfiguration) "spark-operator.properties" }}
+ {{- index (.Values.operatorConfiguration) "spark-operator.properties" | nindent 4 -}}
+{{- end }}
+ metrics.properties: |+
+{{- if index (.Values.operatorConfiguration) "metrics.properties" }}
+ {{- index (.Values.operatorConfiguration) "metrics.properties" | nindent 4 -}}
+{{- end }}
+---
+{{- if .Values.operatorConfiguration.dynamicConfig.create }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: spark-kubernetes-operator-dynamic-configuration
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "spark-operator.dynamicConfigLabels" . | nindent 4 }}
+ annotations:
+ {{- toYaml .Values.operatorConfiguration.dynamicConfig.annotations | nindent 4 }}
+{{- with .Values.operatorConfiguration.dynamicConfig.data }}
+data:
+ {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- end }}
diff --git a/build-tools/helm/spark-kubernetes-operator/templates/tests/test-rbac.yaml b/build-tools/helm/spark-kubernetes-operator/templates/tests/test-rbac.yaml
new file mode 100644
index 0000000..33d8fa3
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/templates/tests/test-rbac.yaml
@@ -0,0 +1,63 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+apiVersion: v1
+kind: Pod
+metadata:
+ name: "{{ include "spark-operator.name" . }}-test-operator-rbac"
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "spark-operator.commonLabels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": test
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed
+spec:
+ containers:
+ - name: kubectl
+ image: bitnami/kubectl:latest
+ command: ['bash', '-c' ]
+ args: [
+ 'kubectl auth can-i list sparkapplications --all-namespaces',
+ 'kubectl auth can-i create pods --all-namespaces',
+ 'kubectl auth can-i create services --all-namespaces',
+ 'kubectl auth can-i create configmaps --all-namespaces',
+ 'kubectl auth can-i create persistentvolumeclaims',
+ 'kubectl auth can-i create events --all-namespaces'
+ ]
+ serviceAccountName: {{ .Values.operatorRbac.serviceAccount.name }}
+ restartPolicy: Never
+---
+apiVersion: v1
+kind: Pod
+metadata:
+ name: "{{ include "spark-operator.name" . }}-test-app-rbac"
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "spark-operator.commonLabels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": test
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed
+spec:
+ containers:
+ - name: kubectl
+ image: bitnami/kubectl:latest
+ command: ['bash', '-c' ]
+ args: [
+ 'kubectl auth can-i create pods -n {{ .Release.Namespace }}',
+ 'kubectl auth can-i create configmaps -n {{ .Release.Namespace }}',
+ 'kubectl auth can-i create services -n {{ .Release.Namespace }}',
+ 'kubectl auth can-i create persistentvolumeclaims -n {{ .Release.Namespace }}'
+ ]
+ serviceAccountName: {{ .Values.appResources.serviceAccount.name }}
+ restartPolicy: Never
diff --git a/build-tools/helm/spark-kubernetes-operator/values.yaml b/build-tools/helm/spark-kubernetes-operator/values.yaml
new file mode 100644
index 0000000..87fda1e
--- /dev/null
+++ b/build-tools/helm/spark-kubernetes-operator/values.yaml
@@ -0,0 +1,185 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+image:
+ repository: spark-kubernetes-operator
+ pullPolicy: IfNotPresent
+ tag: 0.1.0
+ # If image digest is set then it takes precedence and the image tag will be ignored
+ # digest: ""
+
+imagePullSecrets: [ ]
+
+operatorDeployment:
+ # Replicas must be 1 unless leader election is enabled
+ replicas: 1
+ # Strategy type must be 'Recreate' unless leader election is enabled
+ strategy:
+ type: Recreate
+ operatorPod:
+ priorityClassName: null
+ annotations: { }
+ labels: { }
+ affinity: { }
+ nodeSelector: { }
+ # Node tolerations for operator pod assignment
+ # https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
+ tolerations: [ ]
+ # Topology spread constrains
+ # https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/
+ topologySpreadConstraints: [ ]
+ operatorContainer:
+ jvmArgs: "-Dfile.encoding=UTF8"
+ env:
+ envFrom:
+ volumeMounts: { }
+ resources:
+ limits:
+ cpu: "1"
+ ephemeral-storage: 2Gi
+ memory: 2Gi
+ requests:
+ cpu: "1"
+ ephemeral-storage: 2Gi
+ memory: 2Gi
+ probes:
+ port: 19091
+ livenessProbe:
+ periodSeconds: 10
+ initialDelaySeconds: 30
+ startupProbe:
+ failureThreshold: 30
+ periodSeconds: 10
+ # By default, operator container is configured to comply restricted standard
+ # https://kubernetes.io/docs/concepts/security/pod-security-standards/
+ securityContext:
+ allowPrivilegeEscalation: false
+ capabilities:
+ drop:
+ - ALL
+ runAsNonRoot: true
+ runAsUser: 185
+ seccompProfile:
+ type: RuntimeDefault
+ additionalContainers: { }
+ volumes: { }
+ securityContext: { }
+ dnsPolicy:
+ dnsConfig:
+
+operatorRbac:
+ serviceAccount:
+ create: true
+ name: "spark-operator"
+ clusterRole:
+ create: true
+ name: "spark-operator-clusterrole"
+ clusterRoleBinding:
+ create: true
+ name: "spark-operator-clusterrolebinding"
+ role:
+ # If enabled, a Role would be created inside each app namespace, as configured
+ # in {appResources.namespaces.data} for operator. Please enable *at least one* of
+ # {operatorRbac.clusterRole.create} or {operatorRbac.role.create} for operator
+ # unless permission is provided separately from the chart
+ create: false
+ name: "spark-operator-role"
+ roleBinding:
+ # If enabled, a RoleBinding would be created inside each app namespace, as configured
+ # in {appResources.namespaces.data} for operator. Please enable *at least one* of
+ # {operatorRbac.clusterRoleBinding.create} or {operatorRbac.roleBinding.create} for
+ # operator unless permission is provided separately from the chart
+ create: false
+ name: "spark-operator-rolebinding"
+ # It's possible to make the rolebinding refers to role / clusterrole as applicable
+ roleRef:
+ kind: Role
+ name: "spark-operator-role"
+ configManagement:
+ # Role to enable operator loading hot properties from config map
+ create: true
+ roleName: "spark-operator-config-monitor"
+ roleBindingName: "spark-operator-config-monitor-role-binding"
+ labels:
+ "app.kubernetes.io/component": "operator-rbac"
+
+appResources:
+ # Create namespace(s), service account(s), clusterrole, role(s) and rolebinding(s) for SparkApps
+ namespaces:
+ create: true
+ # When enabled, value for conf property `spark.operator.watched.namespaces` would be set to
+ # the values provided in {appResources.namespaces.data} field as well
+ overrideWatchedNamespaces: true
+ data:
+ # - "spark-1"
+ # - "spark-2"
+ serviceAccount:
+ create: true
+ name: "spark"
+ role:
+ # When enabled, a role would be created in each app namespace, as configured in
+ # {appResources.namespaces.data} for Spark apps. Please enable *at least one* of
+ # {appResources.roles.create} or {appResources.clusterRole.create} for
+ # Spark app unless permission is provided separately from the chart
+ create: false
+ name: "spark-app-role"
+ clusterRole:
+ # When enabled, a clusterrole would be created for Spark apps. Please enable *at least one* of
+ # {appResources.roles.create} or {appResources.clusterRole.create} for Spark app unless
+ # permission is provided separately from the chart
+ create: true
+ name: "spark-app-clusterrole"
+ roleBinding:
+ create: true
+ name: "spark-app-rolebinding"
+ sparkApplicationSentinel:
+ create: false
+ name: "spark-app-sentinel"
+ sentinelNamespaces:
+ data:
+ # - "spark-1"
+ # - "spark-2"
+ # When enabled, a sentinel resources will be deployed to namespace(s) provided/
+ # Note that sentinelNamespaces list should be a subset of {appResources.namespaces.data}
+ # - "default"
+ # App resources are by default annotated to avoid app abort due to operator upgrade
+ annotations:
+ "helm.sh/resource-policy": keep
+ # labels to be added on app resources
+ labels:
+ "app.kubernetes.io/component": "spark-app"
+
+operatorConfiguration:
+ # If set to true, below properties would be appended to default conf files under conf/
+ # Otherwise, below would override default conf files
+ append: true
+ log4j2.properties: |+
+ # Logging Overrides
+ # rootLogger.level=DEBUG
+ spark-operator.properties: |+
+ # Property Overrides. e.g.
+ # spark.kubernetes.operator.reconciler.intervalSeconds=60
+ metrics.properties: |+
+ # Metrics Properties Overrides
+ dynamicConfig:
+ # Enable this for hot properties loading.
+ enable: false
+ # Enable this to create a config map for hot property loading
+ create: false
+ annotations:
+ "helm.sh/resource-policy": keep
+ data:
+ # Spark Operator Config Runtime Properties Overrides. e.g.
+ spark.kubernetes.operator.reconciler.intervalSeconds: 60
diff --git a/dev/.rat-excludes b/dev/.rat-excludes
index 57dc552..4234634 100644
--- a/dev/.rat-excludes
+++ b/dev/.rat-excludes
@@ -15,3 +15,4 @@ RELEASE
build
.*gradle
.*json
+.helmignore
diff --git a/spark-operator-api/build.gradle b/spark-operator-api/build.gradle
index 25c7069..8b1dfc3 100644
--- a/spark-operator-api/build.gradle
+++ b/spark-operator-api/build.gradle
@@ -32,3 +32,11 @@ tasks.register('finalizeGeneratedCRD', Exec) {
println "Updating PrinterColumns for generated CRD"
commandLine 'sh', './src/main/resources/printer-columns.sh'
}
+
+// Copy generated yaml to Helm charts
+tasks.register('relocateGeneratedCRD', Copy) {
+ dependsOn finalizeGeneratedCRD
+ from "build/classes/java/main/META-INF/fabric8/sparkapplications.org.apache.spark-v1.yml"
+ into "../build-tools/helm/spark-kubernetes-operator/crds"
+ rename '(.+).yml', '$1.yaml'
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org