This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new b3cf5589 [SPARK-59596] Fix Helm chart configuration table in `operations.md` to match `values.yaml`
b3cf5589 is described below
commit b3cf558937823d0b6e1cc0d691435b606bc23429
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Sep 17 03:05:49 2026 -0700
[SPARK-59596] Fix Helm chart configuration table in `operations.md` to match `values.yaml`
### What changes were proposed in this pull request?
This PR aims to fix the Helm chart configuration table in `docs/operations.md` to be consistent with the Helm chart (`values.yaml` and `templates/*.yaml`).
**Fix the wrong keys, descriptions and default values**
| Parameter | Before | After |
|---|---|---|
| Key | `operatorDeployment.replica` | `operatorDeployment.replicas` |
| Key | `operatorDeployment.additionalContainers` | `operatorDeployment.operatorPod.additionalContainers` |
| Default value of `operatorDeployment.operatorPod.operatorContainer.jvmArgs` | `-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC` | `-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC -XX:InitialRAMPercentage=80 -XX:MaxRAMPercentage=80 -XX:+AlwaysPreTouch -XX:+UseCompactObjectHeaders` |
| Description of `operatorRbac.serviceAccount.name` | Name of the operator Role. | Name of the operator ServiceAccount. |
| Description of `operatorRbac.configManagement.create` | ... (hot property loading and leader election). | ... (hot property loading from ConfigMap). Requires `dynamicConfig` with the `configMap` source. |
| Default value of `operatorRbac.configManagement.roleName` | `spark-operator-config-role` | `"spark-operator-config-monitor"` |
| Key | `operatorRbac.configManagement.roleBinding` | `operatorRbac.configManagement.roleBindingName` |
| Key | `workloadResources.serviceAccounts.create` | `workloadResources.serviceAccount.create` |
| Key | `workloadResources.serviceAccounts.name` | `workloadResources.serviceAccount.name` |
| Key | `workloadResources.sparkApplicationSentinel.sentinelNamespaces` | `workloadResources.sparkApplicationSentinel.sentinelNamespaces.data` |
| Key | `workloadResources.sparkClusterSentinel.sentinelNamespaces` | `workloadResources.sparkClusterSentinel.sentinelNamespaces.data` |
**Add the missing keys**
| Parameter | Default value |
|---|---|
| `nameOverride` | |
| `operatorDeployment.networkPolicy.enabled` | `false` |
| `operatorDeployment.networkPolicy.enable` (deprecated) | |
| `operatorDeployment.networkPolicy.metricsIngress` | |
| `operatorDeployment.operatorPod.affinity` | |
| `operatorDeployment.operatorPod.tolerations` | |
| `operatorDeployment.operatorPod.dnsPolicy` | |
| `operatorDeployment.operatorPod.operatorContainer.volumeMounts` | |
| `operatorDeployment.operatorPod.operatorContainer.metrics.port` | `19090` |
| `operatorRbac.annotations` | |
| `operatorConfiguration.configMap.annotations` | |
| `operatorConfiguration.configMap.labels` | |
| `operatorConfiguration.dynamicConfig.source` | `configMap` |
### Why are the changes needed?
The documentation is inconsistent with the actual Helm chart. Some documented keys do not exist in the chart, so users cannot configure the chart by following the documentation. In addition, several supported keys are not documented.
### Does this PR introduce _any_ user-facing change?
No behavior change. This is a documentation-only fix.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
Closes #839 from dongjoon-hyun/SPARK-59596.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
docs/operations.md | 35 ++++++++++++++++++++++++-----------
1 file changed, 24 insertions(+), 11 deletions(-)
diff --git a/docs/operations.md b/docs/operations.md
index 11a4d099..766ab06b 100644
--- a/docs/operations.md
+++ b/docs/operations.md
@@ -83,25 +83,34 @@ following table:
| image.tag | The image tag of spark-kubernetes-operator. | 1.1.0-SNAPSHOT |
| image.digest | The image digest of spark-kubernetes-operator. If set then it takes precedence and the image tag will be ignored. | |
| imagePullSecrets | The image pull secrets of spark-kubernetes-operator. | |
-| operatorDeployment.replica | Operator replica count. Must be 1 unless leader election is configured. | 1 |
+| nameOverride | Override the chart name used in the names and labels of the created resources. | |
+| operatorDeployment.replicas | Operator replica count. Must be 1 unless leader election is configured. | 1 |
| operatorDeployment.strategy.type | Operator pod upgrade strategy. Must be Recreate unless leader election is configured. | Recreate |
+| operatorDeployment.networkPolicy.enabled | When enabled, a NetworkPolicy allows ingress to the operator pod only on the health probe and metrics ports. Requires a CNI plugin with NetworkPolicy support. | false |
+| operatorDeployment.networkPolicy.enable | Deprecated, use `operatorDeployment.networkPolicy.enabled`. Still honored: `enable: true` wins over `enabled: false`. Removed in chart 2.0.0 (SPARK-59533). | |
+| operatorDeployment.networkPolicy.metricsIngress | List of NetworkPolicyPeer(s) allowed to reach the metrics port, e.g. the Prometheus scraper. When empty, ingress to the metrics port is denied. | |
| operatorDeployment.operatorPod.annotations | Custom annotations to be added to the operator pod | |
| operatorDeployment.operatorPod.labels | Custom labels to be added to the operator pod | |
+| operatorDeployment.operatorPod.affinity | Custom affinity to be added to the operator pod. | |
| operatorDeployment.operatorPod.nodeSelector | Custom nodeSelector to be added to the operator pod. | |
+| operatorDeployment.operatorPod.tolerations | Custom tolerations to be added to the operator pod. | |
| operatorDeployment.operatorPod.topologySpreadConstraints | Custom topologySpreadConstraints to be added to the operator pod. | |
+| operatorDeployment.operatorPod.dnsPolicy | DNS policy to be used by the operator pod. | |
| operatorDeployment.operatorPod.dnsConfig | DNS configuration to be used by the operator pod. | |
| operatorDeployment.operatorPod.volumes | Additional volumes to be added to the operator pod. | |
| operatorDeployment.operatorPod.priorityClassName | Priority class name to be used for the operator pod | |
| operatorDeployment.operatorPod.securityContext | Security context overrides for the operator pod | |
-| operatorDeployment.operatorPod.operatorContainer.jvmArgs | JVM arg override for the operator container. | `"-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC"` |
+| operatorDeployment.operatorPod.operatorContainer.jvmArgs | JVM arg override for the operator container. | `"-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC -XX:InitialRAMPercentage=80 -XX:MaxRAMPercentage=80 -XX:+AlwaysPreTouch -XX:+UseCompactObjectHeaders"` |
| operatorDeployment.operatorPod.operatorContainer.env | Custom env to be added to the operator container. | |
| operatorDeployment.operatorPod.operatorContainer.envFrom | Custom envFrom to be added to the operator container, e.g. for downward API. | |
+| operatorDeployment.operatorPod.operatorContainer.volumeMounts | Additional volume mounts to be added to the operator container. | |
| operatorDeployment.operatorPod.operatorContainer.probes | Probe config for the operator container. | |
+| operatorDeployment.operatorPod.operatorContainer.metrics.port | Metrics port of the operator container. | 19090 |
| operatorDeployment.operatorPod.operatorContainer.securityContext | Security context overrides for the operator container. | run as non root for baseline security standard compliance |
| operatorDeployment.operatorPod.operatorContainer.resources | Resources for the operator container. | memory 2Gi, ephemeral storage 2Gi and 1 cpu |
-| operatorDeployment.additionalContainers | Additional containers to be added to the operator pod, e.g. sidecar. | |
+| operatorDeployment.operatorPod.additionalContainers | Additional containers to be added to the operator pod, e.g. sidecar. | |
| operatorRbac.serviceAccount.create | Whether to create service account for operator to use. | true |
-| operatorRbac.serviceAccount.name | Name of the operator Role. | `"spark-operator"` |
+| operatorRbac.serviceAccount.name | Name of the operator ServiceAccount. | `"spark-operator"` |
| operatorRbac.clusterRole.create | Whether to create ClusterRole for operator to use. | true |
| operatorRbac.clusterRole.name | Name of the operator ClusterRole. | `"spark-operator-clusterrole"` |
| operatorRbac.clusterRoleBinding.create | Whether to create ClusterRoleBinding for operator to use. | true |
@@ -111,11 +120,12 @@ following table:
| operatorRbac.roleBinding.create | Whether to create RoleBinding for operator to use. At least one of `clusterRoleBinding.create` or `roleBinding.create` should be enabled | false |
| operatorRbac.roleBinding.name | Name of the operator RoleBinding in each workload namespace(s). | `"spark-operator-rolebinding"` |
| operatorRbac.roleBinding.roleRef | RoleRef for the created Operator RoleBinding. Override this when you want the created RoleBinding refer to ClusterRole / Role that's different from the default operator Role. | Refers to default `operatorRbac.role.name` |
-| operatorRbac.configManagement.create | Enable this to create a Role for operator configuration management (hot property loading and leader election). | true |
-| operatorRbac.configManagement.roleName | Role name for operator configuration management. | `spark-operator-config-role` |
-| operatorRbac.configManagement.roleBinding | RoleBinding name for operator configuration management. | `"spark-operator-config-monitor-role-binding"` |
+| operatorRbac.configManagement.create | Enable this to create a Role for operator configuration management (hot property loading from ConfigMap). Requires `dynamicConfig` with the `configMap` source. | true |
+| operatorRbac.configManagement.roleName | Role name for operator configuration management. | `"spark-operator-config-monitor"` |
+| operatorRbac.configManagement.roleBindingName | RoleBinding name for operator configuration management. | `"spark-operator-config-monitor-role-binding"` |
| operatorRbac.kueue.enabled | Grant the operator access to Kueue `workloads`, `resourceflavors`, `workloadpriorityclasses` and to `priorityclasses`. The cluster-scoped ones need `clusterRole.create`. See [Optional Prerequisites](#optional-prerequisites). | false |
| operatorRbac.labels | Labels to be applied on all created `operatorRbac` resources. | `"app.kubernetes.io/component": "operator-rbac"` |
+| operatorRbac.annotations | Annotations to be applied on all created `operatorRbac` resources. | |
| workloadResources.namespaces.create | Whether to create dedicated namespaces for Spark workload. | true |
| workloadResources.namespaces.overrideWatchedNamespaces | When enabled, operator would by default only watch namespace(s) provided in data field. | true |
| workloadResources.namespaces.data | List of namespaces to create for Spark workload. The chart namespace would be used if this is empty. | |
@@ -125,23 +135,26 @@ following table:
| workloadResources.role.name | Name for Spark workload Role. | "spark-workload-role" |
| workloadResources.roleBinding.create | When enabled, a RoleBinding would be created in each namespace for Spark workload. This shall be enabled unless access is configured from 3rd party. | true |
| workloadResources.roleBinding.name | Name of the Spark workload RoleBinding. | "spark-workload-rolebinding" |
-| workloadResources.serviceAccounts.create | Whether to create a service account for Spark workload. | true |
-| workloadResources.serviceAccounts.name | The name of Spark workload service account. | `spark` |
+| workloadResources.serviceAccount.create | Whether to create a service account for Spark workload. | true |
+| workloadResources.serviceAccount.name | The name of Spark workload service account. | `spark` |
| workloadResources.labels | Labels to be applied for all workload resources. | `"app.kubernetes.io/component": "spark-workload"` |
| workloadResources.annotations | Annotations to be applied for all workload resources. | `"helm.sh/resource-policy": keep` |
| workloadResources.sparkApplicationSentinel.create | If enabled, sentinel resources will be created for operator to watch and reconcile for the health probe purpose. | false |
| workloadResources.sparkApplicationSentinel.name | Name for sentinel resources. | "spark-app-sentinel" |
-| workloadResources.sparkApplicationSentinel.sentinelNamespaces | A list of namespaces where sentinel resources will be created in. Note that these namespaces have to be a subset of `workloadResources.namespaces.data`. | |
+| workloadResources.sparkApplicationSentinel.sentinelNamespaces.data | A list of namespaces where sentinel resources will be created in. Note that these namespaces have to be a subset of `workloadResources.namespaces.data`. | |
| workloadResources.sparkClusterSentinel.create | If enabled, sentinel resources will be created for operator to watch and reconcile for the health probe purpose. | false |
| workloadResources.sparkClusterSentinel.name | Name for sentinel resources. | "spark-cluster-sentinel" |
-| workloadResources.sparkClusterSentinel.sentinelNamespaces | A list of namespaces where sentinel resources will be created in. Note that these namespaces have to be a subset of `workloadResources.namespaces.data`. | |
+| workloadResources.sparkClusterSentinel.sentinelNamespaces.data | A list of namespaces where sentinel resources will be created in. Note that these namespaces have to be a subset of `workloadResources.namespaces.data`. | |
| operatorConfiguration.append | If set to true, below conf file & properties would be appended to default conf. Otherwise, they would override default properties. | true |
+| operatorConfiguration.configMap.annotations | Annotations to be applied for the operator configuration config map. | |
+| operatorConfiguration.configMap.labels | Labels to be applied for the operator configuration config map. | |
| operatorConfiguration.log4j2.properties | The default log4j2 configuration. | Refer default [log4j2.properties](../build-tools/helm/spark-kubernetes-operator/conf/log4j2.properties) |
| operatorConfiguration.spark-operator.properties | The default operator configuration. | |
| operatorConfiguration.metrics.properties | The default operator metrics (sink) configuration. | |
| operatorConfiguration.dynamicConfig.create | If set to true, a config map would be created & watched by operator as source of truth for hot properties loading. | false |
| operatorConfiguration.dynamicConfig.enabled | If set to true, operator would honor the created config map as source of truth for hot properties loading. | false |
| operatorConfiguration.dynamicConfig.enable | Deprecated, use `operatorConfiguration.dynamicConfig.enabled`. Still honored: `enable: true` wins over `enabled: false`. Removed in chart 2.0.0 (SPARK-59533). | |
+| operatorConfiguration.dynamicConfig.source | Source of the dynamic config: `configMap` (watched via informer, requires RBAC to read config maps) or `file` (periodically reloaded from the mounted config map). | `configMap` |
| operatorConfiguration.dynamicConfig.annotations | Annotations to be applied for the dynamicConfig resources. | `"helm.sh/resource-policy": keep` |
| operatorConfiguration.dynamicConfig.data | Data field (key-value pairs) that acts as hot properties in the config map. | `spark.kubernetes.operator.reconciler.intervalSeconds: "60"` |
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org