You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "peter-toth (via GitHub)" <gi...@apache.org> on 2026/09/16 12:09:35 UTC
[PR] [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt` [spark-kubernetes-operator]
peter-toth opened a new pull request, #830:
URL: https://github.com/apache/spark-kubernetes-operator/pull/830
### What changes were proposed in this pull request?
This PR adds `templates/NOTES.txt` to the Helm chart. It prints a deprecation warning when a values file still carries `operatorDeployment.networkPolicy.enable` or `operatorConfiguration.dynamicConfig.enable`, the legacy keys that [SPARK-59504](https://issues.apache.org/jira/browse/SPARK-59504) replaced with `enabled`.
`hasKey` is an exact presence signal for these two, because SPARK-59504 removed `enable` from the chart defaults. A key set to `false` still warns: the deprecation is about the key, not its value.
The ASF header sits inside a `{{- /* ... */ -}}` comment, so `skywalking-eyes` sees it and `helm install` does not print it.
### Why are the changes needed?
The deprecation is announced in `values.yaml`, `values.schema.json` and `docs/operations.md`, but `helm install` and `helm upgrade` say nothing. Docs only reach people who go looking. When [SPARK-59533](https://issues.apache.org/jira/browse/SPARK-59533) removes the keys in chart `2.0.0`, a user still on `networkPolicy.enable: true` loses their NetworkPolicy with no error.
### Does this PR introduce _any_ user-facing change?
Yes. `helm install` and `helm upgrade` now print a NOTES section. Users on the current `enabled` keys see one line naming the installed version:
```
NOTES:
Apache Spark Kubernetes Operator 1.1.0-SNAPSHOT is installed.
```
Users still on a legacy key additionally get a warning naming the replacement and the removal target:
```
NOTES:
Apache Spark Kubernetes Operator 1.1.0-SNAPSHOT is installed.
WARNING: operatorDeployment.networkPolicy.enable is deprecated, use
operatorDeployment.networkPolicy.enabled instead. The legacy key is still honored:
the NetworkPolicy is created when either key is true. It will be removed in chart
2.0.0, see SPARK-59533.
```
### How was this patch tested?
- `helm lint --strict` passes with the default values and with each legacy key set.
- Rendered the notes for all five combinations and confirmed the warning fires on key presence only:
| values | notes |
|---|---|
| defaults | no warning |
| `networkPolicy.enable=true` | networkPolicy warning |
| `dynamicConfig.enable=true` | dynamicConfig warning |
| both legacy keys set to `false` | both warnings |
| both new `enabled` keys set | no warning |
- New `Validate the deprecated helm key warning` step in the `helm-tests / configmap-metadata` job: upgrades with both legacy keys and asserts both warnings are printed, then upgrades back to the default values and fails if any warning is printed. `helm template` does not render `NOTES.txt`, so a real cluster is the only place this is observable.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #830:
URL: https://github.com/apache/spark-kubernetes-operator/pull/830#discussion_r4027115427
##########
.github/workflows/build_and_test.yml:
##########
@@ -290,6 +290,22 @@ jobs:
build-tools/helm/spark-kubernetes-operator/
minikube docker-env --unset
helm test spark
+ - name: Validate the deprecated helm key warning
+ if: matrix.test-group == 'configmap-metadata'
+ run: |
+ # `helm template` does not render NOTES.txt, so an install or upgrade against a
Review Comment:
`helm install --dry-run=client` renders `NOTES.txt` without a cluster (verified locally with `KUBECONFIG=/dev/null`), so a real cluster is not required here. Could we move this check to the `lint` job, next to `Validate deprecated helm values are still honored`? That avoids coupling it to the unrelated `configmap-metadata` group and avoids applying a NetworkPolicy and dynamicConfig RBAC changes to the live release.
```yaml
- name: Validate the deprecated helm key warning
run: |
notes=$(helm install spark build-tools/helm/spark-kubernetes-operator --dry-run=client \
--set operatorDeployment.networkPolicy.enable=true \
--set operatorConfiguration.dynamicConfig.enable=true)
echo "$notes" | grep -q 'operatorDeployment.networkPolicy.enable is deprecated'
echo "$notes" | grep -q 'operatorConfiguration.dynamicConfig.enable is deprecated'
if helm install spark build-tools/helm/spark-kubernetes-operator --dry-run=client \
| grep -q 'is deprecated'; then
echo "Deprecation warning printed without a legacy key"; exit 1
fi
```
If so, please also update the PR description's "a real cluster is the only place this is observable".
##########
build-tools/helm/spark-kubernetes-operator/templates/NOTES.txt:
##########
@@ -0,0 +1,31 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+Apache Spark Kubernetes Operator {{ .Chart.AppVersion }} is installed.
+{{- if hasKey .Values.operatorDeployment.networkPolicy "enable" }}
Review Comment:
nit: `hasKey` fails on a nil map. With `--set operatorDeployment.networkPolicy=null`, the install now fails here:
```
NOTES.txt:18:21 ... wrong type for value; expected map[string]interface {}; got interface {}
```
The current chart renders fine with the same input because `_helpers.tpl` uses `or $np.enabled $np.enable`. It is an unrealistic input, but it could be guarded with `hasKey (.Values.operatorDeployment.networkPolicy | default dict) "enable"` (and likewise for `dynamicConfig`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #830: [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt`
URL: https://github.com/apache/spark-kubernetes-operator/pull/830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt` [spark-kubernetes-operator]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on code in PR #830:
URL: https://github.com/apache/spark-kubernetes-operator/pull/830#discussion_r4027295582
##########
.github/workflows/build_and_test.yml:
##########
@@ -290,6 +290,22 @@ jobs:
build-tools/helm/spark-kubernetes-operator/
minikube docker-env --unset
helm test spark
+ - name: Validate the deprecated helm key warning
+ if: matrix.test-group == 'configmap-metadata'
+ run: |
+ # `helm template` does not render NOTES.txt, so an install or upgrade against a
Review Comment:
Moved to the `lint` job, description updated. One wrinkle: on helm 3.18.6 that command still fails with `Kubernetes cluster unreachable`, even with `KUBECONFIG=/dev/null`. The runner has Helm 4.2.4 so CI is fine, but a local run on helm 3.x will report a false failure.
##########
build-tools/helm/spark-kubernetes-operator/templates/NOTES.txt:
##########
@@ -0,0 +1,31 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+Apache Spark Kubernetes Operator {{ .Chart.AppVersion }} is installed.
+{{- if hasKey .Values.operatorDeployment.networkPolicy "enable" }}
Review Comment:
Reproduced and fixed with `| default dict` on both toggles. `--set operatorConfiguration.dynamicConfig=null` still fails, but identically on `main`: `_helpers.tpl:137` reads `.dynamicConfig.source` unguarded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59540] Warn about deprecated Helm `enable` keys via `NOTES.txt` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #830:
URL: https://github.com/apache/spark-kubernetes-operator/pull/830#issuecomment-5699392026
Thank you, @peter-toth!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org