You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/09/18 20:23:51 UTC
(spark) branch branch-4.3 updated: [SPARK-59641][K8S] Redact `spark.exit-exception` annotation with `spark.redaction.string.regex`
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.3 by this push:
new cde8720ff402 [SPARK-59641][K8S] Redact `spark.exit-exception` annotation with `spark.redaction.string.regex`
cde8720ff402 is described below
commit cde8720ff402b3dfca5f2758be0f108a68a3416b
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Fri Sep 18 13:23:22 2026 -0700
[SPARK-59641][K8S] Redact `spark.exit-exception` annotation with `spark.redaction.string.regex`
### What changes were proposed in this pull request?
This PR aims to redact the exit exception annotation, `spark.exit-exception`, with `spark.redaction.string.regex` when `spark.kubernetes.driver.annotateExitException` is enabled.
In addition, this PR documents the following.
- The annotation is visible to anyone who can get the driver pod.
- `spark.redaction.string.regex` is used for the exit exception annotation on Kubernetes.
### Why are the changes needed?
`spark.kubernetes.driver.annotateExitException` (SPARK-53335) stores the stringified exit exception into the driver pod annotation without any redaction. Since an exception message may contain sensitive information like JDBC URLs, it is visible to anyone who can get the driver pod.
`spark.redaction.string.regex` is the existing configuration to redact the strings produced by Spark, like the output of SQL explain commands. This PR reuses it so that users can hide the sensitive parts of the exit exception without a new configuration.
### Does this PR introduce _any_ user-facing change?
No behavior change by default because `spark.redaction.string.regex` has no default value. If a user sets `spark.redaction.string.regex`, the matching parts of the `spark.exit-exception` annotation are replaced with `*********(redacted)`.
### How was this patch tested?
Pass the CIs with the newly added test case.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
Closes #58911 from dongjoon-hyun/SPARK-59641.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 6aa2688a21e3fbc9db9abc2a66e47a951fa94702)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../org/apache/spark/internal/config/package.scala | 3 ++-
docs/configuration.md | 2 +-
docs/running-on-kubernetes.md | 2 ++
.../scala/org/apache/spark/deploy/k8s/Config.scala | 4 +++-
.../deploy/k8s/SparkKubernetesDiagnosticsSetter.scala | 4 +++-
.../k8s/SparkKubernetesDiagnosticsSetterSuite.scala | 19 +++++++++++++++++++
6 files changed, 30 insertions(+), 4 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/internal/config/package.scala b/core/src/main/scala/org/apache/spark/internal/config/package.scala
index d18be6a060a2..1e02e29896ef 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/package.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/package.scala
@@ -1336,7 +1336,8 @@ package object config {
ConfigBuilder("spark.redaction.string.regex")
.doc("Regex to decide which parts of strings produced by Spark contain sensitive " +
"information. When this regex matches a string part, that string part is replaced by a " +
- "dummy value. This is currently used to redact the output of SQL explain commands.")
+ "dummy value. This is currently used to redact the output of SQL explain commands and " +
+ "the exit exception annotation on Kubernetes.")
.version("2.2.0")
.regexConf
.createOptional
diff --git a/docs/configuration.md b/docs/configuration.md
index da29b9924043..965507ef8734 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -827,7 +827,7 @@ Apart from these, the following properties are also available, and may be useful
<td>
Regex to decide which parts of strings produced by Spark contain sensitive information.
When this regex matches a string part, that string part is replaced by a dummy value.
- This is currently used to redact the output of SQL explain commands.
+ This is currently used to redact the output of SQL explain commands and the exit exception annotation on Kubernetes.
</td>
<td>2.2.0</td>
</tr>
diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md
index 4e15a45e403d..ec613958df04 100644
--- a/docs/running-on-kubernetes.md
+++ b/docs/running-on-kubernetes.md
@@ -1714,6 +1714,8 @@ See the [configuration page](configuration.html) for information on Spark config
<td><code>false</code></td>
<td>
If set to true, Spark will store the exit exception failed applications in the Kubernetes API server using the <code>spark.exit-exception</code> annotation.
+ Note that the annotation is visible to anyone who can get the driver pod. The parts of the exit exception matching
+ <code>spark.redaction.string.regex</code> are redacted.
</td>
<td>4.1.0</td>
</tr>
diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
index 7fd21dfdcb0f..60d61028476f 100644
--- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
+++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
@@ -919,7 +919,9 @@ private[spark] object Config extends Logging {
val KUBERNETES_ANNOTATE_EXIT_EXCEPTION =
ConfigBuilder("spark.kubernetes.driver.annotateExitException")
.doc("If set to true, Spark will store the exit exception failed applications in" +
- s" the Kubernetes API server using the $EXIT_EXCEPTION_ANNOTATION annotation.")
+ s" the Kubernetes API server using the $EXIT_EXCEPTION_ANNOTATION annotation. Note that" +
+ " the annotation is visible to anyone who can get the driver pod. The parts of the exit" +
+ " exception matching `spark.redaction.string.regex` are redacted.")
.version("4.1.0")
.booleanConf
.createWithDefault(false)
diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetter.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetter.scala
index 5717a8480917..57cc23da9198 100644
--- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetter.scala
+++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetter.scala
@@ -26,6 +26,7 @@ import org.apache.spark.deploy.k8s.Config._
import org.apache.spark.deploy.k8s.Constants.EXIT_EXCEPTION_ANNOTATION
import org.apache.spark.deploy.k8s.SparkKubernetesClientFactory.ClientType
import org.apache.spark.internal.Logging
+import org.apache.spark.internal.config.STRING_REDACTION_PATTERN
import org.apache.spark.util.{SparkStringUtils, Utils}
/**
@@ -59,7 +60,8 @@ private[spark] class SparkKubernetesDiagnosticsSetter(clientProvider: Kubernetes
}
override def setDiagnostics(throwable: Throwable, conf: SparkConf): Unit = {
- val diagnostics = SparkStringUtils.abbreviate(Utils.stringifyException(throwable),
+ val diagnostics = SparkStringUtils.abbreviate(
+ Utils.redact(conf.get(STRING_REDACTION_PATTERN), Utils.stringifyException(throwable)),
KUBERNETES_EXIT_EXCEPTION_MESSAGE_LIMIT_BYTES)
Utils.tryWithResource(clientProvider.create(conf)) { client =>
conf.get(KUBERNETES_DRIVER_POD_NAME).foreach { podName =>
diff --git a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetterSuite.scala b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetterSuite.scala
index 5a00b6f6b74e..14aae74adf3a 100644
--- a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetterSuite.scala
+++ b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SparkKubernetesDiagnosticsSetterSuite.scala
@@ -30,6 +30,7 @@ import org.apache.spark.{SparkConf, SparkFunSuite}
import org.apache.spark.deploy.k8s.Config._
import org.apache.spark.deploy.k8s.Constants.EXIT_EXCEPTION_ANNOTATION
import org.apache.spark.deploy.k8s.Fabric8Aliases.PODS
+import org.apache.spark.internal.config.STRING_REDACTION_PATTERN
import org.apache.spark.util.Utils
class SparkKubernetesDiagnosticsSetterSuite extends SparkFunSuite
@@ -80,4 +81,22 @@ class SparkKubernetesDiagnosticsSetterSuite extends SparkFunSuite
assert(podCaptor.getValue.getMetadata.getAnnotations.get(EXIT_EXCEPTION_ANNOTATION)
== Utils.stringifyException(diagnostics))
}
+
+ test("setDiagnostics should redact diagnostics with spark.redaction.string.regex") {
+ val diagnostics = new Throwable("Failed to connect to jdbc:db://host?password=mySecret")
+ val conf = new SparkConf()
+ .set(KUBERNETES_DRIVER_MASTER_URL, k8sClusterManagerUrl)
+ .set(KUBERNETES_NAMESPACE, namespace)
+ .set(KUBERNETES_DRIVER_POD_NAME, driverPodName)
+ .set(STRING_REDACTION_PATTERN.key, "password=[^\\s]*")
+
+ setter.setDiagnostics(diagnostics, conf)
+
+ val podCaptor: ArgumentCaptor[Pod] = ArgumentCaptor.forClass(classOf[Pod])
+ verify(driverPodOperations).patch(any(classOf[PatchContext]), podCaptor.capture())
+
+ val annotation = podCaptor.getValue.getMetadata.getAnnotations.get(EXIT_EXCEPTION_ANNOTATION)
+ assert(!annotation.contains("mySecret"))
+ assert(annotation.contains(Utils.REDACTION_REPLACEMENT_TEXT))
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org