[spark] branch branch-2.4 updated: [SPARK-27244][CORE][TEST][FOLLOWUP] toDebugString redacts sensitive information

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-2.4 by this push:
     new 66dfece  [SPARK-27244][CORE][TEST][FOLLOWUP] toDebugString redacts sensitive information
66dfece is described below

commit 66dfece03ca93452fbba980a0e5170cfff3910c6
Author: gatorsmile <ga...@gmail.com>
AuthorDate: Sat Mar 30 22:58:28 2019 -0700

    [SPARK-27244][CORE][TEST][FOLLOWUP] toDebugString redacts sensitive information
    
    ## What changes were proposed in this pull request?
    This PR is a FollowUp of https://github.com/apache/spark/pull/24196. It improves the test case by using the parameters that are being used in the actual scenarios.
    
    ## How was this patch tested?
    N/A
    
    Closes #24257 from gatorsmile/followupSPARK-27244.
    
    Authored-by: gatorsmile <ga...@gmail.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
    (cherry picked from commit 92b6f86f6d25abbc2abbf374e77c0b70cd1779c7)
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../test/scala/org/apache/spark/SparkConfSuite.scala  | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/core/src/test/scala/org/apache/spark/SparkConfSuite.scala b/core/src/test/scala/org/apache/spark/SparkConfSuite.scala
index 21424bd..dcbfc93 100644
--- a/core/src/test/scala/org/apache/spark/SparkConfSuite.scala
+++ b/core/src/test/scala/org/apache/spark/SparkConfSuite.scala
@@ -339,12 +339,19 @@ class SparkConfSuite extends SparkFunSuite with LocalSparkContext with ResetSyst
     }
   }
 
-  test("SPARK-27244 toDebugString should redact passwords") {
-    val conf = new SparkConf().set("dummy.password", "dummy-password")
-    conf.validateSettings()
-
-    assert(conf.get("dummy.password") === "dummy-password")
-    assert(conf.toDebugString.contains(s"dummy.password=${Utils.REDACTION_REPLACEMENT_TEXT}"))
+  test("SPARK-27244 toDebugString redacts sensitive information") {
+    val conf = new SparkConf(loadDefaults = false)
+      .set("dummy.password", "dummy-password")
+      .set("spark.hadoop.hive.server2.keystore.password", "1234")
+      .set("spark.hadoop.javax.jdo.option.ConnectionPassword", "1234")
+      .set("spark.regular.property", "regular_value")
+    assert(conf.toDebugString ==
+      s"""
+        |dummy.password=${Utils.REDACTION_REPLACEMENT_TEXT}
+        |spark.hadoop.hive.server2.keystore.password=${Utils.REDACTION_REPLACEMENT_TEXT}
+        |spark.hadoop.javax.jdo.option.ConnectionPassword=${Utils.REDACTION_REPLACEMENT_TEXT}
+        |spark.regular.property=regular_value
+      """.stripMargin.trim)
   }
 
   val defaultIllegalValue = "SomeIllegalValue"


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org