You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/12/26 12:56:45 UTC

[GitHub] [spark] ajithme opened a new pull request #27016: [SPARK-30360] Redact classpath entries in Spark UI

ajithme opened a new pull request #27016: [SPARK-30360] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016
 
 
   Currently SPARK history server display the classpath entries in the Environment tab with classpaths redacted, this is because EventLog file has the entry values redacted while writing. But when same is seen from a running application UI, its seen that it is not redacted.
   
   ### What changes were proposed in this pull request?
   EnvironmentPage can redact the classpath entries before display.
   
   
   ### Why are the changes needed?
   Redaction is needed from security view and to keep behaviour consistent between history server and application UI
   
   
   ### Does this PR introduce any user-facing change?
   NO
   
   
   ### How was this patch tested?
   Tested manually to verify on UI
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569186064
 
 
   Ah there's token-provider... OK that's not really no-op. Personally it seems better to exclude redact for system classpath, but that's just 2 cents and I'd hear about @srowen voice on this, as it might be trivial and no big deal on tolerating.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569172221
 
 
   +1 on @srowen . The thing is, redacting classpath would be probably no-op. If we could imagine some case classpath entries could be redacted accidentally, then maybe yes it's worth excluding, but if I get correctly, there's such case being reported.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme edited a comment on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme edited a comment on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569094610
 
 
   Keep it consistent with UI and eventlogs, as i traced it was done as part of event log and UI redact initial commit
   https://github.com/apache/spark/commit/237c3b9642a1a7c5e7884824b21877590d5d0b3b
   
   Another way would be not redacting classpath entries in eventlogs and UI

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569095184
 
 
   agree, so should the event log redact for classpath entries be avoided.?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298520
 
 
   **[Test build #115867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115867/testReport)** for PR 27016 at commit [`6acb93b`](https://github.com/apache/spark/commit/6acb93b8c1efc67b9c006f39965ba20489fa3564).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569073710
 
 
   What about the classpath would need to be redacted?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298855
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569096134
 
 
   Okay, ill update the PR to drop classpath redact from UI and eventlogs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569055211
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569324776
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298855
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen closed pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
srowen closed pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569185418
 
 
   Currently with master installer and default configuration, this is the behaviour in Spark UI and History server UI
   
   SparkUI:
   ![image](https://user-images.githubusercontent.com/22072336/71501372-276bcf80-2890-11ea-83a8-d696b3c71472.png)
   
   
   Spark History Server:
   ![image](https://user-images.githubusercontent.com/22072336/71501406-600ba900-2890-11ea-9901-a0db98f37098.png)
   
   So is this default redact of classpath only in history server UI acceptable.? @srowen @HeartSaVioR 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569094756
 
 
   But what would it even need to redact? I don't think consistency itself is an argument, because the classpath doesn't have user-supplied secrets on it. Right?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569324443
 
 
   **[Test build #115867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115867/testReport)** for PR 27016 at commit [`6acb93b`](https://github.com/apache/spark/commit/6acb93b8c1efc67b9c006f39965ba20489fa3564).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569055378
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298858
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20659/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569324779
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115867/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569100902
 
 
   Here https://github.com/apache/spark/blob/8d3eed33eee0d6e5d2e4d4907869ca6226d9e37f/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L258

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298520
 
 
   **[Test build #115867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115867/testReport)** for PR 27016 at commit [`6acb93b`](https://github.com/apache/spark/commit/6acb93b8c1efc67b9c006f39965ba20489fa3564).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on a change in pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#discussion_r361675571
 
 

 ##########
 File path: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala
 ##########
 @@ -255,9 +255,14 @@ private[spark] class EventLoggingListener(
     // ...
     // where jvmInformation, sparkProperties, etc. are sequence of tuples.
     // We go through the various  of properties and redact sensitive information from them.
-    val redactedProps = event.environmentDetails.map{ case (name, props) =>
-      name -> Utils.redact(sparkConf, props)
-    }
+    val noRedactProps = Seq("Classpath Entries")
+    val redactedProps = event.environmentDetails
+      .map {
 
 Review comment:
   nit: This can be done like below, as they fit in one-line per each:
   
   ```
   val redactedProps = event.environmentDetails.map {
     case (name, props) if noRedactProps.contains(name) => name -> props
     case (name, props) => name -> Utils.redact(sparkConf, props)
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569298858
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20659/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569280368
 
 
   **[Test build #4977 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4977/testReport)** for PR 27016 at commit [`a65a6fb`](https://github.com/apache/spark/commit/a65a6fbff4de793982dbcd0ad38444fa57af8894).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569193780
 
 
   Agree. @srowen @HeartSaVioR i have updated the PR to avoid redacting classpath entries. Please let me know if you have a different opinion.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569055211
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569095907
 
 
   Right, which is why I'm wondering why this change is needed

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569324779
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115867/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on a change in pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#discussion_r361681151
 
 

 ##########
 File path: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala
 ##########
 @@ -255,9 +255,14 @@ private[spark] class EventLoggingListener(
     // ...
     // where jvmInformation, sparkProperties, etc. are sequence of tuples.
     // We go through the various  of properties and redact sensitive information from them.
-    val redactedProps = event.environmentDetails.map{ case (name, props) =>
-      name -> Utils.redact(sparkConf, props)
-    }
+    val noRedactProps = Seq("Classpath Entries")
+    val redactedProps = event.environmentDetails
+      .map {
 
 Review comment:
   Done. Modified as per the suggestion

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569442784
 
 
   Merged to master

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569055378
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569094610
 
 
   Keep it consistent with UI and eventlogs, as i traced it was done as part of event log and UI redact initial commit.. another way would be not redacting classpath entries in eventlogs and UI

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569297277
 
 
   Jenkins retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] ajithme commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27016: [SPARK-30360] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569054184
 
 
   cc @dongjoon-hyun @srowen 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569295873
 
 
   **[Test build #4977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4977/testReport)** for PR 27016 at commit [`a65a6fb`](https://github.com/apache/spark/commit/a65a6fbff4de793982dbcd0ad38444fa57af8894).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569280368
 
 
   **[Test build #4977 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4977/testReport)** for PR 27016 at commit [`a65a6fb`](https://github.com/apache/spark/commit/a65a6fbff4de793982dbcd0ad38444fa57af8894).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569107418
 
 
   Yes that happens to redact the classpath entries, which isn't necessary it seems, but I don't think it's worth excluding specifically. I just don't think it's worth explicitly including elsewhere, as it shouldn't make any difference

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27016: [SPARK-30360][UI] Redact classpath entries in Spark UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569099041
 
 
   Maybe I'm missing it, but I don't see it currently explicitly redacted anywhere, including the event log listener.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27016: [SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
URL: https://github.com/apache/spark/pull/27016#issuecomment-569324776
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org