You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/12/26 13:17:11 UTC
[GitHub] [spark] ajithme opened a new pull request #27018: [SPARK-30361]
Monitoring URL do not redact information about environment
ajithme opened a new pull request #27018: [SPARK-30361] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018
UI and event logs redact sensitive information. But the monitoring URL, https://spark.apache.org/docs/latest/monitoring.html#rest-api , specifically /applications/[app-id]/environment does not, which is a security issue.
### What changes were proposed in this pull request?
REST api response is redacted before sending it
### Why are the changes needed?
If no redaction is done for rest API call, it can leak security information
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Tested manually
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#discussion_r361586501
##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -97,7 +99,15 @@ private[v1] class AbstractApplicationResource extends BaseAppResource {
@GET
@Path("environment")
- def environmentInfo(): ApplicationEnvironmentInfo = withUI(_.store.environmentInfo())
+ def environmentInfo(): ApplicationEnvironmentInfo = withUI { ui =>
+ val envInfo = ui.store.environmentInfo()
+ new v1.ApplicationEnvironmentInfo(
+ envInfo.runtime,
+ Utils.redact(ui.conf, envInfo.sparkProperties),
+ Utils.redact(ui.conf, envInfo.hadoopProperties),
+ Utils.redact(ui.conf, envInfo.systemProperties),
+ Utils.redact(ui.conf, envInfo.classpathEntries))
Review comment:
jar name containing key words like password, token etc.. i have updated the PR to skip redacting classpath.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569337570
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569314102
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20662/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569337573
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115870/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569287070
**[Test build #4978 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4978/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#discussion_r361567478
##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -97,7 +99,15 @@ private[v1] class AbstractApplicationResource extends BaseAppResource {
@GET
@Path("environment")
- def environmentInfo(): ApplicationEnvironmentInfo = withUI(_.store.environmentInfo())
+ def environmentInfo(): ApplicationEnvironmentInfo = withUI { ui =>
+ val envInfo = ui.store.environmentInfo()
+ new v1.ApplicationEnvironmentInfo(
+ envInfo.runtime,
+ Utils.redact(ui.conf, envInfo.sparkProperties),
+ Utils.redact(ui.conf, envInfo.hadoopProperties),
+ Utils.redact(ui.conf, envInfo.systemProperties),
+ Utils.redact(ui.conf, envInfo.classpathEntries))
Review comment:
+1. I'm not sure what's the point of applying redact to classpath entries, and why exposing classpath entries without redacting is considered as security issue. Moreover I'm not sure what would be matched to redact patterns from classpath entries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569612183
Gentle ping @vanzin @srowen @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018: [SPARK-30361]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569057995
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569314097
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569313023
Jenkins retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569313808
**[Test build #115870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115870/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569287070
**[Test build #4978 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4978/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569704868
Merged to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme edited a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
ajithme edited a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569305602
> **[Test build #4978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4978/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
>
> * This patch **fails Spark unit tests**.
> * This patch merges cleanly.
> * This patch adds no public classes.
test failures do not look related to this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018: [SPARK-30361]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569057803
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569314097
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361] Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569057803
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569337256
**[Test build #115870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115870/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569305663
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569313808
**[Test build #115870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115870/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569314102
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20662/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27018: [SPARK-30361] Monitoring
URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27018: [SPARK-30361] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569057158
cc @dongjoon-hyun @srowen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#discussion_r361475318
##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -97,7 +99,15 @@ private[v1] class AbstractApplicationResource extends BaseAppResource {
@GET
@Path("environment")
- def environmentInfo(): ApplicationEnvironmentInfo = withUI(_.store.environmentInfo())
+ def environmentInfo(): ApplicationEnvironmentInfo = withUI { ui =>
+ val envInfo = ui.store.environmentInfo()
+ new v1.ApplicationEnvironmentInfo(
+ envInfo.runtime,
+ Utils.redact(ui.conf, envInfo.sparkProperties),
+ Utils.redact(ui.conf, envInfo.hadoopProperties),
+ Utils.redact(ui.conf, envInfo.systemProperties),
+ Utils.redact(ui.conf, envInfo.classpathEntries))
Review comment:
As in the other PR, I don't know if we need to redact the classpath?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569305093
**[Test build #4978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4978/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#discussion_r361586501
##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -97,7 +99,15 @@ private[v1] class AbstractApplicationResource extends BaseAppResource {
@GET
@Path("environment")
- def environmentInfo(): ApplicationEnvironmentInfo = withUI(_.store.environmentInfo())
+ def environmentInfo(): ApplicationEnvironmentInfo = withUI { ui =>
+ val envInfo = ui.store.environmentInfo()
+ new v1.ApplicationEnvironmentInfo(
+ envInfo.runtime,
+ Utils.redact(ui.conf, envInfo.sparkProperties),
+ Utils.redact(ui.conf, envInfo.hadoopProperties),
+ Utils.redact(ui.conf, envInfo.systemProperties),
+ Utils.redact(ui.conf, envInfo.classpathEntries))
Review comment:
i have updated the PR to skip redacting classpath.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569337570
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569057995
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27018:
[SPARK-30361][REST] Monitoring URL do not redact information about
environment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569337573
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115870/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27018: [SPARK-30361][REST]
Monitoring URL do not redact information about environment
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27018: [SPARK-30361][REST] Monitoring URL do not redact information about environment
URL: https://github.com/apache/spark/pull/27018#issuecomment-569305602
retest this please
> **[Test build #4978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4978/testReport)** for PR 27018 at commit [`6a9ea54`](https://github.com/apache/spark/commit/6a9ea54f7dc54c6dacb5b365f6c76812df556a5b).
>
> * This patch **fails Spark unit tests**.
> * This patch merges cleanly.
> * This patch adds no public classes.
test failures do not look related to this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org