You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/09 08:13:30 UTC
[GitHub] [spark] LuciferYang opened a new pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
LuciferYang opened a new pull request #32105:
URL: https://github.com/apache/spark/pull/32105
### What changes were proposed in this pull request?
Line 425 in `MasterSuite` is considered as unused expression by Intellij IDE,
https://github.com/apache/spark/blob/bfba7fadd2e65c853971fb2983bdea1c52d1ed7f/core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala#L421-L426
If we merge lines 424 and 425 into one as:
```
System.getProperty("spark.ui.proxyBase") should startWith (s"$reverseProxyUrl/proxy/worker-")
```
this assertion will fail:
```
- master/worker web ui available behind front-end reverseProxy *** FAILED ***
The code passed to eventually never returned normally. Attempted 45 times over 5.091914027 seconds. Last failure message: "http://proxyhost:8080/path/to/spark" did not start with substring "http://proxyhost:8080/path/to/spark/proxy/worker-". (MasterSuite.scala:405)
```
`System.getProperty("spark.ui.proxyBase")` should be `reverseProxyUrl` because `Master#onStart` and `Worker#handleRegisterResponse` have not changed it.
So the main purpose of this pr is to fix the condition of this assertion.
### Why are the changes needed?
Bug fix.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816611338
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137122/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816521001
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816613117
**[Test build #137132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137132/testReport)** for PR 32105 at commit [`0c77a48`](https://github.com/apache/spark/commit/0c77a4849bff7312ea05199e792cd2a44381c788).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816645311
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41710/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816682288
thx~ @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816599023
**[Test build #137122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137122/testReport)** for PR 32105 at commit [`9f69eb2`](https://github.com/apache/spark/commit/9f69eb2ef843c57d5bdbc4dd6b4775388472d19a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816715772
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137132/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816645274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #32105:
URL: https://github.com/apache/spark/pull/32105#discussion_r610490122
##########
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##########
@@ -421,8 +421,7 @@ class MasterSuite extends SparkFunSuite
// with LocalCluster, we have masters and workers in the same JVM, each overwriting
// system property spark.ui.proxyBase.
// so we need to manage this property explicitly for test
- System.getProperty("spark.ui.proxyBase") should startWith
- (s"$reverseProxyUrl/proxy/worker-")
+ System.getProperty("spark.ui.proxyBase") should startWith (reverseProxyUrl)
Review comment:
Thanks for the fix!
Then we should remove line 421~423, and line 425 too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816545704
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41700/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816675245
Thanks, merging to master/3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on a change in pull request #32105:
URL: https://github.com/apache/spark/pull/32105#discussion_r610518472
##########
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##########
@@ -421,8 +421,7 @@ class MasterSuite extends SparkFunSuite
// with LocalCluster, we have masters and workers in the same JVM, each overwriting
// system property spark.ui.proxyBase.
// so we need to manage this property explicitly for test
- System.getProperty("spark.ui.proxyBase") should startWith
- (s"$reverseProxyUrl/proxy/worker-")
+ System.getProperty("spark.ui.proxyBase") should startWith (reverseProxyUrl)
Review comment:
ok ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816509899
**[Test build #137122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137122/testReport)** for PR 32105 at commit [`9f69eb2`](https://github.com/apache/spark/commit/9f69eb2ef843c57d5bdbc4dd6b4775388472d19a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816645311
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41710/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816545284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816707475
**[Test build #137132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137132/testReport)** for PR 32105 at commit [`0c77a48`](https://github.com/apache/spark/commit/0c77a4849bff7312ea05199e792cd2a44381c788).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816715772
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137132/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816611338
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137122/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816521001
cc @gengliangwang, I revert code to SPARK-20044, `Manual test` have same result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816613117
**[Test build #137132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137132/testReport)** for PR 32105 at commit [`0c77a48`](https://github.com/apache/spark/commit/0c77a4849bff7312ea05199e792cd2a44381c788).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816545704
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41700/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32105:
URL: https://github.com/apache/spark/pull/32105#issuecomment-816509899
**[Test build #137122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137122/testReport)** for PR 32105 at commit [`9f69eb2`](https://github.com/apache/spark/commit/9f69eb2ef843c57d5bdbc4dd6b4775388472d19a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on a change in pull request #32105:
URL: https://github.com/apache/spark/pull/32105#discussion_r610530408
##########
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##########
@@ -421,8 +421,7 @@ class MasterSuite extends SparkFunSuite
// with LocalCluster, we have masters and workers in the same JVM, each overwriting
// system property spark.ui.proxyBase.
// so we need to manage this property explicitly for test
- System.getProperty("spark.ui.proxyBase") should startWith
- (s"$reverseProxyUrl/proxy/worker-")
+ System.getProperty("spark.ui.proxyBase") should startWith (reverseProxyUrl)
Review comment:
513fc3e cleanup these lines and 0c77a48 change `startWith` to `be`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #32105: [SPARK-35004][TEST] Fix Incorrect assertion of `master/worker web ui available behind front-end reverseProxy` in MasterSuite
Posted by GitBox <gi...@apache.org>.
gengliangwang closed pull request #32105:
URL: https://github.com/apache/spark/pull/32105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org