You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/12/18 18:02:13 UTC
[GitHub] [spark] yaooqinn opened a new pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
yaooqinn opened a new pull request #30844:
URL: https://github.com/apache/spark/pull/30844
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
We have `spark.shuffle.io.connectionTimeout` for timeout checking when shuffle clients establish connections with shuffle servers. We also use it to check the established connection is idled or dead too. If we want a connection that can keep alive for a long time, we may increase the blocking time to create a shuffle client if the server is busy at that time.
In practice, we always use connection timeout and session timeout to control these two different phases.
This PR adds `spark.shuffle.io.sessionTimeout `to check an established connection for being idled or dead
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
yes, w/ new conf spark.shuffle.io.sessionTimeout ` to check an established connection for being idled or dead
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
modified tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748269641
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37628/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748267091
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37628/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748275039
**[Test build #133033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133033/testReport)** for PR 30844 at commit [`e51904f`](https://github.com/apache/spark/commit/e51904fc05c4a97966761b7bb4f976580a1db9ab).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748248266
**[Test build #133029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133029/testReport)** for PR 30844 at commit [`9f9f2f6`](https://github.com/apache/spark/commit/9f9f2f60ff774db33c83ce82559a258e07e76a6a).
* This patch **fails Java style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748500997
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133077/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748242638
**[Test build #133029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133029/testReport)** for PR 30844 at commit [`9f9f2f6`](https://github.com/apache/spark/commit/9f9f2f60ff774db33c83ce82559a258e07e76a6a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748514308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37677/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748346568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133033/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] otterc commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
otterc commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748520144
@yaooqinn
In https://github.com/apache/spark/pull/30312, we have introduced a `connectionCreationTimeout` and I think the motivation for that change and this one is the same. So, we should decide which implementation, related with this config, we would want go ahead with.
cc @mridulm @Ngone51 @Victsm @tgravescs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748312286
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748275039
**[Test build #133033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133033/testReport)** for PR 30844 at commit [`e51904f`](https://github.com/apache/spark/commit/e51904fc05c4a97966761b7bb4f976580a1db9ab).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748248295
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748273705
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37628/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748514308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37677/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748248295
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133029/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] otterc commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
otterc commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748746063
> IMHO, `connectionCreationTimeout` seems to be not a common usage. But `sessionTimeout` and `connectionTimeout` are always used as a pair in RPC environments.
Just stating the reason for why we went with creating `connectionCreationTimeout` instead of `idle/sessionTimeout`.
We wanted to reduce the timeout for creating connections. Ideally this should be much lower and for our clusters we have set this to 30s. However, the default value for `connectionTimeout` is high which makes sense because it is also used as idleTimeout and since shuffle services can be overloaded, they may take longer to respond. The problem with creating a separate config for `idle/sessionTimeout` and **not** for `connectionCreation` is that some users (frameworks) may have also over-written the value of `connectionTimeout` to a much higher value. So, even if want to enforce all these jobs to use smaller timeouts for creating connections, they will still be using a larger timeouts for that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748323909
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748434468
+CC @otterc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748346568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133033/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748299482
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748500997
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133077/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748338963
**[Test build #133033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133033/testReport)** for PR 30844 at commit [`e51904f`](https://github.com/apache/spark/commit/e51904fc05c4a97966761b7bb4f976580a1db9ab).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748242638
**[Test build #133029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133029/testReport)** for PR 30844 at commit [`9f9f2f6`](https://github.com/apache/spark/commit/9f9f2f60ff774db33c83ce82559a258e07e76a6a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748446189
cc @cloud-fan @maropu @HyukjinKwon too thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748323909
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn closed pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #30844:
URL: https://github.com/apache/spark/pull/30844
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #30844: [SPARK-33842][Core][Shuffle] Add spark.shuffle.io.sessionTimeout to check an established connection for being idled or dead
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #30844:
URL: https://github.com/apache/spark/pull/30844#issuecomment-748728489
Oh, sorry @otterc, I didn't notice #30312.
IMHO, `connectionCreationTimeout` seems to be not a common usage. But `sessionTimeout` and `connectionTimeout` are always used as a pair in RPC environments.
#30312 seems will get merged soon. I am fine w/ that too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org