You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "jin xing (JIRA)" <ji...@apache.org> on 2017/07/25 15:47:00 UTC
[jira] [Commented] (SPARK-21530) Update description of
spark.shuffle.maxChunksBeingTransferred
[ https://issues.apache.org/jira/browse/SPARK-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100236#comment-16100236 ]
jin xing commented on SPARK-21530:
----------------------------------
I will send follow-up PR soon.
Thanks [~tgraves]
> Update description of spark.shuffle.maxChunksBeingTransferred
> -------------------------------------------------------------
>
> Key: SPARK-21530
> URL: https://issues.apache.org/jira/browse/SPARK-21530
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle
> Affects Versions: 2.3.0
> Reporter: Thomas Graves
>
> We should expand the description of spark.shuffle.maxChunksBeingTransferred to include what happens when the max is hit. In this case it actually closes incoming connections so if the retry logic on the reducer doesn't happen could end up in failure.
> The way its currently worded doesn't tell users bad things can happen when the max is hit.
> introduced with https://github.com/apache/spark/pull/18388/files
> SPARK-21175
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org