You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2017/07/25 14:31:00 UTC

[jira] [Created] (SPARK-21530) Update description of spark.shuffle.maxChunksBeingTransferred

Thomas Graves created SPARK-21530:
-------------------------------------

             Summary: Update description of spark.shuffle.maxChunksBeingTransferred
                 Key: SPARK-21530
                 URL: https://issues.apache.org/jira/browse/SPARK-21530
             Project: Spark
          Issue Type: Improvement
          Components: Shuffle
    Affects Versions: 2.3.0
            Reporter: Thomas Graves


We should expand the description of spark.shuffle.maxChunksBeingTransferred to include what happens when the max is hit. In this case it actually closes incoming connections so if the retry logic on the reducer doesn't happen could end up in failure.  

The way its currently worded doesn't tell users bad things can happen when the max is hit.

introduced with https://github.com/apache/spark/pull/18388/files

SPARK-21175



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org