[jira] [Commented] (SPARK-32922) Add support for ShuffleBlockFetcherIterator to read from merged shuffle partitions and to fallback to original shuffle blocks if encountering failures

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-32922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17319865#comment-17319865 ] 

Apache Spark commented on SPARK-32922:
--------------------------------------

User 'otterc' has created a pull request for this issue:
https://github.com/apache/spark/pull/32140

> Add support for ShuffleBlockFetcherIterator to read from merged shuffle partitions and to fallback to original shuffle blocks if encountering failures
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-32922
>                 URL: https://issues.apache.org/jira/browse/SPARK-32922
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Shuffle, Spark Core
>    Affects Versions: 3.1.0
>            Reporter: Min Shen
>            Priority: Major
>
> With the extended MapOutputTracker, the reducers can now get the task input data from the merged shuffle partitions for more efficient shuffle data fetch. The reducers should also be able to fallback to fetching the original unmarked blocks if it encounters failures when fetching the merged shuffle partitions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org