You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "jinhai (Jira)" <ji...@apache.org> on 2021/10/20 06:18:00 UTC

[jira] [Resolved] (SPARK-37029) Modify the assignment logic of dirFetchRequests variables

     [ https://issues.apache.org/jira/browse/SPARK-37029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

jinhai resolved SPARK-37029.
----------------------------
    Resolution: Won't Fix

> Modify the assignment logic of dirFetchRequests variables
> ---------------------------------------------------------
>
>                 Key: SPARK-37029
>                 URL: https://issues.apache.org/jira/browse/SPARK-37029
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle
>    Affects Versions: 3.1.2, 3.2.0
>            Reporter: jinhai
>            Priority: Major
>
> In the ShuffleBlockFetcherIterator.fetchHostLocalBlocks method, we generate dirFetchRequests based on externalShuffleServiceEnabled. But in fact, the MapStatus object generated in the shuffle write phase had already generated the BlockManagerId object according to externalShuffleServiceEnabled in the BlockManager.initialize method.
> So we don't need to judge it again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org