You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "jinhai (Jira)" <ji...@apache.org> on 2021/10/17 09:46:00 UTC

[jira] [Created] (SPARK-37029) Modify the assignment logic of dirFetchRequests variables

jinhai created SPARK-37029:
------------------------------

             Summary: Modify the assignment logic of dirFetchRequests variables
                 Key: SPARK-37029
                 URL: https://issues.apache.org/jira/browse/SPARK-37029
             Project: Spark
          Issue Type: Improvement
          Components: Shuffle
    Affects Versions: 3.2.0, 3.1.2
            Reporter: jinhai


In the ShuffleBlockFetcherIterator.fetchHostLocalBlocks method, we generate dirFetchRequests based on externalShuffleServiceEnabled. But in fact, the MapStatus object generated in the shuffle write phase had already generated the BlockManagerId object according to externalShuffleServiceEnabled in the BlockManager.initialize method.

So we don't need to judge it again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org