[jira] [Created] (TEZ-3207) Have the unordered ShuffleManager support fetching multiple partitions from the same source task

["Ming Ma (JIRA)" <ji...@apache.org>]

Ming Ma created TEZ-3207:
----------------------------

             Summary: Have the unordered ShuffleManager support fetching multiple partitions from the same source task
                 Key: TEZ-3207
                 URL: https://issues.apache.org/jira/browse/TEZ-3207
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Ming Ma


The ordered grouped {{ShuffleScheduler}} can support fetching multiple partitions from the same source task. But for the unordered ShuffleManager, it only supports one partition per source task due to the following issue where {{identifier}} doesn't take partition id into account.

{noformat}
  public void addKnownInput(String hostName, int port,
      InputAttemptIdentifier srcAttemptIdentifier, int srcPhysicalIndex) {
    String identifier = InputHost.createIdentifier(hostName, port);
    InputHost host = knownSrcHosts.get(identifier);
    ....
  }
{noformat}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)