[jira] [Updated] (TEZ-3207) Add support for fetching multiple partitions from the same source task to UnorderedKVInput

["Ming Ma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ming Ma updated TEZ-3207:
-------------------------
    Summary: Add support for fetching multiple partitions from the same source task to UnorderedKVInput  (was: Have the unordered ShuffleManager support fetching multiple partitions from the same source task)

> Add support for fetching multiple partitions from the same source task to UnorderedKVInput
> ------------------------------------------------------------------------------------------
>
>                 Key: TEZ-3207
>                 URL: https://issues.apache.org/jira/browse/TEZ-3207
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Ming Ma
>
> The ordered grouped {{ShuffleScheduler}} can support fetching multiple partitions from the same source task. But for the unordered ShuffleManager, it only supports one partition per source task due to the following issue where {{identifier}} doesn't take partition id into account.
> {noformat}
>   public void addKnownInput(String hostName, int port,
>       InputAttemptIdentifier srcAttemptIdentifier, int srcPhysicalIndex) {
>     String identifier = InputHost.createIdentifier(hostName, port);
>     InputHost host = knownSrcHosts.get(identifier);
>     ....
>   }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)