You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org> on 2012/08/03 18:16:03 UTC
[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers
for map tasks, there-by allowing fine-grained control on num-maps for users
without need for CombineFileInputFormat etc.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428211#comment-13428211 ]
Tsuyoshi OZAWA commented on MAPREDUCE-3902:
-------------------------------------------
IMHO, the 2nd topic(combining per container) should be moved, because the change seems to be too big.
If there are no counter opinion, I'm going to create new ticket to deal with the 2nd topic as a sub-task of MAPREDUCe-3902.
> MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3902
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3902
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster, mrv2
> Reporter: Arun C Murthy
> Assignee: Siddharth Seth
> Attachments: MAPREDUCE-3902.2.patch, MAPREDUCE-3902.patch
>
>
> The MR AM is now in a great position to reuse containers across (map) tasks. This is something similar to JVM re-use we had in 0.20.x, but in a significantly better manner:
> # Consider data-locality when re-using containers
> # Consider the new shuffle - ensure that reduces fetch output of the whole container at once (i.e. all maps)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira