You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org> on 2012/08/01 08:46:33 UTC
[jira] [Created] (MAPREDUCE-4502) Multi-level aggregation with
combining the result of maps per node/rack
Tsuyoshi OZAWA created MAPREDUCE-4502:
-----------------------------------------
Summary: Multi-level aggregation with combining the result of maps per node/rack
Key: MAPREDUCE-4502
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4502
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: applicationmaster, mrv2
Reporter: Tsuyoshi OZAWA
The shuffle costs is expensive in Hadoop in spite of the
existence of combiner, because the scope of combining is limited
within only one MapTask. To solve this problem, it's a good way to aggregate the result of maps per node/rack by launch combiner.
This JIRA is to implement the multi-level aggregation infrastructure, including combining per container(MAPREDUCE-3902 is related), coordinating containers by application master without breaking fault tolerance of jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira