You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2012/07/07 02:27:34 UTC
[jira] [Updated] (MAPREDUCE-4400) Fix performance regression for
small jobs/workflows
[ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated MAPREDUCE-4400:
-------------------------------
Attachment: mapreduce-4400-branch-1.patch
MAPREDUCE-2450 fixed the communication race condition that can cause occasional 1 minute timeout but make the PROGRESS_INTERVAL sleep pretty much mandatory. Any tasks including setup and cleanup tasks would need to sleep at least 3 seconds to finish.
The patch make the wait interruptable when tasks finish.
With this patch and MAPREDUCE-4399 and outofband heartbeats, PigMix is about 20% faster than 0.20.2. These patches makes Hadoop 1.x series on par with Hadoop 2.x in terms of general performance.
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
> Key: MAPREDUCE-4400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.20.203.0, 1.0.3
> Reporter: Luke Lu
> Assignee: Luke Lu
> Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira