You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2009/01/16 12:49:59 UTC
[jira] Updated: (HADOOP-3327) Shuffling fetchers waited too long
between map output fetch re-tries
[ https://issues.apache.org/jira/browse/HADOOP-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-3327:
--------------------------------------------
Attachment: patch-3327.txt
Attaching a patch for review, while i continue my testing.
I have done simple tests with patch. Results are as follows:
With read timeouts simulated for 4 map outputs :
* On Single node cluster with 2 maps, 1 reducer and maxMapRuntime 6sec.
** Job took 41 mins 4sec without the patch
** Job took 22mins 2 sec with the patch
* 20 node cluster with 50 maps, 6 reducers and maxMapRuntime 6sec
** Job took 18 mins 23sec without the patch
** Job took 6mins 32sec with the patch.
* 20 node cluster with 50 maps, 6 reducers with maxMapRuntime 2 mins 33 sec
** Job took 30mins 0sec without the patch
** Job took 8mins 30sec with the patch
> Shuffling fetchers waited too long between map output fetch re-tries
> --------------------------------------------------------------------
>
> Key: HADOOP-3327
> URL: https://issues.apache.org/jira/browse/HADOOP-3327
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Runping Qi
> Assignee: Amareshwari Sriramadasu
> Attachments: hadoop-3327-v1.patch, hadoop-3327-v2.patch, hadoop-3327-v3.patch, hadoop-3327.patch, patch-3327.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.