[jira] Commented: (MAPREDUCE-1184) mapred.reduce.slowstart.completed.maps is too high by default

["Matei Zaharia (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773768#action_12773768 ] 

Matei Zaharia commented on MAPREDUCE-1184:
------------------------------------------

This is a good idea. Ideally though, we might actually want slow start to depend on the amount of map output data and the rate at which data can be copied. If you have a job with only a few MB of map output per reducer, setting slow start as high as 95% isn't going to impact your response time too much. On the other hand, if you have a job where the maps "explode" the output and you know that the bulk of your time will be spent in the shuffle phase, you might want to set it lower.

> mapred.reduce.slowstart.completed.maps is too high by default
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-1184
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1184
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Allen Wittenauer
>
> By default, this value is set to 5%.  I believe for most real world situations the code isn't efficient enough to be set this low.  This should be higher, probably around the 50% mark, especially given the predominance of non-FIFO schedulers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.