You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "zhihai xu (JIRA)" <ji...@apache.org> on 2015/02/02 22:57:37 UTC
[jira] [Commented] (MAPREDUCE-6143) add configuration for
mapreduce speculative execution in MR2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302005#comment-14302005 ]
zhihai xu commented on MAPREDUCE-6143:
--------------------------------------
thanks [~rkanter] for the review and commit.
> add configuration for mapreduce speculative execution in MR2
> -------------------------------------------------------------
>
> Key: MAPREDUCE-6143
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6143
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 2.5.1
> Reporter: zhihai xu
> Assignee: zhihai xu
> Fix For: 2.7.0
>
> Attachments: MAPREDUCE-6143.000.patch, MAPREDUCE-6143.001.patch, MAPREDUCE-6143.002.patch
>
>
> add configuration for mapreduce speculative execution in MR2.
> Currently mapreduce.job.speculative.speculativecap and mapreduce.job.speculative.slownodethreshold are not used for MR2 mapreduce speculative execution any more.
> We should make the following hardcode constants in DefaultSpeculator configurable for MR2 Map Reduce speculative execution:
> private static final long SOONEST_RETRY_AFTER_NO_SPECULATE = 1000L * 1L;
> private static final long SOONEST_RETRY_AFTER_SPECULATE = 1000L * 15L;
> private static final double PROPORTION_RUNNING_TASKS_SPECULATABLE = 0.1;
> private static final double PROPORTION_TOTAL_TASKS_SPECULATABLE = 0.01;
> private static final int MINIMUM_ALLOWED_SPECULATIVE_TASKS = 10;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)