You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2016/03/09 05:08:40 UTC
[jira] [Commented] (YARN-3054) Preempt policy in FairScheduler may
cause mapreduce job never finish
[ https://issues.apache.org/jira/browse/YARN-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186463#comment-15186463 ]
Karthik Kambatla commented on YARN-3054:
----------------------------------------
[~peng.zhang] - the problem reported is not quite clear, mind elaborating it further?
> Preempt policy in FairScheduler may cause mapreduce job never finish
> --------------------------------------------------------------------
>
> Key: YARN-3054
> URL: https://issues.apache.org/jira/browse/YARN-3054
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Peng Zhang
>
> Preemption policy is related with schedule policy now. Using comparator of schedule policy to find preemption candidate cannot guarantee a subset of containers never be preempted. And this may cause tasks to be preempted periodically before they finish. So job cannot make any progress.
> I think preemption in YARN should got below assurance:
> 1. Mapreduce jobs can get additional resources when others are idle;
> 2. Mapreduce jobs for one user in one queue can still progress with its min share when others preempt resources back.
> Maybe always preempt the latest app and container can get this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)