[jira] [Created] (YARN-3054) Preempt policy in FairScheduler may cause mapreduce job never finish

["Peng Zhang (JIRA)" <ji...@apache.org>]

Peng Zhang created YARN-3054:
--------------------------------

             Summary: Preempt policy in FairScheduler may cause mapreduce job never finish
                 Key: YARN-3054
                 URL: https://issues.apache.org/jira/browse/YARN-3054
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.6.0
            Reporter: Peng Zhang


Preemption policy is related with schedule policy now. Using comparator of schedule policy to find preemption candidate cannot guarantee a subset of containers never be preempted. And this may cause tasks to be preempted periodically before they finish. So job cannot make any progress. 

I think preemption in YARN should got below assurance:
1. Mapreduce jobs can get additional resources when others are idle;
2. Mapreduce jobs for one user in one queue can still progress with its min share when others preempt resources back.

Maybe always preempt the latest app and container can get this? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)