[jira] [Commented] (YARN-6769) Put the no demand queue after the most in FairSharePolicy#compare

["daemon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077525#comment-16077525 ] 

daemon commented on YARN-6769:
------------------------------

diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
index f8cdb45929..e930b80e45 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
    b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
@@ -79,6  79,19 @@ public String getName() {
 
     @Override
     public int compare(Schedulable s1, Schedulable s2) {
       Resource demand1 = s1.getDemand();
       Resource demand2 = s2.getDemand();
       // Put the schedulable which does not require resource to
       // the end. So the other schedulable can get resource as soon as
       // possible though it use resource greater then it minShare or demand.
       if (demand1.equals(Resources.none()) &&
           !demand2.equals(Resources.none())) {
         return 1;
       } else if (demand2.equals(Resources.none()) &&
           !demand1.equals(Resources.none())) {
         return -1;
       }
       
       double minShareRatio1, minShareRatio2;
       double useToWeightRatio1, useToWeightRatio2;
       double weight1, weight2;
@@ -86,9  99,9 @@ public int compare(Schedulable s1, Schedulable s2) {
       Resource resourceUsage1 = s1.getResourceUsage();
       Resource resourceUsage2 = s2.getResourceUsage();
       Resource minShare1 = Resources.min(RESOURCE_CALCULATOR, null,
-          s1.getMinShare(), s1.getDemand());
           s1.getMinShare(), demand1);
       Resource minShare2 = Resources.min(RESOURCE_CALCULATOR, null,
-          s2.getMinShare(), s2.getDemand());
           s2.getMinShare(), demand2);
       boolean s1Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,
           resourceUsage1, minShare1);
       boolean s2Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,


> Put the no demand queue after the most in FairSharePolicy#compare
> -----------------------------------------------------------------
>
>                 Key: YARN-6769
>                 URL: https://issues.apache.org/jira/browse/YARN-6769
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.7.2
>            Reporter: daemon
>            Priority: Minor
>             Fix For: 2.9.0
>
>
> When use fairsheduler as RM scheduler, before assign container we will sort all queues or applications. 
> We will use FairSharePolicy#compare as the comparator， but the comparator is not so perfect.
> It have a problem as blow:
> 1. when a queue use resource over minShare(minResources), it will put behind the queue whose demand is zeor.
> so it will greater opportunity to get the resource although it do not want. It will waste schedule time when assign container
> to queue or application.
> I have fix it, and I will upload the patch to the jira.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org