You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "daemon (JIRA)" <ji...@apache.org> on 2017/07/07 03:23:00 UTC
[jira] [Issue Comment Deleted] (YARN-6769) Put the no demand queue
after the most in FairSharePolicy#compare
[ https://issues.apache.org/jira/browse/YARN-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
daemon updated YARN-6769:
-------------------------
Comment: was deleted
(was: diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
index f8cdb45929..e930b80e45 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
@@ -79,6 +79,19 @@ public String getName() {
@Override
public int compare(Schedulable s1, Schedulable s2) {
+ Resource demand1 = s1.getDemand();
+ Resource demand2 = s2.getDemand();
+ // Put the schedulable which does not require resource to
+ // the end. So the other schedulable can get resource as soon as
+ // possible though it use resource greater then it minShare or demand.
+ if (demand1.equals(Resources.none()) &&
+ !demand2.equals(Resources.none())) {
+ return 1;
+ } else if (demand2.equals(Resources.none()) &&
+ !demand1.equals(Resources.none())) {
+ return -1;
+ }
+
double minShareRatio1, minShareRatio2;
double useToWeightRatio1, useToWeightRatio2;
double weight1, weight2;
@@ -86,9 +99,9 @@ public int compare(Schedulable s1, Schedulable s2) {
Resource resourceUsage1 = s1.getResourceUsage();
Resource resourceUsage2 = s2.getResourceUsage();
Resource minShare1 = Resources.min(RESOURCE_CALCULATOR, null,
- s1.getMinShare(), s1.getDemand());
+ s1.getMinShare(), demand1);
Resource minShare2 = Resources.min(RESOURCE_CALCULATOR, null,
- s2.getMinShare(), s2.getDemand());
+ s2.getMinShare(), demand2);
boolean s1Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,
resourceUsage1, minShare1);
boolean s2Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,
)
> Put the no demand queue after the most in FairSharePolicy#compare
> -----------------------------------------------------------------
>
> Key: YARN-6769
> URL: https://issues.apache.org/jira/browse/YARN-6769
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.7.2
> Reporter: daemon
> Priority: Minor
> Fix For: 2.9.0
>
>
> When use fairsheduler as RM scheduler, before assign container we will sort all queues or applications.
> We will use FairSharePolicy#compare as the comparator, but the comparator is not so perfect.
> It have a problem as blow:
> 1. when a queue use resource over minShare(minResources), it will put behind the queue whose demand is zeor.
> so it will greater opportunity to get the resource although it do not want. It will waste schedule time when assign container
> to queue or application.
> I have fix it, and I will upload the patch to the jira.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org