You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by doom <43...@qq.com> on 2020/09/08 02:22:56 UTC

Question about FSLeafQueue.updateStarvedAppsMinshare

Hi all:
i find the FSLeafQueue.updateStarvedAppsMinshare is responsible for `Distribute queue's minshare starvation to a set of apps`, 



1. 

so when appMinShare<=pending [ pending is queue's minshare starvation]:
pending = pending - appMinShare&nbsp;&nbsp; // allocat part of pending to appMinShare
app's MinshareStarvation = appMinShare


2. 

when appMinShare&gt;pending:
2.1 (the src code)

appMinShare = appMinShare - pending
pending = 0
app's MinshareStarvation = appMinShare


2.2(what i thought)

what confuse me is&nbsp; `appMinShare = appMinShare - pending`,&nbsp; when pending is not enough, i thought it should allocat all the rest of pending to appMinShare:

appMinShare = pending
pending = 0
app's MinshareStarvation = appMinShare




So&nbsp; 2.1 and 2.2 which one is right?&nbsp; if 2.1 is right,&nbsp; what dose `appMinShare = appMinShare - pending` mean? 






Thanx!!!




       
         

https://github.com/apache/hadoop/blob/34fe74da0e9c68173e1de196c496b9cfca029618/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java#L250





org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue


```java

&nbsp; /**
&nbsp;&nbsp; * Distribute minshare starvation to a set of apps
&nbsp;&nbsp; * @param appsWithDemand set of apps
&nbsp;&nbsp; * @param minShareStarvation minshare starvation to distribute
&nbsp;&nbsp; */
&nbsp; private void updateStarvedAppsMinshare(
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final TreeSet<FSAppAttempt&gt; appsWithDemand,
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final Resource minShareStarvation) {
&nbsp;&nbsp;&nbsp; Resource pending = Resources.clone(minShareStarvation);

&nbsp;&nbsp;&nbsp; // Keep adding apps to the starved list until the unmet demand goes over
&nbsp;&nbsp;&nbsp; // the remaining minshare
&nbsp;&nbsp;&nbsp; for (FSAppAttempt app : appsWithDemand) {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (!Resources.isNone(pending)) {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Resource appMinShare = app.getPendingDemand();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Resources.subtractFromNonNegative(
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; appMinShare, app.getFairshareStarvation());

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (Resources.greaterThan(policy.getResourceCalculator(),
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; scheduler.getClusterResource(), appMinShare, pending)) {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Resources.subtractFromNonNegative(appMinShare, pending);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; pending = none();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } else {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Resources.subtractFromNonNegative(pending, appMinShare);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; app.setMinshareStarvation(appMinShare);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; context.getStarvedApps().addStarvedApp(app);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } else {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // Reset minshare starvation in case we had set it in a previous
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // iteration
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; app.resetMinshareStarvation();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&nbsp;&nbsp;&nbsp; }
&nbsp; }
```