[jira] [Created] (YARN-11682) Legacy auto created queue in absolute mode has zero capacity after creation during app recovery

["Brian Goerlitz (Jira)" <ji...@apache.org>]

Brian Goerlitz created YARN-11682:
-------------------------------------

             Summary: Legacy auto created queue in absolute mode has zero capacity after creation during app recovery
                 Key: YARN-11682
                 URL: https://issues.apache.org/jira/browse/YARN-11682
             Project: Hadoop YARN
          Issue Type: Bug
          Components: capacity scheduler
            Reporter: Brian Goerlitz


During recovery of a running app in a legacy auto created queue configured in absolute mode the configured min resources will be set to zero as NodeManagers have not registered yet (clusterResource is zero).

{code:java}
GuaranteedOrZeroCapacityOverTimePolicy.getInitialLeafQueueConfiguration(AbstractAutoCreatedLeafQueue leafQueue)
...
   if (availableCapacity >= leafQueueTemplateCapacities
            .getAbsoluteCapacity(nodeLabel)) {
          updateCapacityFromTemplate(capacities, nodeLabel);
          activate(leafQueue, nodeLabel);
        } else{
          updateToZeroCapacity(capacities, nodeLabel, leafQueue);
        }

GuaranteedOrZeroCapacityOverTimePolicy.updateToZeroCapacity(QueueCapacities capacities, String nodeLabel, AbstractLeafQueue leafQueue)

private void updateToZeroCapacity(QueueCapacities capacities,
      String nodeLabel, AbstractLeafQueue leafQueue) {
    capacities.setCapacity(nodeLabel, 0.0f);
    capacities.setMaximumCapacity(nodeLabel,
        leafQueueTemplateCapacities.getMaximumCapacity(nodeLabel));
    leafQueue.getQueueResourceQuotas().
        setConfiguredMinResource(nodeLabel, Resource.newInstance(0, 0));
  }

{code}

When a NodeManager is registered, {{AbstractCSQueue.updateEffectiveResources(Resource clusterResource)}} is called, but specifically absolute mode queues are updated using the configured min resource, which will now be zero.

{code:java}
AbstractCSQueue.updateEffectiveResources(Resource clusterResource)
...
if (getCapacityConfigType().equals(
          CapacityConfigType.ABSOLUTE_RESOURCE)) {
        newEffectiveMinResource = createNormalizedMinResource(
            usageTracker.getQueueResourceQuotas().getConfiguredMinResource(label),
            ((AbstractParentQueue) parent).getEffectiveMinRatio(label));
...
   usageTracker.getQueueResourceQuotas().setEffectiveMinResource(label,
          newEffectiveMinResource);
{code}

Reinitializing the queue via a config change will correctly recalculate the capacity based on current clusterResource.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org