[jira] [Created] (YARN-4133) Containers to be preempted leaks in FairScheduler preemption logic.

["zhihai xu (JIRA)" <ji...@apache.org>]

zhihai xu created YARN-4133:
-------------------------------

             Summary: Containers to be preempted leaks in FairScheduler preemption logic.
                 Key: YARN-4133
                 URL: https://issues.apache.org/jira/browse/YARN-4133
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.7.1
            Reporter: zhihai xu
            Assignee: zhihai xu


Containers to be preempted leaks in FairScheduler preemption logic. It may cause missing preemption due to containers in {{warnedContainers}} wrongly removed. The problem is in {{preemptResources}}:
There are two issues which can cause containers  wrongly removed from {{warnedContainers}}:
Firstly missing the container state {{RMContainerState.ACQUIRED}} in the condition check:
{code}
(container.getState() == RMContainerState.RUNNING ||
              container.getState() == RMContainerState.ALLOCATED)
{code}
Secondly if  {{isResourceGreaterThanNone(toPreempt)}} return false, we shouldn't remove container from {{warnedContainers}}, We should only remove container from {{warnedContainers}}, if container is not in state {{RMContainerState.RUNNING}}, {{RMContainerState.ALLOCATED}} and {{RMContainerState.ACQUIRED}}.
{code}
      if ((container.getState() == RMContainerState.RUNNING ||
              container.getState() == RMContainerState.ALLOCATED) &&
              isResourceGreaterThanNone(toPreempt)) {
        warnOrKillContainer(container);
        Resources.subtractFrom(toPreempt, container.getContainer().getResource());
      } else {
        warnedIter.remove();
      }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)