[jira] [Commented] (YARN-4133) Containers to be preempted leaks in FairScheduler preemption logic.

["Xianyin Xin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735959#comment-14735959 ] 

Xianyin Xin commented on YARN-4133:
-----------------------------------

Hi [~zxu], it seems the current preemption logic has many problems. I just updated one in [https://issues.apache.org/jira/browse/YARN-4120?focusedCommentId=14735952&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14735952]. I think a logic refactor is need, what do you think?

> Containers to be preempted leaks in FairScheduler preemption logic.
> -------------------------------------------------------------------
>
>                 Key: YARN-4133
>                 URL: https://issues.apache.org/jira/browse/YARN-4133
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.7.1
>            Reporter: zhihai xu
>            Assignee: zhihai xu
>         Attachments: YARN-4133.000.patch
>
>
> Containers to be preempted leaks in FairScheduler preemption logic. It may cause missing preemption due to containers in {{warnedContainers}} wrongly removed. The problem is in {{preemptResources}}:
> There are two issues which can cause containers  wrongly removed from {{warnedContainers}}:
> Firstly missing the container state {{RMContainerState.ACQUIRED}} in the condition check:
> {code}
> (container.getState() == RMContainerState.RUNNING ||
>               container.getState() == RMContainerState.ALLOCATED)
> {code}
> Secondly if  {{isResourceGreaterThanNone(toPreempt)}} return false, we shouldn't remove container from {{warnedContainers}}, We should only remove container from {{warnedContainers}}, if container is not in state {{RMContainerState.RUNNING}}, {{RMContainerState.ALLOCATED}} and {{RMContainerState.ACQUIRED}}.
> {code}
>       if ((container.getState() == RMContainerState.RUNNING ||
>               container.getState() == RMContainerState.ALLOCATED) &&
>               isResourceGreaterThanNone(toPreempt)) {
>         warnOrKillContainer(container);
>         Resources.subtractFrom(toPreempt, container.getContainer().getResource());
>       } else {
>         warnedIter.remove();
>       }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)