You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "gu-chi (JIRA)" <ji...@apache.org> on 2015/06/01 05:58:17 UTC

[jira] [Commented] (YARN-3730) scheduler reserve more resource than required

    [ https://issues.apache.org/jira/browse/YARN-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566919#comment-14566919 ] 

gu-chi commented on YARN-3730:
------------------------------

Thx Naga, as improvements r not merged to my current using version, so this feature is not invoked, will set "yarn.scheduler.capacity.reservations-continue-look-all-nodes" to false on version 2.7.0 and check the outcome.

> scheduler reserve more resource than required
> ---------------------------------------------
>
>                 Key: YARN-3730
>                 URL: https://issues.apache.org/jira/browse/YARN-3730
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: gu-chi
>
> Using capacity scheduler, environment is 3 NM and each has 9 vcores, I ran a spark task with 4 executors and each executor 5 cores, as suspected, only 1 executor not able to start and will be reserved, but actually more containers are reserved. This way, I can not run some other smaller tasks. As I checked the capacity scheduler, the 'needContainers' method in LeafQueue.java has a computation of 'starvation', this cause the scenario of more container reserved than required, any idea or suggestion on this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)