You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Cindy Li (JIRA)" <ji...@apache.org> on 2013/11/14 02:10:20 UTC
[jira] [Commented] (YARN-291) [Umbrella] Dynamic resource
configuration
[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822052#comment-13822052 ]
Cindy Li commented on YARN-291:
-------------------------------
JunPing,
I'm trying to find out where you handle different cases of commitTimeout, and getting to this in schedulerUtils.java:
// TODO process resource over-commitment case (allocated containers
// > total capacity) in different option by getting value of
// overCommitTimeoutMillis.
What's your thought on this? Are you working on it?
In our case, we would need to set overCommitTimeoutMills to be >0, and the capacity(memory/vore) to be zero for node to be put into maintenance. What I would expect is:
#1. The scheduler stops scheduling new application attempts to the node
#2. Clean up containers on that node after overCommitTimeoutMills.
>From my understanding, #1 is done once the capacity is set to 0, but what do you plan to put there for #2? YARN-999 seems not solving #2.
> [Umbrella] Dynamic resource configuration
> -----------------------------------------
>
> Key: YARN-291
> URL: https://issues.apache.org/jira/browse/YARN-291
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, scheduler
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, YARN-291-CoreAndAdmin.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, YARN-291-YARNClientCommandline-04.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch
>
>
> The current Hadoop YARN resource management logic assumes per node resource is static during the lifetime of the NM process. Allowing run-time configuration on per node resource will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized. More background and design details can be found in attached proposal.
--
This message was sent by Atlassian JIRA
(v6.1#6144)