You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Inigo Goiri (JIRA)" <ji...@apache.org> on 2016/06/09 16:39:21 UTC
[jira] [Comment Edited] (YARN-5215) Scheduling containers based on
external load in the servers
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322829#comment-15322829 ]
Inigo Goiri edited comment on YARN-5215 at 6/9/16 4:38 PM:
-----------------------------------------------------------
[~jlowe], my view on this feature is more like making the scheduler aware of a "container" that is not managed by YARN and has a dynamic size. Before moving forward with this, we should agree that this is a proper abstraction and complementary to the overcommit effort.
Regarding the UI stuff, let's try to make a list of what to expose and I'll give it a try. My first idea is to expose the following per node/queue/cluster:
* Node utilization
* Container utilization
* External utilization
* Allocated/unallocated resources
Can you think on any other metric/aggregation we should be reporting?
was (Author: elgoiri):
[~jlowe], my view on this feature is more like making the scheduler our of a "container" that is not managed by YARN. This is a dynamic size container. Before moving forward with this, we should agree that this is a proper abstraction and complementary to the overcommit effort.
Regarding the UI stuff, let's try to make a list of what to expose and I'll give it a try. My first idea is to expose the following per node/queue/cluster:
* Node utilization
* Container utilization
* External utilization
* Allocated/unallocated resources
Can you think on any other metric/aggregation we should be reporting?
> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>
> Key: YARN-5215
> URL: https://issues.apache.org/jira/browse/YARN-5215
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Inigo Goiri
> Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the resources. The proposal is to use the utilization information in the node and the containers to estimate how much is consumed by external processes and schedule based on this estimation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org