[jira] [Comment Edited] (YARN-3926) Extend the YARN resource model for easier resource-type management and profiles

["Varun Vasudev (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197763#comment-15197763 ] 

Varun Vasudev edited comment on YARN-3926 at 3/16/16 5:41 PM:
--------------------------------------------------------------

bq. That should work... But I feel, maybe allow mismatch should be the default. If NM has a super-set of RMs resource types, it will just be ignored, If sub-set, then for those specific resource-types, RM will assign a 0 value for the NM.

I don't have any particular preference - I can see scenarios for all 3. I'm fine with making allow mismatch the default.

bq. We can also add admin API on the RM to add / remove allowable resource types on the fly.

This should be do-able but we need to go through how this will affect on running apps.


was (Author: vvasudev):
bq. That should work... But I feel, maybe allow mismatch should be the default. If NM has a super-set of RMs resource types, it will just be ignored, If sub-set, then for those specific resource-types, RM will assign a 0 value for the NM.

I don't have any particular preference - I can see scenarios for all 3. I'm fine with making allow mismatch the default.

bq. We can also add admin API on the RM to add / remove allowable resource types on the fly.

This should be do-able but we need to go through the affect on running apps.

> Extend the YARN resource model for easier resource-type management and profiles
> -------------------------------------------------------------------------------
>
>                 Key: YARN-3926
>                 URL: https://issues.apache.org/jira/browse/YARN-3926
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Proposal for modifying resource model and profiles.pdf
>
>
> Currently, there are efforts to add support for various resource-types such as disk(YARN-2139), network(YARN-2140), and  HDFS bandwidth(YARN-2681). These efforts all aim to add support for a new resource type and are fairly involved efforts. In addition, once support is added, it becomes harder for users to specify the resources they need. All existing jobs have to be modified, or have to use the minimum allocation.
> This ticket is a proposal to extend the YARN resource model to a more flexible model which makes it easier to support additional resource-types. It also considers the related aspect of “resource profiles” which allow users to easily specify the various resources they need for any given container.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)