[jira] [Comment Edited] (YARN-796) Allow for (admin) labels on nodes and resource-requests

["Arun C Murthy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934174#comment-13934174 ] 

Arun C Murthy edited comment on YARN-796 at 3/13/14 10:00 PM:
--------------------------------------------------------------

Back to this, some thoughts:

* Admin interface
** Labels are specified by admins (node configuration, dynamic add/remove via rmadmin).
** Each scheduler (CS, FS) can pick how they want labels specified in their configs
** Dynamically added labels are, initially, not persisted across RM restarts. So, these need to be manually edited into yarn-site.xml, ACLs into capacity-scheduler.xml etc.
** By default, all nodes have a *default* label, but admins can explicitly set a list of labels and drop the *default* label.
** Queues have label ACLs i.e. admins can specify, per queue, what labels can be used by applications per queue

* End-user interface
** Applications can ask for containers on nodes with specific labels as part of the RR; however, host-specific RRs with labels are illegal i.e. labels are allowed only for  rack & * RRs: results in InvalidResourceRequestException
** RR with a non-existent label (point in time) is illegal: results in InvalidResourceRequestException
** RR with label without appropriate ACL results in InvalidResourceRequestException (do we want a special InvalidResourceRequestACLException?)
** Initially, RRs can ask for multiple labels with the expectation that it's an AND operation





was (Author: acmurthy):
Back to this, some thoughts:

* Admin interface
** Labels are specified by admins (node configuration, dynamic add/remove via rmadmin).
** Each scheduler (CS, FS) can pick how they want labels specified in their configs
** Dynamically added labels are, initially, not persisted across RM restarts. So, these need to be manually edited into capacity-scheduler.xml etc.
** By default, all nodes have a *default* label, but admins can explicitly set a list of labels and drop the *default* label.
** Queues have label ACLs i.e. admins can specify, per queue, what labels can be used by applications per queue

* End-user interface
** Applications can ask for containers on nodes with specific labels as part of the RR; however, host-specific RRs with labels are illegal i.e. labels are allowed only for  rack & * RRs: results in InvalidResourceRequestException
** RR with a non-existent label (point in time) is illegal: results in InvalidResourceRequestException
** RR with label without appropriate ACL results in InvalidResourceRequestException (do we want a special InvalidResourceRequestACLException?)
** Initially, RRs can ask for multiple labels with the expectation that it's an AND operation




> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.



--
This message was sent by Atlassian JIRA
(v6.2#6252)