[jira] [Commented] (YARN-3409) Add constraint node labels

["Daniel Templeton (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093622#comment-16093622 ] 

Daniel Templeton commented on YARN-3409:
----------------------------------------

The big advantage I see is that we don't have two duplicate sets of code, one for labels and one for resources.  For example, constraint checking, soft resource requests (or relaxed partitions), script-driven label/resource values, etc.  Since labels and resources are very similar concepts (the same concept in HPC schedulers), they have many of the same needs.  It seems silly to build everything twice.

On the idea of not needing the labels during placement, what about the labels that derive their value from a script?  The value for the label could change between when the allocation is made and when the AM launches the container.  Shouldn't the NM be given the full list of constraints (resources and labels) so that it can validate that the allocation is still valid before starting the container?

> Add constraint node labels
> --------------------------
>
>                 Key: YARN-3409
>                 URL: https://issues.apache.org/jira/browse/YARN-3409
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, capacityscheduler, client
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf, YARN-3409.WIP.001.patch
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to determinate how resources of a special set of nodes could be shared by a group of entities (like teams, departments, etc.). Partitions of a cluster has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has priority to use the partition).
> - Percentage of capacities can apply on partition (Market team has 40% minimum capacity and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of node’s hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >= 2.20 && JDK.version >= 8u20 && x86_64).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org