You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Weiwei Yang (JIRA)" <ji...@apache.org> on 2018/03/21 12:01:00 UTC
[jira] [Comment Edited] (YARN-8013) Support APP-TAG namespace for
allocation tags
[ https://issues.apache.org/jira/browse/YARN-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407788#comment-16407788 ]
Weiwei Yang edited comment on YARN-8013 at 3/21/18 12:00 PM:
-------------------------------------------------------------
V2 patch uploaded. The patch basically has following changes
* Renamed {{app-label}} to {{app-tag}}
* Implemented {{AllocationTagNamespace#AppTag}}
* Removed the check in {{SingleConstraintAppPlacementAllocator}} as now we support all namespace types
* Added tests in {{TestAllocationTagsNamespace}} and {{TestPlacementConstraintsUtil}}
Once this task is done, next step I will work on YARN-8015 to completely support inter-app constraints in app allocator. During we should be able to remove most of the checks in {{SingleConstraintAppPlacementAllocator}} and delegate all placement constraint checks to {{PlacementConstraintsUtil}}.
Ping [~leftnoteasy], [~asuresh], [~kkaranasos], could you please help to review. Thanks a lot!
was (Author: cheersyang):
V2 patch uploaded. The patch basically has following changes
* Renamed {{app-label}} to {{app-tag}}
* Implemented {{AllocationTagNamespace#AppTag}}
* Removed the check in {{SingleConstraintAppPlacementAllocator}} as now we support all namespace types
* Added tests in {{TestAllocationTagsNamespace}} and {{TestPlacementConstraintsUtil}}
Once this task is done, next step I will work on YARN-8015 to completely support inter-app constraints in app allocator. During we should be able to remove most of the checks in {{SingleConstraintAppPlacementAllocator}} and delegate all placement constraint checks to {{PlacementConstraintsUtil}}.
> Support APP-TAG namespace for allocation tags
> ---------------------------------------------
>
> Key: YARN-8013
> URL: https://issues.apache.org/jira/browse/YARN-8013
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-8013.001.patch, YARN-8013.002.patch
>
>
> YARN-1461 adds *Application Tag* concept to Yarn applications, user is able to annotate application with multiple tags to classify apps. We can leverage this to represent a namespace for a certain group of apps. So instead of calling *app-label*, propose to call it *app-tag*.
> A typical use case is,
> There are a lot of TF jobs running on Yarn, and some of them are consuming resources heavily. So we want to limit number of PS on each node for such BIG players but ignore those SMALL ones. To achieve this, we can do following steps:
> # Add application tag "big-tf" to these big TF jobs
> # For each PS request, we add "ps" source tag and map it to constraint "{color:#d04437}notin, node, tensorflow/ps{color}" or "{color:#d04437}cardinality, node, tensorflow/ps{color}{color:#d04437}, 0, 2{color}" for finer grained controls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org