You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Manikandan R (Jira)" <ji...@apache.org> on 2019/09/24 18:00:00 UTC
[jira] [Comment Edited] (YARN-9841) Capacity scheduler: add support
for combined %user + %primary_group mapping
[ https://issues.apache.org/jira/browse/YARN-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937024#comment-16937024 ]
Manikandan R edited comment on YARN-9841 at 9/24/19 5:59 PM:
-------------------------------------------------------------
Thanks [~pbacsko]. Attached .001.patch for review.
While working on this JIRA, had come across below observations:
# As documented in [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html,|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html] tried "u:user2:%primary_group" mapping and don't think it is working as expected. Expected queue o/p is, queue name similar to primary group of the user, but it is not the case. Where as, "u:%user:%primary_group" mapping is working as expected.
# Use case of "u:%user:parentqueue.%user" mapping doesn't return expected o/p when it is working in conjunction with "u:%user:%primary_group" mapping. Where as, Using "u:%user:parentqueue.%user" mapping alone is working as expected.
Created a separate junit patch to validate these observations. Can you please validate this? We can raise separate JIRA's to address these issues based on your confirmation.
was (Author: manirajv06@gmail.com):
Thanks [~pbacsko]. Attached .001.patch for review.
While working on this JIRA, had come across below observations:
# As documented in [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html,|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html] tried "u:user2:%primary_group" mapping and don't think it is working as expected. Expected queue o/p is, queue name similar to primary group of the user, but it is not the case.
# Use case of "u:%user:parentqueue.%user" mapping doesn't return expected o/p when it is working in conjunction with "u:%user:%primary_group" mapping. Where as, Using "u:%user:parentqueue.%user" mapping alone is working as expected.
Created a separate junit patch to validate these observations. Can you please validate this? We can raise separate JIRA's to address these issues based on your confirmation.
> Capacity scheduler: add support for combined %user + %primary_group mapping
> ---------------------------------------------------------------------------
>
> Key: YARN-9841
> URL: https://issues.apache.org/jira/browse/YARN-9841
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Peter Bacsko
> Assignee: Manikandan R
> Priority: Major
> Attachments: YARN-9841.001.patch, YARN-9841.junit.patch
>
>
> Right now in CS, using {{%primary_group}} with a parent queue is only possible this way:
> {{u:%user:parentqueue.%primary_group}}
> Looking at https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/UserGroupMappingPlacementRule.java, we cannot do something like:
> {{u:%user:%primary_group.%user}}
> Fair Scheduler supports a nested rule where such a placement/mapping rule is possible. This improvement would reduce this feature gap.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org