[jira] [Assigned] (YARN-10958) Use correct configuration for Group service init in CSMappingPlacementRule

["Szilard Nemeth (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-10958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Szilard Nemeth reassigned YARN-10958:
-------------------------------------

    Assignee: Szilard Nemeth

> Use correct configuration for Group service init in CSMappingPlacementRule
> --------------------------------------------------------------------------
>
>                 Key: YARN-10958
>                 URL: https://issues.apache.org/jira/browse/YARN-10958
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Peter Bacsko
>            Assignee: Szilard Nemeth
>            Priority: Major
>
> There is a potential problem in {{CSMappingPlacementRule.java}}:
> {noformat}
>     if (groups == null) {
>       groups = Groups.getUserToGroupsMappingService(conf);
>     }
> {noformat}
> The problem is, we're supposed to pass {{scheduler.getConf()}}. The "conf" object is the config for capacity scheduler, which does not include the property which selects the group service provider. Therefore, the current code just works by chance, because Group mapping service is already initialized at this point. See the original fix in YARN-10053.
> Also, need a unit test to verify it.
> Idea:
>  # Create a Configuration object in which the property "hadoop.security.group.mapping" refers to an existing a test implementation.
>  # Add a new method to {{Groups}} which nulls out the singleton instance, eg. {{Groups.reset()}}.
>  # Create a mock CapacityScheduler where {{getConf()}} and {{getConfiguration()}} contain different settings for "hadoop.security.group.mapping". Since {{getConf()}} is the service config, this should return the config object created in step #1.
>  # Create an instance of {{CSMappingPlacementRule}} with a single primary group rule.
>  # Run the placement evaluation.
>  # Expected: returned queue matches what is supposed to be coming from the test group mapping service ("testuser" --> "testqueue").
>  # Modify "hadoop.security.group.mapping" in the config object created in step #1.
>  # Call {{Groups.refresh()}} which changes the group mapping ("testuser" --> "testqueue2"). This requires that the test group mapping service implement {{GroupMappingServiceProvider.cacheGroupsRefresh()}}.
>  # Create a new instance of {{CSMappingPlacementRule}}.
>  # Run the placement evaluation again
>  # Expected: with the same user, the target queue has changed.
> This looks convoluted, but these steps make sure that:
>  # {{CSMappingPlacementRule}} will force the initialization of groups.
>  # We select the correct configuration for group service init.
>  # We don't create a new {{Groups}} instance if the singleton is initialized, so we cover the original problem described in YARN-10597.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org