[jira] [Created] (YARN-10958) Use correct configuration for Group service init in CSMappingPlacementRule

["Peter Bacsko (Jira)" <ji...@apache.org>]

Peter Bacsko created YARN-10958:
-----------------------------------

             Summary: Use correct configuration for Group service init in CSMappingPlacementRule
                 Key: YARN-10958
                 URL: https://issues.apache.org/jira/browse/YARN-10958
             Project: Hadoop YARN
          Issue Type: Bug
            Reporter: Peter Bacsko


There is a potential problem in {{CSMappingPlacementRule.java}}:
{noformat}
    if (groups == null) {
      groups = Groups.getUserToGroupsMappingService(conf);
    }
{noformat}
The problem is, we're supposed to pass {{scheduler.getConf()}}. The "conf" object is the config for capacity scheduler, which does not include the property which selects the group service provider. Therefore, the current code just works by chance, because Group mapping service is already initialized at this point. See the original fix in YARN-10053.

Also, need a unit test to verify it.

Idea:
 # Create a Configuration object in which the property "hadoop.security.group.mapping" refers to an existing a test implementation.
 # Add a new method to {{Groups}} which nulls out the singleton instance, eg. {{Groups.reset()}}.
 # Create a mock CapacityScheduler where {{getConf()}} and {{getConfiguration()}} contain different settings for "hadoop.security.group.mapping". Since {{getConf()}} is the service config, this should return the config object created in step #1.
 # Create an instance of {{CSMappingPlacementRule}} with a single primary group rule.
 # Run the placement evaluation.
 # Expected: returned queue matches what is supposed to be coming from the test group mapping service ("testuser" --> "testqueue").
 # Modify "hadoop.security.group.mapping" in the config object created in step #1.
 # Call {{Groups.refresh()}} which changes the group mapping ("testuser" --> "testqueue2"). This requires that the test group mapping service implement {{GroupMappingServiceProvider.cacheGroupsRefresh()}}.
 # Create a new instance of {{CSMappingPlacementRule}}.
 # Run the placement evaluation again
 # Expected: with the same user, the target queue has changed.

This looks convoluted, but these steps make sure that:
 # {{CSMappingPlacementRule}} will force the initialization of groups.
 # We select the correct configuration for group service init.
 # We don't create a new {{Groups}} instance if the singleton is initialized, so we cover the original problem described in YARN-10597.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org