You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Greg Senia (JIRA)" <ji...@apache.org> on 2015/11/06 16:35:27 UTC
[jira] [Updated] (YARN-4336) YARN NodeManager - Container
Initialization - Excessive load on NSS/LDAP
[ https://issues.apache.org/jira/browse/YARN-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Greg Senia updated YARN-4336:
-----------------------------
Attachment: YARN-4336-tactical.txt
tactical fix
> YARN NodeManager - Container Initialization - Excessive load on NSS/LDAP
> ------------------------------------------------------------------------
>
> Key: YARN-4336
> URL: https://issues.apache.org/jira/browse/YARN-4336
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.4.0, 2.4.1, 2.6.0, 2.7.0, 2.6.1, 2.7.1
> Environment: NSS w/ SSSD or Dell/Quest - VASD
> Reporter: Greg Senia
> Assignee: Greg Senia
> Attachments: YARN-4336-tactical.txt
>
>
> Hi folks after performing some debug for our Unix Engineering and Active Directory teams it was discovered that on YARN Container Initialization a call via Hadoop Common AccessControlList.java:
> for(String group: ugi.getGroupNames()) {
> if (groups.contains(group)) {
> return true;
> }
> }
> Unfortunately with the security call to check access on "appattempt_XXXXXXXXXXXXX_XXXXX_XXXXX" will always return false but will make unnecessary calls to NameSwitch service on linux which will call things like SSSD/Quest VASD which will then initiate LDAP calls looking for non existent userid's causing excessive load on LDAP.
> For now our tactical work around is as follows:
> /**
> * Checks if a user represented by the provided {@link UserGroupInformation}
> * is a member of the Access Control List
> * @param ugi UserGroupInformation to check if contained in the ACL
> * @return true if ugi is member of the list
> */
> public final boolean isUserInList(UserGroupInformation ugi) {
> if (allAllowed || users.contains(ugi.getShortUserName())) {
> return true;
> } else {
> String patternString = "^appattempt_\\d+_\\d+_\\d+$";
> Pattern pattern = Pattern.compile(patternString);
> Matcher matcher = pattern.matcher(ugi.getShortUserName());
> boolean matches = matcher.matches();
> if (matches) {
> LOG.debug("Bailing !! AppAttempt Matches DONOT call UGI FOR GROUPS!!");;
> return false;
> }
>
>
> for(String group: ugi.getGroupNames()) {
> if (groups.contains(group)) {
> return true;
> }
> }
> }
> return false;
> }
> public boolean isUserAllowed(UserGroupInformation ugi) {
> return isUserInList(ugi);
> }
> Example of VASD Debug log showing the lookups for one task attempt 32 of them:
> One task:
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC...@EXNSD.EXA.EXAMPLE.COM> with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>, base=<>, scope=<sub>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)