You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Xiaoyu Yao (Jira)" <ji...@apache.org> on 2020/07/09 18:36:00 UTC
[jira] [Updated] (HADOOP-17079) Optimize UGI#getGroups by adding
UGI#getGroupsSet
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiaoyu Yao updated HADOOP-17079:
--------------------------------
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Release Note:
Added a UserGroupMapping#getGroupsSet() API and deprecate UserGroupMapping#getGroups.
The UserGroupMapping#getGroups() can be expensive as it involves Set->List conversion. For user with large group membership (i.e., > 1000 groups), we recommend using getGroupSet to avoid the conversion and fast membership look up.
was:
Added a UserGroupMapping#getGroupsSet() API.
The UserGroupMapping#getGroups() can be expensive as it involves Set->List conversion. For user with large group membership (i.e., > 1000 groups), we recommend using getGroupSet to avoid the conversion and fast membership look up.
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Optimize UGI#getGroups by adding UGI#getGroupsSet
> -------------------------------------------------
>
> Key: HADOOP-17079
> URL: https://issues.apache.org/jira/browse/HADOOP-17079
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17079.002.patch, HADOOP-17079.003.patch, HADOOP-17079.004.patch, HADOOP-17079.005.patch, HADOOP-17079.006.patch, HADOOP-17079.007.patch
>
>
> UGI#getGroups has been optimized with HADOOP-13442 by avoiding the List->Set->List conversion. However the returned list is not optimized to contains lookup, especially the user's group membership list is huge (thousands+) . This ticket is opened to add a UGI#getGroupsSet and use Set#contains() instead of List#contains() to speed up large group look up while minimize List->Set conversions in Groups#getGroups() call.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org