You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2020/10/08 23:51:26 UTC
Re: Review Request 72854: RANGER-2986: Performance improvements for
Ranger usersync with AD/LDAP sync source
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72854/#review222006
-----------------------------------------------------------
Ship it!
Ship It!
- Abhay Kulkarni
On Sept. 30, 2020, 10:44 p.m., Sailaja Polavarapu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72854/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2020, 10:44 p.m.)
>
>
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2986
> https://issues.apache.org/jira/browse/RANGER-2986
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Refactored some of the usersync code. Added new APIs to batch update users, groups, and memberships to ranger admin for better performance. Also, consolidated code for AD/LDAP sync with and without incremental sync as well as some extra configuration. Made code imporvements to in-memory cache and delta computation in usersync.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/admin-web.xml fcab9abe9
> distro/src/main/assembly/usersync.xml 8d66c38d8
> pom.xml 1d31be052
> security-admin/pom.xml 0a0692d61
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java ca492fece
> security-admin/src/main/java/org/apache/ranger/db/XXGroupUserDao.java 1e41e708a
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 1198c6dfb
> security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java e299f1fc7
> security-admin/src/main/java/org/apache/ranger/service/XGroupUserService.java bf39f5ac4
> security-admin/src/main/java/org/apache/ranger/view/VXUsersGroupRoleAssignments.java 848a2c1bd
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml 7d9e145fe
> ugsync-util/.gitignore PRE-CREATION
> ugsync-util/pom.xml PRE-CREATION
> ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java PRE-CREATION
> ugsync/pom.xml 8a1431c30
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapDeltaUserGroupBuilder.java 011c9c600
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java 2df7dfb12
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java ca3aad87d
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/PolicyMgrUserGroupBuilder.java d27518119
> ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java 2c98bd818
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/FileSyncSourceInfo.java a6348d578
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/GetXGroupListResponse.java 5f1e9af04
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/GetXUserGroupListResponse.java cf6957dc7
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/GetXUserListResponse.java 809a84779
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/GroupUserInfo.java a2cfa7bcc
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/LdapSyncSourceInfo.java 54802a040
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/MUserInfo.java 4f865afdd
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/UgsyncAuditInfo.java 65ac89b0a
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/UnixSyncSourceInfo.java 6e5df9d90
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/UserGroupInfo.java 936acca07
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/UsersGroupRoleAssignments.java e6cabdbd2
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/XGroupInfo.java b61f39cf7
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/XUserGroupInfo.java 06b21e0ca
> ugsync/src/main/java/org/apache/ranger/unixusersync/model/XUserInfo.java bee63237b
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/FileSourceUserGroupBuilder.java 58505854d
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java ec986b91d
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java e1540c677
> ugsync/src/main/java/org/apache/ranger/usergroupsync/AbstractUserGroupSource.java 4a54e5786
> ugsync/src/main/java/org/apache/ranger/usergroupsync/UserGroupSink.java 368c4f84d
> ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestFileSourceUserGroupBuilder.java b29a1b0c5
> ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java 56c3f6a4a
> ugsync/src/test/java/org/apache/ranger/usergroupsync/LdapPolicyMgrUserGroupBuilderTest.java e10f632dc
> ugsync/src/test/java/org/apache/ranger/usergroupsync/PolicyMgrUserGroupBuilderTest.java b0ce87230
> ugsync/src/test/java/org/apache/ranger/usergroupsync/TestLdapUserGroup.java 6393b3dd3
>
>
> Diff: https://reviews.apache.org/r/72854/diff/3/
>
>
> Testing
> -------
>
> 1. Updated unit tests accordingly.
> 2. Patched cluster with these changes and verified the functionality
> 3. Also, ran some performace tests with 1000 users, 1000 groups, and 3500 group memberships
>
>
> Thanks,
>
> Sailaja Polavarapu
>
>