You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Sailaja Polavarapu (Jira)" <ji...@apache.org> on 2020/09/08 22:50:00 UTC

[jira] [Created] (RANGER-2986) Performance improvements for Ranger usersync

Sailaja Polavarapu created RANGER-2986:
------------------------------------------

             Summary: Performance improvements for Ranger usersync
                 Key: RANGER-2986
                 URL: https://issues.apache.org/jira/browse/RANGER-2986
             Project: Ranger
          Issue Type: Bug
          Components: Ranger, usersync
    Affects Versions: 2.1.0
            Reporter: Sailaja Polavarapu
            Assignee: Sailaja Polavarapu


Currently Ranger Usersync updates users and groups to Ranger Admin one at a time. This causes lot of traffic between userync and ranger admin during initial sync. Instead update users, groups, and memberships in batches after computing the deltas. Batch update also applies for subsequent sync cycle.

Also, in case of AD/LDAP sync source with incremental sync, usersync doesn't maintain in-memory cache to compute deltas. During every restart of usersync, usersync performance full sync with AD/LDAP and updates all the users and groups to ranger admin (even though there are no updates to users and groups at the source). This can be avoided by caching the users and groups that are existing in ranger db during startup and compute deltas at usersync like other sync source code paths. 

Currently building initial cache in usersync is taking considerable amount of time especially when the no. of users and groups are large. This needs to be improved as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)