You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Ramachandran Krishnan <ra...@gmail.com> on 2022/12/08 12:47:52 UTC
Review Request 74245: RANGER-3899:Policy creation takes more time when there are more users, groups, roles
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74245/
-----------------------------------------------------------
Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan Periasamy.
Bugs: RANGER-3899
https://issues.apache.org/jira/browse/RANGER-3899
Repository: ranger
Description
-------
Analysis is when policy has 500 users with 5 policy items.
we perform total 526 DB read calls and 504 DB write, 3 DB Batch write calls. This is the reason for policy creation time where 1000 plus DB calls are done sequentially.
Time is taken to create the policy :4~8 seconds
After Fix :After changing to batch create of XXPolicyRefUser instead of calling 1 DB write XXPolicyRefUser for every user of the policy:
When Policy contains 500 users,5 access Types (permissions),12 resources (1DB,1Table,10 columns)
we perform total 526 DB read calls and 6 DB write calls, 5 DB Batch write calls.
Time is taken to create the policy :2.6~4 seconds
Diffs
-----
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 6cc3509d8
Diff: https://reviews.apache.org/r/74245/diff/1/
Testing
-------
Thanks,
Ramachandran Krishnan
Re: Review Request 74245: RANGER-3899:Policy creation takes more time when there are more users, groups, roles
Posted by Ramachandran Krishnan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74245/
-----------------------------------------------------------
(Updated Dec. 12, 2022, 6:19 a.m.)
Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan Periasamy.
Bugs: RANGER-3899
https://issues.apache.org/jira/browse/RANGER-3899
Repository: ranger
Description (updated)
-------
Analysis is when policy has 500 users with 5 policy items.
we perform total 526 DB read calls and 504 DB write, 3 DB Batch write calls. This is the reason for policy creation time where 1000 plus DB calls are done sequentially.
Time is taken to create the policy :4~8 seconds
1st#Improvement:
After Fix :After changing to batch create of XXPolicyRefUser instead of calling 1 DB write XXPolicyRefUser for every user of the policy:
When Policy contains 500 users,5 access Types (permissions),12 resources (1DB,1Table,10 columns)
we perform total 526 DB read calls and 6 DB write calls, 5 DB Batch write calls.
Time is taken to create the policy :2.6~4 seconds
2nd#Improvement:
Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
Fixing this part will reduce the DB read calls to a very minimum which will improve performance drastically.
The policy contains 500 users,5 access Types (permissions),12 resources (1DB,1Table,10 columns)
After fix:
DB Read count:27
Time taken to create the policy: 1.4 to 2 secs (The policy contains 500 users,5 access Types (permissions),12 resources (1DB,1Table,10 columns))
DB read calls will be reduced to 27 from 526
Time is taken to create the policy :1.4~2 seconds
Diffs (updated)
-----
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 6cc3509d8
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 8ca4f62bb
security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 4e5b692b5
security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 97bc2680a
security-admin/src/main/resources/META-INF/jpa_named_queries.xml 85c8b6213
Diff: https://reviews.apache.org/r/74245/diff/2/
Changes: https://reviews.apache.org/r/74245/diff/1-2/
Testing (updated)
-------
Tested Created policies via load testing
Thanks,
Ramachandran Krishnan