[jira] [Resolved] (RANGER-3900) Roles deletion Takes time in Apache Ranger when there are more users,groups,roles

["Madhan Neethiraj (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RANGER-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Madhan Neethiraj resolved RANGER-3900.
--------------------------------------
    Fix Version/s: 3.0.0
                   2.4.0
       Resolution: Fixed

[~ramackri] - thank you for the patch. The patch has been committed in the following branches:

{noformat}
commit 017c7402773f069ff576785a1d51ac790d63e2c7 (HEAD -> master, origin/master, origin/HEAD)
Author: Ramachandran Krishnan <ra...@gmail.com>
Date:   Tue Oct 11 08:58:53 2022 +0530

    RANGER-3900: performance improvement in roles deletion

    Signed-off-by: Madhan Neethiraj <ma...@apache.org>
{noformat}


{noformat}
commit 9b81ac05d75596f92b12183fda452a871c959f62 (HEAD -> ranger-2.4, origin/ranger-2.4)
Author: Ramachandran Krishnan <ra...@gmail.com>
Date:   Tue Oct 11 08:58:53 2022 +0530

    RANGER-3900: performance improvement in roles deletion

    Signed-off-by: Madhan Neethiraj <ma...@apache.org>
    (cherry picked from commit 017c7402773f069ff576785a1d51ac790d63e2c7)
{noformat}

> Roles deletion Takes time in Apache Ranger when there are more users,groups,roles
> ---------------------------------------------------------------------------------
>
>                 Key: RANGER-3900
>                 URL: https://issues.apache.org/jira/browse/RANGER-3900
>             Project: Ranger
>          Issue Type: Improvement
>          Components: Ranger
>    Affects Versions: 3.0.0
>            Reporter: Ramachandran
>            Assignee: Ramachandran
>            Priority: Major
>             Fix For: 3.0.0, 2.4.0
>
>
> *{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff} {color}
>  
> {code:java}
>  
> 1. Getting XXRole by roleName:
> XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
> 2. Ensuring role is not in policy 
> Long roleRefPolicyCount = daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
> 3. Ensuring role is not in another role
> Long roleRefRoleCount = daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
> 4. Fetching RangerRole by id
> RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
> 5. Dereferencing all the users from the role.
>  for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB Read call
> xRoleUserDao.remove(xxRoleRefUser);   → 1DB Write call for each xxRoleRefUser
> }
> 6. Dereferencing all the groups from the role
> for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { → 1DB Read call
> xRoleGroupDao.remove(xxRoleRefGroup);  → 1DB Write call for each xxRoleRefGroup
> }
> 7. Dereferencing all other roles from the role.
> for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB Read call
> xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
> }
> 8. Delete the rangerRole in DB
>   roleService.delete(role);  → 1DB Write call
> 9. Create TrxLog
> bizUtil.createTrxLog(trxLogList) → 1DB Write call
>  
> {code}
>  
> *{color:#0000ff}Total number of DB calls involved for the below role deletion in Apache Ranger:{color}*
>  
> *Deleting the role which contains more users:*
> 1. role contains 100 users 
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |102|6|0|1.01 seconds|
>  
> 1 role contains 500 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |502|6|0| 2.241 seconds|
> 1 role contains 1000 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |1002|6|0|3.14 seconds|
>  
> 1 role contains 1000 users ,1000 roles,1000 groups
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |3002|6|0|11.577  seconds|
>  
> {color:#ff0000}Proposal{color}:
> {color:#ff0000}We can try a DB batch write call instead of 1 DB write call for every user deference{color}
>  
> After Fix
> 1 role contains 500 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |3|6|0|0.666 seconds  |
> 1 role contains 1000 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |3|6|0|0.835 seconds|
>  
> 1 role contains 1000 users ,1000 roles,1000 groups
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |5|6|0|1.021  seconds|
> {color:#ff0000}Recommendation :{color}
> When we reduce the number of users added to the role -→ Total number of DB calls will be reduced 
> Instead of adding roles to the users, can we add those users into the group. And then link that group to the role
> if we use groups instead of directly adding users into the role, the total number of DB write calls will be reduced to 2 from 1002  for 1k users 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)