[jira] [Updated] (RANGER-3900) Roles deletion Takes time in Apache Ranger when there are more users,groups,roles

["Ramachandran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RANGER-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ramachandran updated RANGER-3900:
---------------------------------
    Description: 
*{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff} {color}

 
{code:java}
 
1. Getting XXRole by roleName:
XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
2. Ensuring role is not in policy 
Long roleRefPolicyCount = daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
3. Ensuring role is not in another role
Long roleRefRoleCount = daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
4. Fetching RangerRole by id
RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
5. Dereferencing all the users from the role.
 for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB Read call
xRoleUserDao.remove(xxRoleRefUser);   → 1DB Write call for each xxRoleRefUser
}
6. Dereferencing all the groups from the role
for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { → 1DB Read call
xRoleGroupDao.remove(xxRoleRefGroup);  → 1DB Write call for each xxRoleRefGroup
}
7. Dereferencing all other roles from the role.
for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB Read call
xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
}
8. Delete the rangerRole in DB
  roleService.delete(role);  → 1DB Write call

9. Create TrxLog
bizUtil.createTrxLog(trxLogList) → 1DB Write call
 
{code}
 

*{color:#0000ff}Total number of DB calls involved for the below role deletion in Apache Ranger:{color}*

 

*Deleting the role which contains more users:*

1. role contains 100 users 
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|102|6|0|1.01 seconds|

 

1 role contains 500 users  
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|502|6|0| 2.241 seconds|

1 role contains 1000 users  
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|1002|6|0|3.14 seconds|

 
 
{color:#ff0000}Recommendation :{color}

When we reduce the number of users added to the role -→ Total number of DB calls will be reduced 

Instead of adding roles to the users, can we add those users into the group. And then link that group to the role

if we use groups instead of directly adding users into the role, the total number of DB write calls will be reduced to 2 from 1002  for 1k users 

 

{color:#ff0000}Proposal{color}:

{color:#ff0000}We can try a DB batch write call instead of 1 DB write call for every user deference{color}

 

After Fix

1 role contains 500 users  
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
|3|6|0|0.666 seconds  |

1 role contains 1000 users  
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
|3|6|0|0.835 seconds|

 

1 role contains 1000 users ,1000 roles,1000 groups
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
|5|6|0|1.021  seconds|

 

  was:
*{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff} {color}

 
{code:java}
 
1. Getting XXRole by roleName:
XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
2. Ensuring role is not in policy 
Long roleRefPolicyCount = daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
3. Ensuring role is not in another role
Long roleRefRoleCount = daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
4. Fetching RangerRole by id
RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
5. Dereferencing all the users from the role.
 for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB Read call
xRoleUserDao.remove(xxRoleRefUser);   → 1DB Write call for each xxRoleRefUser
}
6. Dereferencing all the groups from the role
for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { → 1DB Read call
xRoleGroupDao.remove(xxRoleRefGroup);  → 1DB Write call for each xxRoleRefGroup
}
7. Dereferencing all other roles from the role.
for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB Read call
xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
}
8. Delete the rangerRole in DB
  roleService.delete(role);  → 1DB Write call

9. Create TrxLog
bizUtil.createTrxLog(trxLogList) → 1DB Write call
 
{code}
 

*{color:#0000ff}Total number of DB calls involved for the below role deletion in Apache Ranger:{color}*

 

*Deleting the role which contains more users:*

1. role contains 100 users 
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|102|6|0|1.01 seconds|

 

1 role contains 500 users  
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|502|6|0| 2.241 seconds|

1 role contains 1000 users  
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
|1002|6|0|3.14 seconds|

 

*Deleting the user (Which was associated with the role . Now that role got deleted)*

 
||Number of users ||Time is taken to delete each user||
|1000| 0.8 to 1.5  seconds|

 

*Deleting the user (Which was not associated with any role)*
||Number of users ||Time is taken to delete each user||
|1000| 0.8 to 1.2  seconds|

{color:#FF0000}Recommendation :{color}

When we reduce the number of users added to the role -→ Total number of DB calls will be reduced 

Instead of adding roles to the users, can we add those users into the group. And then link that group to the role

if we use groups instead of directly adding users into the role, the total number of DB write call will be reduced to 2 from 1002  for 1k users 

 

{color:#ff0000}Proposal{color}:

{color:#ff0000}We can try DB batch write call instead of 1 DB write call for every user deference {color}

 


> Roles deletion Takes time in Apache Ranger when there are more users,groups,roles
> ---------------------------------------------------------------------------------
>
>                 Key: RANGER-3900
>                 URL: https://issues.apache.org/jira/browse/RANGER-3900
>             Project: Ranger
>          Issue Type: Improvement
>          Components: Ranger
>    Affects Versions: 3.0.0
>            Reporter: Ramachandran
>            Assignee: Ramachandran
>            Priority: Major
>
> *{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff} {color}
>  
> {code:java}
>  
> 1. Getting XXRole by roleName:
> XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
> 2. Ensuring role is not in policy 
> Long roleRefPolicyCount = daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
> 3. Ensuring role is not in another role
> Long roleRefRoleCount = daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
> 4. Fetching RangerRole by id
> RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
> 5. Dereferencing all the users from the role.
>  for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB Read call
> xRoleUserDao.remove(xxRoleRefUser);   → 1DB Write call for each xxRoleRefUser
> }
> 6. Dereferencing all the groups from the role
> for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { → 1DB Read call
> xRoleGroupDao.remove(xxRoleRefGroup);  → 1DB Write call for each xxRoleRefGroup
> }
> 7. Dereferencing all other roles from the role.
> for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB Read call
> xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
> }
> 8. Delete the rangerRole in DB
>   roleService.delete(role);  → 1DB Write call
> 9. Create TrxLog
> bizUtil.createTrxLog(trxLogList) → 1DB Write call
>  
> {code}
>  
> *{color:#0000ff}Total number of DB calls involved for the below role deletion in Apache Ranger:{color}*
>  
> *Deleting the role which contains more users:*
> 1. role contains 100 users 
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |102|6|0|1.01 seconds|
>  
> 1 role contains 500 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |502|6|0| 2.241 seconds|
> 1 role contains 1000 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete the role||
> |1002|6|0|3.14 seconds|
>  
>  
> {color:#ff0000}Recommendation :{color}
> When we reduce the number of users added to the role -→ Total number of DB calls will be reduced 
> Instead of adding roles to the users, can we add those users into the group. And then link that group to the role
> if we use groups instead of directly adding users into the role, the total number of DB write calls will be reduced to 2 from 1002  for 1k users 
>  
> {color:#ff0000}Proposal{color}:
> {color:#ff0000}We can try a DB batch write call instead of 1 DB write call for every user deference{color}
>  
> After Fix
> 1 role contains 500 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |3|6|0|0.666 seconds  |
> 1 role contains 1000 users  
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |3|6|0|0.835 seconds|
>  
> 1 role contains 1000 users ,1000 roles,1000 groups
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the role||
> |5|6|0|1.021  seconds|
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)