[jira] [Created] (RANGER-3899) Policy creation takes more when policy contains more users

["Ramachandran (Jira)" <ji...@apache.org>]

Ramachandran created RANGER-3899:
------------------------------------

             Summary: Policy creation takes more when policy contains more users 
                 Key: RANGER-3899
                 URL: https://issues.apache.org/jira/browse/RANGER-3899
             Project: Ranger
          Issue Type: Improvement
          Components: Ranger
    Affects Versions: 3.0.0
            Reporter: Ramachandran


*{color:#0000ff}Policy Creation Steps in Apache Ranger:{color}*{color:#0000ff} {color}
||{color:#0000ff}1. Get the service by Name:{color}
{color:#800080}RangerService service = getServiceByName(policy.getService());{*}→ {color:#ff0000}1DB Read call{color}{*}{color}
{color:#0000ff}2. Get XXServiceDef by Name{color}
{color:#800080}XXServiceDef xServiceDef = daoMgr.getXXServiceDef().findByName(service.getType());{color} {color:#800080}*→ {color:#ff0000}1DB Read call{color}*{color}
{color:#0000ff}3. Get the existing XXPolicy by Name{color}
{color:#800080}XXPolicy existing = daoMgr.getXXPolicy().findByNameAndServiceIdAndZoneId(policy.getName(), service.getId(), zoneId);{color} {color:#800080}*→ {color:#ff0000}1DB Read call{color}*{color}
{color:#0000ff}4. Create a policy{color}
{color:#800080}policy = policyService.create(policy, true); *→ {color:#ff0000}1DB Write call{color}*{color}
{color:#0000ff}5. Get policy by Id{color}
{color:#800000}XXPolicy xCreatedPolicy = daoMgr.getXXPolicy().getById(policy.getId());{color} {color:#800080}*→ {color:#ff0000}1DB Read call{color}*{color}
{color:#0000ff}6. createObjectDataHistory for the newly created policy{color}
{color:#800000}dataHistService.createObjectDataHistory(createdPolicy, RangerDataHistService.ACTION_CREATE);{color} {color:#800080}*→ {color:#ff0000}1DB Write call{color}*{color}
{color:#0000ff}7. createTrxLog for the newly created policy{color}
{color:#993300}bizUtil.createTrxLog(trxLogList);{color} {color:#800080}*→ {color:#ff0000}1DB Write call{color}*{color}
{color:#0000ff}8. Create Policy Label for the newly created policy{color}
createOrMapLabels(xCreatedPolicy, uniquePolicyLabels); {color:#800080}*→ {color:#ff0000}1DB Write call{color}*{color}
{color:#0000ff}9. Create Policy Mapping For Ref Table{color}
policyRefUpdater.createNewPolMappingForRefTable(policy, xCreatedPolicy, xServiceDef);
{color:#0000ff}1. Getting roleNames,groupNames,userNames,accessTypes,dataMaskTypes,conditionTypes from each policy items of the policy{color}
{color:#0000ff}2. Getting resourceNames from the policy{color}
3.{color:#0000ff}For each resource{color}
  {color:#0000ff}Getting the XXResourceDef by resource and policy.getId() {color}
  {color:#993300}XXResourceDef xResDef = daoMgr.getXXResourceDef().findByNameAndPolicyId(resource, policy.getId());{color} {color:#800080}*→ {color:#ff0000}1DB Read call for each resource{color}*{color}
  {color:#0000ff}populate XXPolicyRefResource{color}
{color:#0000ff}4. Create the batch of XXPolicyRefResource in DB{color} 
  {color:#993300}daoMgr.getXXPolicyRefResource().batchCreate(xPolResources);{color} {color:#800080}*→ {color:#ff0000}1DB Batch Write call{color}*{color}
{color:#0000ff}5.For each roleNames {color}
{color:#0000ff}   Getting XXRole by roleName{color}
   {color:#993300}XXRole xRole = daoMgr.getXXRole().findByRoleName(role);{color} {color:#800080}*→ {color:#ff0000}1DB Read call for each role{color}*{color}
{color:#0000ff}   populate XXPolicyRefRole{color}
{color:#0000ff}6. Create the batch of XXPolicyRefRole in DB{color}
    {color:#993300}daoMgr.getXXPolicyRefResource().batchCreate(xPolRoles); {color}{color:#800080}*→ {color:#ff0000}1DB Batch Write call{color}*{color}
{color:#0000ff}7.For each groupNames{color}
{color:#0000ff}Getting the XXGroup by groupName{color}
{color:#993300}XXGroup xGroup = daoMgr.getXXGroup().findByGroupName(context.group.getName());{color}{color:#800080}*→ {color:#ff0000}1DB Read call for each group{color}*{color}
{color:#0000ff}populate XXPolicyRefGroup{color}
{color:#0000ff}Insert into DB{color}
{color:#993300}daoMgr.getXXPolicyRefGroup().create(xPolGroup);{color:#800080}*→ {color:#ff0000}1DB write call for each group{color}*{color}{color}
{color:#0000ff}8.For each userNames {color}
{color:#0000ff}Getting XXUser by userName{color}
{color:#993300}XXUser xUser = daoMgr.getXXUser().findByUserName(user);{color} {color:#800080}*→ {color:#ff0000}1DB Read call for each user{color}*{color}
{color:#0000ff}populate XXPolicyRefUser{color}
{color:#993300}daoMgr.getXXPolicyRefUser().create(xPolUser); {color:#800080}*→ {color:#ff0000}1DB write call for each user{color}*{color}{color}
{color:#0000ff}9.For each accessTypes{color}
{color:#0000ff}Getting the XXAccessTypeDef by accessType and xPolicy.getService() {color}
{color:#800080}XXAccessTypeDef xAccTypeDef = daoMgr.getXXAccessTypeDef().findByNameAndServiceId(accessType, xPolicy.getService()); *→ {color:#ff0000}1DB Read call for each accesType{color}*{color}
{color:#0000ff}populate XXAccessTypeDef{color}
{color:#0000ff}10. Create the batch of xPolAccesses in DB{color}
{color:#993300}daoMgr.getXXPolicyRefAccessType().batchCreate(xPolAccesses);{color} {color:#800080}*→ {color:#ff0000}1DB Batch Write call{color}*{color}
{color:#0000ff}11.For each conditionTypes{color}
{color:#0000ff}Getting the XXPolicyConditionDef by condition and xServiceDef.getId(){color}
{color:#993300}XXPolicyConditionDef xPolCondDef = daoMgr.getXXPolicyConditionDef().findByServiceDefIdAndName(xServiceDef.getId(), condition){color}; {color:#800080}*→ {color:#ff0000}1DB Read call for each conditionType{color}*{color}
{color:#0000ff}populate XXPolicyConditionDef{color}
{color:#0000ff}12. Create the batch of xPolConds in DB{color}
{color:#993300}daoMgr.getXXPolicyRefCondition().batchCreate(xPolConds);{color}  {color:#800080}*→ {color:#ff0000}1DB Batch Write call{color}*{color}||

*{color:#0000ff}Total number of DB calls involved for the below Policy creation in Apache Ranger:{color}*

1.Policy contains 500 users,5 access Types (permissions),12 resources (1DB,1Table,10 columns)

     
||DB Write count||DB Read count||DB Batch Write count||Time taken to create the policy||
|504|526|3|{color:#ff0000}4~8 seconds{color}|

 

 2.Policy contains 50 roles,5 access Types (permissions),12 resources (1DB,1Table,10 columns)

 
||DB Write count||DB Read count||DB Batch Write count||Time taken to create the policy||
|4|76|4|{color:#ff0000}~1 second{color}|

 

{color:#ff0000}Note :{color}

When we reduce the number of users added into policy directly -→ Total number of DB calls will be reduced 

We can use {color:#008000}roles or groups {color} into the policy instead of adding users directly

{color:#ff0000}Proposal :{color}

{color:#ff0000}We can try {color:#993300}{color:#800080}*{color:#ff0000}DB batch write call instead of 1 DB write call for every user{color}*{color}{color}{color}

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)