You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2019/10/05 22:27:12 UTC
Review Request 71584: RANGER-2510: Support for Incremental tag
updates to
improve performance - handle updates to tag policies correctly
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/
-----------------------------------------------------------
Review request for ranger and Madhan Neethiraj.
Summary (updated)
-----------------
RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly
Bugs: RANGER-2510
https://issues.apache.org/jira/browse/RANGER-2510
Repository: ranger
Description (updated)
-------
Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
Diff: https://reviews.apache.org/r/71584/diff/1/
Testing (updated)
-------
Tested:
1. policy-cache is correctly updated.
2. Policy evaluation when tag policies are updated.
3. Policy evaluation when tags are updated.
Thanks,
Abhay Kulkarni
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
> On Oct. 6, 2019, 5:26 a.m., Ramesh Mani wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
> > Lines 106 (patched)
> > <https://reviews.apache.org/r/71584/diff/1/?file=2168192#file2168192line106>
> >
> > Is there a reason why we cannot have setIsShared in in RangerPolicyEngine Interface?
setIsShared() is an internal housekeeping function, so it not appropriate to put it in the interface which others may refer to.
- Abhay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218096
-----------------------------------------------------------
On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 5, 2019, 10:27 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/1/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218096
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
Lines 106 (patched)
<https://reviews.apache.org/r/71584/#comment305631>
Is there a reason why we cannot have setIsShared in in RangerPolicyEngine Interface?
- Ramesh Mani
On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 5, 2019, 10:27 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/1/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218101
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
Lines 82 (patched)
<https://reviews.apache.org/r/71584/#comment305636>
It will help to not to make that assumption here. Instead consider the following:
this.policyEngine = policyEngine;
if (other != null) {
Map<RangerContextEnricher, Object> localReference = other.requestContextEnrichers;
this.rangerPluginContext = other.rangerPluginContext;
this.requestContextEnrichers = MapUtils.isNotEmpty(localReference) ? new ConcurrentHashMap<>(localReference) : new ConcurrentHashMap<>();
} else {
this.rangerPluginContext = null;
this.requestContextEnrichers = new ConcurrentHashMap<>();
}
- Madhan Neethiraj
On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 5, 2019, 10:27 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/1/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
> On Oct. 6, 2019, 4:13 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
> > Lines 82 (patched)
> > <https://reviews.apache.org/r/71584/diff/1/?file=2168195#file2168195line82>
> >
> > when 'other' is null but 'policyEngine' is not (refer to call from RangerPolicyEngineImpl.java #230), this.policyEngine is set to null #92. This perhaps isn't right? Please review.
The condition that policyEngine not null and other is null will never happen when a policy-engine is being constructed using policy-deltas from an existing policy-engine. Please refer to (RangerPolicyEngineImpl.java #269). I can take out the check in RangerAuthContext.java construction, if that is confusing.
- Abhay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218098
-----------------------------------------------------------
On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 5, 2019, 10:27 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/1/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218098
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
Lines 82 (patched)
<https://reviews.apache.org/r/71584/#comment305633>
when 'other' is null but 'policyEngine' is not (refer to call from RangerPolicyEngineImpl.java #230), this.policyEngine is set to null #92. This perhaps isn't right? Please review.
- Madhan Neethiraj
On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 5, 2019, 10:27 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/1/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218103
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Oct. 6, 2019, 11:07 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> -----------------------------------------------------------
>
> (Updated Oct. 6, 2019, 11:07 p.m.)
>
>
> Review request for ranger and Madhan Neethiraj.
>
>
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
>
>
> Diff: https://reviews.apache.org/r/71584/diff/2/
>
>
> Testing
> -------
>
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 71584: RANGER-2510: Support for Incremental tag
updates
to improve performance - handle updates to tag policies correctly
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/
-----------------------------------------------------------
(Updated Oct. 6, 2019, 11:07 p.m.)
Review request for ranger and Madhan Neethiraj.
Changes
-------
Addressed review comments
Bugs: RANGER-2510
https://issues.apache.org/jira/browse/RANGER-2510
Repository: ranger
Description
-------
Ensure that policy cache is correctly updated when only resource policy is updated. Also, ensure that updated policy-engine is used for evaluation when policy deltas are used to build policy engine.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java 5dae0c12b
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java 576d5e5bb
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java ae88c73ea
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java 6cd1df69e
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 1325a4020
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java 9c50f8a33
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java ef5f1d53f
agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 596f5e841
agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java f6beac675
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 51e08e14b
Diff: https://reviews.apache.org/r/71584/diff/2/
Changes: https://reviews.apache.org/r/71584/diff/1-2/
Testing
-------
Tested:
1. policy-cache is correctly updated.
2. Policy evaluation when tag policies are updated.
3. Policy evaluation when tags are updated.
Thanks,
Abhay Kulkarni