You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2023/03/28 03:35:04 UTC

Review Request 74366: RANGER-4136: Incorrect processing of tag-deltas by RangerTagEnricher - Part 2

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74366/
-----------------------------------------------------------

Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.

Bugs: RANGER-4136
https://issues.apache.org/jira/browse/RANGER-4136

Repository: ranger

Description
-------

Ranger supports incremental tag downloads when configured. When a mapping for a service-resource is changed, the tag download request provides only the updated service-resource. RangerTagEnricher is expected to remove the existing service-resource and the associated tags from its cache and add it back with updated associated tags.

Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource.

More debugging help is added to monitor the process flow for the incremental tag processing.

Diffs
-----

agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java 198d24d97
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java b5b26702c
agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceTags.java 9b902789a

Diff: https://reviews.apache.org/r/74366/diff/1/

Testing
-------

Passed all unit tests. Verified that incremental tagged entities are correctly processed and properly cleaned up.

Thanks,

Abhay Kulkarni

Re: Review Request 74366: RANGER-4136: Incorrect processing of tag-deltas by RangerTagEnricher - Part 2

Posted by Abhay Kulkarni <ak...@hortonworks.com>.


> On April 4, 2023, 6:54 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
> > Lines 582 (patched)
> > <https://reviews.apache.org/r/74366/diff/3/?file=2275535#file2275535line593>
> >
> >     I would suggest to avoid updating oldMatchers, as this may be a collection referenced within enrichedServiceTags. Instead, consider handling skipping of entries in notMatched within the following for loop at #585.

oldMatchers need to be updated, otherwise, line 602 will remove notMatched service-resource-matchers from resourceMatchers passed to this function, which will remove non-SELF matched matchers from the Trie indexes as well.


- Abhay


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74366/#review225350
-----------------------------------------------------------


On April 4, 2023, 5:52 p.m., Abhay Kulkarni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74366/
> -----------------------------------------------------------
> 
> (Updated April 4, 2023, 5:52 p.m.)
> 
> 
> Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4136
>     https://issues.apache.org/jira/browse/RANGER-4136
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Ranger supports incremental tag downloads when configured. When a mapping for a service-resource is changed, the tag download request provides only the updated service-resource. RangerTagEnricher is expected to remove the existing service-resource and the associated tags from its cache and add it back with updated associated tags.
> 
> Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource.
> 
> More debugging help is added to monitor the process flow for the incremental tag processing.
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java 198d24d97 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceTags.java 9b902789a 
> 
> 
> Diff: https://reviews.apache.org/r/74366/diff/3/
> 
> 
> Testing
> -------
> 
> Passed all unit tests. Verified that incremental tagged entities are correctly processed and properly cleaned up.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 74366: RANGER-4136: Incorrect processing of tag-deltas by RangerTagEnricher - Part 2

Posted by Madhan Neethiraj <ma...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74366/#review225350
-----------------------------------------------------------


Fix it, then Ship it!




Given the complex and critical nature of handling deltas, consider adding unit tests. This can be tracked with a separate JIRA.


agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
Lines 582 (patched)
<https://reviews.apache.org/r/74366/#comment314004>

    I would suggest to avoid updating oldMatchers, as this may be a collection referenced within enrichedServiceTags. Instead, consider handling skipping of entries in notMatched within the following for loop at #585.


- Madhan Neethiraj


On April 4, 2023, 5:52 p.m., Abhay Kulkarni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74366/
> -----------------------------------------------------------
> 
> (Updated April 4, 2023, 5:52 p.m.)
> 
> 
> Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4136
>     https://issues.apache.org/jira/browse/RANGER-4136
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Ranger supports incremental tag downloads when configured. When a mapping for a service-resource is changed, the tag download request provides only the updated service-resource. RangerTagEnricher is expected to remove the existing service-resource and the associated tags from its cache and add it back with updated associated tags.
> 
> Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource.
> 
> More debugging help is added to monitor the process flow for the incremental tag processing.
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java 198d24d97 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceTags.java 9b902789a 
> 
> 
> Diff: https://reviews.apache.org/r/74366/diff/3/
> 
> 
> Testing
> -------
> 
> Passed all unit tests. Verified that incremental tagged entities are correctly processed and properly cleaned up.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 74366: RANGER-4136: Incorrect processing of tag-deltas by RangerTagEnricher - Part 2

Posted by Abhay Kulkarni <ak...@hortonworks.com>.

(Updated April 4, 2023, 5:52 p.m.)

Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.

Bugs: RANGER-4136
https://issues.apache.org/jira/browse/RANGER-4136

Repository: ranger

Description
-------

Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource.

More debugging help is added to monitor the process flow for the incremental tag processing.

Diffs (updated)
-----

agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java 198d24d97
agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceTags.java 9b902789a

Diff: https://reviews.apache.org/r/74366/diff/3/

Changes: https://reviews.apache.org/r/74366/diff/2-3/

Testing
-------

Passed all unit tests. Verified that incremental tagged entities are correctly processed and properly cleaned up.

Thanks,

Abhay Kulkarni

Re: Review Request 74366: RANGER-4136: Incorrect processing of tag-deltas by RangerTagEnricher - Part 2

Posted by Abhay Kulkarni <ak...@hortonworks.com>.

(Updated March 29, 2023, 1:46 a.m.)

Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.

Bugs: RANGER-4136
https://issues.apache.org/jira/browse/RANGER-4136

Repository: ranger

Description
-------

Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource.

More debugging help is added to monitor the process flow for the incremental tag processing.

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/74366/diff/2/

Changes: https://reviews.apache.org/r/74366/diff/1-2/

Testing
-------

Passed all unit tests. Verified that incremental tagged entities are correctly processed and properly cleaned up.

Thanks,

Abhay Kulkarni