You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Radhika Kundam <ra...@gmail.com> on 2025/02/10 23:38:32 UTC

Review Request 75360: RANGER-5129: Ranger Dataset search using a date range filter on ValiditySchedule.endDate

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/
-----------------------------------------------------------

Review request for ranger, Madhan Neethiraj and Ramesh Mani.


Bugs: RANGER-5129
    https://issues.apache.org/jira/browse/RANGER-5129


Repository: ranger


Description
-------

Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.

Filtering Criteria:

The dataset must have a validity schedule.
The expiration date of the dataset should be within the provided range:
   validityExpiryFrom – Start of the expiration date range
   validityExpiryTo – End of the expiration date range
This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.

API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryFrom=<yyyy/MM/dd HH:mm:ss>&validityExpiryTo=<yyyy/MM/dd HH:mm:ss>


Diffs
-----

  agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
  security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
  security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 


Diff: https://reviews.apache.org/r/75360/diff/1/


Testing
-------

Tested locally.


Thanks,

Radhika Kundam


Re: Review Request 75360: RANGER-5129: Ranger Dataset search using a date range filter on ValiditySchedule.endDate

Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/#review227243
-----------------------------------------------------------




agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java
Lines 143 (patched)
<https://reviews.apache.org/r/75360/#comment315483>

    validityExpiryFrom => validityExpiryStart
    validityExpiryTo   => validityExpiryEnd



security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1814 (patched)
<https://reviews.apache.org/r/75360/#comment315484>

    "Error while filtering datasets by validity schedule:" => "Error in finding datasets expiring between '" + validityCheckFrom + "' and '" + validityCheckTo + "': "



security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1818 (patched)
<https://reviews.apache.org/r/75360/#comment315486>

    For better readability, I suggest renaming:
    
    isDatasetNotExpiring() => datasetExpires(RangerDataset dataset, RangerValidityScheduleEvaluator expiryPeriod, List<ValidationFailureDetails> failures)
    
    And reverse the condition in #1810.



security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1827 (patched)
<https://reviews.apache.org/r/75360/#comment315485>

    Each call to isDatasetNotExpiring() from #1810 will instantiate the same RangerValidityScheduleEvaluator. To avoid repeated instantiation overheads, consider instantiating in filterDatasetsByValidityExpiration() and send validityScheduleEvaluator instead of validityCheckFilter.
    
    Also, for better readability, I suggest renaming isDatasetNotExpiring() to:
    
    boolean isExpiringDataset(RangerDataset dataset, RangerValidityScheduleEvaluator expiryPeriod, List<ValidationFailureDetails> failures)
    
    And reverse the condition in #1810:
      datasets.removeIf(dataset -> !isExpiringDataset(dataset, expiryPeriod, failures));


- Madhan Neethiraj


On Feb. 10, 2025, 11:38 p.m., Radhika Kundam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75360/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2025, 11:38 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-5129
>     https://issues.apache.org/jira/browse/RANGER-5129
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.
> 
> Filtering Criteria:
> 
> The dataset must have a validity schedule.
> The expiration date of the dataset should be within the provided range:
>    validityExpiryFrom – Start of the expiration date range
>    validityExpiryTo – End of the expiration date range
> This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.
> 
> API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryFrom=<yyyy/MM/dd HH:mm:ss>&validityExpiryTo=<yyyy/MM/dd HH:mm:ss>
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
>   security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
>   security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 
> 
> 
> Diff: https://reviews.apache.org/r/75360/diff/1/
> 
> 
> Testing
> -------
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Radhika Kundam
> 
>


Re: Review Request 75360: RANGER-5129: Ranger Dataset search using a date range filter on ValiditySchedule.endDate

Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/#review227244
-----------------------------------------------------------




security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1801 (patched)
<https://reviews.apache.org/r/75360/#comment315487>

    Please add debug statements for these new methods



security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1849 (patched)
<https://reviews.apache.org/r/75360/#comment315488>

    nit: Please have a space after "," as this may fail the verify pmd check.


- Ramesh Mani


On Feb. 10, 2025, 11:38 p.m., Radhika Kundam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75360/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2025, 11:38 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-5129
>     https://issues.apache.org/jira/browse/RANGER-5129
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.
> 
> Filtering Criteria:
> 
> The dataset must have a validity schedule.
> The expiration date of the dataset should be within the provided range:
>    validityExpiryFrom – Start of the expiration date range
>    validityExpiryTo – End of the expiration date range
> This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.
> 
> API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryFrom=<yyyy/MM/dd HH:mm:ss>&validityExpiryTo=<yyyy/MM/dd HH:mm:ss>
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
>   security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
>   security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 
> 
> 
> Diff: https://reviews.apache.org/r/75360/diff/1/
> 
> 
> Testing
> -------
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Radhika Kundam
> 
>


Re: Review Request 75360: RANGER-5129: Enhance Dataset Filtering by Validity Schedule and Expiration Date Range

Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/#review227251
-----------------------------------------------------------


Ship it!




Ship It!

- Ramesh Mani


On Feb. 11, 2025, 6:26 p.m., Radhika Kundam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75360/
> -----------------------------------------------------------
> 
> (Updated Feb. 11, 2025, 6:26 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-5129
>     https://issues.apache.org/jira/browse/RANGER-5129
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.
> 
> Filtering Criteria:
> 
> The dataset must have a validity schedule.
> The expiration date of the dataset should be within the provided range:
>    validityExpiryFrom – Start of the expiration date range
>    validityExpiryTo – End of the expiration date range
> This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.
> 
> API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryStart=<yyyy/MM/dd HH:mm:ss>&validityExpiryEnd=<yyyy/MM/dd HH:mm:ss>
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
>   security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
>   security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 
> 
> 
> Diff: https://reviews.apache.org/r/75360/diff/2/
> 
> 
> Testing
> -------
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Radhika Kundam
> 
>


Re: Review Request 75360: RANGER-5129: Enhance Dataset Filtering by Validity Schedule and Expiration Date Range

Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/#review227250
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On Feb. 11, 2025, 6:26 p.m., Radhika Kundam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75360/
> -----------------------------------------------------------
> 
> (Updated Feb. 11, 2025, 6:26 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-5129
>     https://issues.apache.org/jira/browse/RANGER-5129
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.
> 
> Filtering Criteria:
> 
> The dataset must have a validity schedule.
> The expiration date of the dataset should be within the provided range:
>    validityExpiryFrom – Start of the expiration date range
>    validityExpiryTo – End of the expiration date range
> This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.
> 
> API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryStart=<yyyy/MM/dd HH:mm:ss>&validityExpiryEnd=<yyyy/MM/dd HH:mm:ss>
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
>   security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
>   security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 
> 
> 
> Diff: https://reviews.apache.org/r/75360/diff/2/
> 
> 
> Testing
> -------
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Radhika Kundam
> 
>


Re: Review Request 75360: RANGER-5129: Enhance Dataset Filtering by Validity Schedule and Expiration Date Range

Posted by Radhika Kundam <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/
-----------------------------------------------------------

(Updated Feb. 11, 2025, 10:26 a.m.)


Review request for ranger, Madhan Neethiraj and Ramesh Mani.


Changes
-------

Updated patch as per the review comments.


Summary (updated)
-----------------

RANGER-5129: Enhance Dataset Filtering by Validity Schedule and Expiration Date Range


Bugs: RANGER-5129
    https://issues.apache.org/jira/browse/RANGER-5129


Repository: ranger


Description
-------

Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.

Filtering Criteria:

The dataset must have a validity schedule.
The expiration date of the dataset should be within the provided range:
   validityExpiryFrom – Start of the expiration date range
   validityExpiryTo – End of the expiration date range
This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.

API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryFrom=<yyyy/MM/dd HH:mm:ss>&validityExpiryTo=<yyyy/MM/dd HH:mm:ss>


Diffs (updated)
-----

  agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
  security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
  security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 


Diff: https://reviews.apache.org/r/75360/diff/2/

Changes: https://reviews.apache.org/r/75360/diff/1-2/


Testing
-------

Tested locally.


Thanks,

Radhika Kundam


Re: Review Request 75360: RANGER-5129: Ranger Dataset search using a date range filter on ValiditySchedule.endDate

Posted by Vyom Tiwari via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75360/#review227246
-----------------------------------------------------------




security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java
Lines 1844 (patched)
<https://reviews.apache.org/r/75360/#comment315490>

    Do we have a test where we have both  'to' and 'from' empty ?


- Vyom Tiwari


On Feb. 10, 2025, 11:38 p.m., Radhika Kundam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75360/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2025, 11:38 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-5129
>     https://issues.apache.org/jira/browse/RANGER-5129
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Enhance the existing enhancedSummary API to support filtering datasets based on their validity schedule and expiration date. The API should return datasets that have a defined validity schedule and whose expiration date falls within the specified date range.
> 
> Filtering Criteria:
> 
> The dataset must have a validity schedule.
> The expiration date of the dataset should be within the provided range:
>    validityExpiryFrom – Start of the expiration date range
>    validityExpiryTo – End of the expiration date range
> This enhancement ensures that only relevant datasets with scheduled validity and expiring within the given timeframe are retrieved.
> 
> API: curl GET <ranger-admin url>/service/gds/dataset/enhancedsummary?validityExpiryFrom=<yyyy/MM/dd HH:mm:ss>&validityExpiryTo=<yyyy/MM/dd HH:mm:ss>
> 
> 
> Diffs
> -----
> 
>   agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java 027c2140c 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java a260a8782 
>   security-admin/src/main/java/org/apache/ranger/biz/GdsDBStore.java 312a67a28 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 6c93d7104 
>   security-admin/src/test/java/org/apache/ranger/rest/TestGdsREST.java 68d62fc31 
> 
> 
> Diff: https://reviews.apache.org/r/75360/diff/1/
> 
> 
> Testing
> -------
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Radhika Kundam
> 
>