You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2017/02/14 05:32:16 UTC
Review Request 56640: Support for using resource-matcher for filtering
policies within a service if service-resource is provided in the filter
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
> On Feb. 20, 2017, 4:35 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java, line 51
> > <https://reviews.apache.org/r/56640/diff/3/?file=1638039#file1638039line51>
> >
> > Consider renaming: "getServiceDefForNonrecursivePathResourceMatchers" ==> "cloneServiceDefForResourceFiltering"
Renamed to "getServiceDefForPolicyFiltering"
> On Feb. 20, 2017, 4:35 a.m., Madhan Neethiraj wrote:
> > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java, line 2151
> > <https://reviews.apache.org/r/56640/diff/3/?file=1638044#file1638044line2151>
> >
> > Consider abstracting the details of updating resourceName in a method like: updateResourceForFilter(filterResources, serviceDef)
Method: "RangerServiceDefHelper.getFilterResourcesForPolicyFiltering(serviceDef, filterResources)"
- Abhay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166043
-----------------------------------------------------------
On Feb. 16, 2017, 10:45 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2017, 10:45 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
>
> A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
>
> The values supported for "resourceMatchScope" option are:
>
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
>
> If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
>
> An example curl command is as follows:
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
>
> This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
>
> Similarly, a command
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
>
> will return only policies which have both database=tmp and udf=demo as one of their policy values.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
> agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
> security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
>
> Diff: https://reviews.apache.org/r/56640/diff/
>
>
> Testing
> -------
>
> Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166043
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 51)
<https://reviews.apache.org/r/56640/#comment237946>
Consider renaming: "getServiceDefForNonrecursivePathResourceMatchers" ==> "cloneServiceDefForResourceFiltering"
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java (line 370)
<https://reviews.apache.org/r/56640/#comment237947>
Qhy should serviceDef be given as argument to this method? Shouldn't the serviceDef be set for the resource-matcher via init(). How is the serviceDef in this method parameter different from the one given in init()?
It will help to add examples/usecases for match being attempted here. Especially details of exits from the 'for' loop at line #370.. value of { matchType, ret } at each iteration and at exit.
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 2151)
<https://reviews.apache.org/r/56640/#comment237952>
Consider abstracting the details of updating resourceName in a method like: updateResourceForFilter(filterResources, serviceDef)
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 2161)
<https://reviews.apache.org/r/56640/#comment237948>
Instead of hardcoding resource names ("path" and "queue") and delimiters (".", "/"), consider reading these from serviceDef.resources.
- Madhan Neethiraj
On Feb. 16, 2017, 10:45 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2017, 10:45 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
>
> A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
>
> The values supported for "resourceMatchScope" option are:
>
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
>
> If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
>
> An example curl command is as follows:
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
>
> This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
>
> Similarly, a command
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
>
> will return only policies which have both database=tmp and udf=demo as one of their policy values.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
> agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
> security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
>
> Diff: https://reviews.apache.org/r/56640/diff/
>
>
> Testing
> -------
>
> Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
> On Feb. 22, 2017, 2:16 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java, line 115
> > <https://reviews.apache.org/r/56640/diff/4/?file=1641490#file1641490line115>
> >
> > "ret" is not populated for non-path resources. intentional?
Yes. Caller needs to know if any changes are to be done to filterResources.
> On Feb. 22, 2017, 2:16 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java, line 386
> > <https://reviews.apache.org/r/56640/diff/4/?file=1641491#file1641491line386>
> >
> > "matchType != MatchType.NONE" ==> "isMatch(scope, matchType)"
> > i.e. break from policyResource iteration only on finding the expected match. Line #398 may not be needed after this update.
> >
> >
> > It might be cleaner to replace lines #384 & #386 with a call to isMatch(accessResource, scope, evalContext)
This is not required.
- Abhay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166287
-----------------------------------------------------------
On Feb. 21, 2017, 10:11 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2017, 10:11 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
>
> A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
>
> The values supported for "resourceMatchScope" option are:
>
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
>
> If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
>
> An example curl command is as follows:
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
>
> This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
>
> Similarly, a command
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
>
> will return only policies which have both database=tmp and udf=demo as one of their policy values.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
> agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
> security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
>
> Diff: https://reviews.apache.org/r/56640/diff/
>
>
> Testing
> -------
>
> Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166287
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 62)
<https://reviews.apache.org/r/56640/#comment238209>
if "matcherClassName" can be null, consider rewriting the condition to:
RangerPathResourceMatcher.class.getName().equals(matcherClassName)
Same for line #94 as well.
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 66)
<https://reviews.apache.org/r/56640/#comment238210>
"wildcard" ==> RangerAbstractResourceMatcher.OPTION_WILD_CARD
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 84)
<https://reviews.apache.org/r/56640/#comment238218>
getFilterResourcesForPolicyFiltering() adds "/*" to path resources - if they don't already end with these (assuming / is the delimiter). This would work only for ANCESTOR matches. Consider renaming this methods appropriately.
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 90)
<https://reviews.apache.org/r/56640/#comment238212>
Consider moving line #90 inside "if" block at #94.
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 98)
<https://reviews.apache.org/r/56640/#comment238215>
"pathSeparatorChar" ==> RangerPathResourceMatcher.OPTION_PATH_SEPARATOR
(need to change the visibility of this constant to public)
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 100)
<https://reviews.apache.org/r/56640/#comment238216>
"org.apache.hadoop.fs.Path.SEPARATOR_CHAR" ==> RangerPathResourceMatcher.DEFAULT_PATH_SEPARATOR_CHAR
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java (line 115)
<https://reviews.apache.org/r/56640/#comment238217>
"ret" is not populated for non-path resources. intentional?
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java (line 381)
<https://reviews.apache.org/r/56640/#comment238222>
setValue(name, null) <== should not be needed - looking at the implementation in RangerAccessResourceImpl.setValue()
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java (line 386)
<https://reviews.apache.org/r/56640/#comment238232>
"matchType != MatchType.NONE" ==> "isMatch(scope, matchType)"
i.e. break from policyResource iteration only on finding the expected match. Line #398 may not be needed after this update.
It might be cleaner to replace lines #384 & #386 with a call to isMatch(accessResource, scope, evalContext)
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 2202)
<https://reviews.apache.org/r/56640/#comment238235>
line #2202 can be:
for (RangerPolicy policy : policies) {
...
}
workList does not seen necessary.
- Madhan Neethiraj
On Feb. 21, 2017, 10:11 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2017, 10:11 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
>
> A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
>
> The values supported for "resourceMatchScope" option are:
>
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
>
> If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
>
> An example curl command is as follows:
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
>
> This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
>
> Similarly, a command
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
>
> will return only policies which have both database=tmp and udf=demo as one of their policy values.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
> agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
> security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
>
> Diff: https://reviews.apache.org/r/56640/diff/
>
>
> Testing
> -------
>
> Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
(Updated Feb. 23, 2017, 9:56 p.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Addressed review comments.
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java 300c1f8
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java PRE-CREATION
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json PRE-CREATION
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166573
-----------------------------------------------------------
Fix it, then Ship it!
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 2138)
<https://reviews.apache.org/r/56640/#comment238554>
I think it will be helpful to document list of valid values (self/ancestor/self_or_ancestor) in SearchFilter.RESOURCE_MATCHER_SCOPE defintion.
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 2200)
<https://reviews.apache.org/r/56640/#comment238555>
foreach (matcher)
foreach(policy)
... // this could endup adding the same policy multiple times, if it matches more than one matcher.
Consider rewritting this as:
foreach (policy)
foreach(matcher)
...
- Madhan Neethiraj
On Feb. 22, 2017, 11:37 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:37 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
>
> A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
>
> The values supported for "resourceMatchScope" option are:
>
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
>
> If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
>
> An example curl command is as follows:
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
>
> This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
>
> Similarly, a command
>
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
>
> will return only policies which have both database=tmp and udf=demo as one of their policy values.
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
> agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java 300c1f8
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
> agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
> agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java PRE-CREATION
> agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json PRE-CREATION
> agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
> security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
>
> Diff: https://reviews.apache.org/r/56640/diff/
>
>
> Testing
> -------
>
> Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
(Updated Feb. 22, 2017, 11:37 p.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Addressed review comments.
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java 300c1f8
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java PRE-CREATION
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json PRE-CREATION
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
(Updated Feb. 21, 2017, 10:11 p.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Addressed review comments
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
(Updated Feb. 16, 2017, 10:45 p.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Added special handling for service types HDFS and YARN(the service-types which use RangerPathResourceMatcher) for simulating self/ancestor scope filtering
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni
Re: Review Request 56640: Support for using resource-matcher for
filtering
policies within a service if service-resource is provided in the filter
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
-----------------------------------------------------------
(Updated Feb. 16, 2017, 12:16 a.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Reworked algorithm for matching policy against a RangerPolicyResourceMatcher
Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383
Repository: ranger
Description
-------
Ranger admin's REST API support retrieving and filtering policies for resource specified in the provided filter. Currently, a simple string-match and wildcard-match is used to filter policies. It is desirable to provide an option to use, for filtering purpose, the same resource-matching algorithm that is used by the policy engine to search policies that need to be evaluated for access determination in the component.
A new option ("resourceMatchScope") will be supported for filtering policies in a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for filtering policies.
The values supported for "resourceMatchScope" option are:
"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
If resourceMatchScope is specified, but its value is not one of "self", "ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".
An example curl command is as follows:
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0&resource:path=/demo&resourceMatchScope=self_or_ancestor'
This will return all access policies for cl1_hadoop service which match path '/demo' or any path that starts with '/demo/'
Similarly, a command
curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0&resource:udf=demo&resource:database=tmp&resourceMatchScope=self
will return only policies which have both database=tmp and udf=demo as one of their policy values.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java 3cdf40b
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java fa2b940
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java 8a784b4
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java 36a9a27
agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 8f6426c
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 15f205a
security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 4fb52a4
Diff: https://reviews.apache.org/r/56640/diff/
Testing
-------
Tested with local VM with and without specifying "resourceMatchScope" option in the filter-spec.
Thanks,
Abhay Kulkarni