You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Gregory Chanan <gc...@cloudera.com> on 2013/10/11 03:13:20 UTC
Review Request 14586: SENTRY-27 Refactor to be able to support different
provider backends (e.g. db vs file)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14586/
-----------------------------------------------------------
Review request for sentry and Brock Noland.
Bugs: SENTRY-27
https://issues.apache.org/jira/browse/SENTRY-27
Repository: sentry
Description
-------
This patch breaks the ties between AuthorizationProviders, PolicyEngines, and ProviderBackends. For example, prior to this patch, specifying HadoopGroupResourceAuthorizationProvider ties you to use the SimpleDBPolicyEngine and the file-based policy parser. This makes it annoying to write new Provider Backends (e.g. db vs file) and service policies (e.g. solr vs hive), because you have to duplicate a bunch of code -- e.g. write a HadoopGroupResourceAuthorizationProvider for each combination.
Highlights:
H1) Creates a provider-common module that has classes that are provider related, but backend agnostic. These are:
RoleValidator.java
PermissionFactory.java
ProviderBackend.java
Roles.java
PolicyEngine.java
GroupMappingService.java
MockGroupMappingServiceProvider.java
H2) Creates a ProviderBackend interface of which the old SimplePolicyParser (now SimpleFileProviderBackend) is the only implementation.
H3) Simplifies the SimpleFileProviderBackend by removing the AtomicReference multi-threaded support that wasn't used.
Processing/parsing also no longer occurs in the constructor: this is to break the dependency that the ProviderBackend had on the PolicyEngines, since the PolicyEngine had to supply the roleValidators at construction time.
H4) The AuthorizationProvider constructors are now of the form: (String resource, PolicyEngine policy), rather than (String resource, String serverName). So they are now policy-engine agnostic (doesn't depend on db-policy based serverName).
You'll notice that the HadoopGroupResourceAuthorizationProvider has a resource param even though it's not used; this is to have the same constructor params as the LocalGroupResourceAuthorizationProvider, since the HiveAuthzBinding assumes these have the same constructor.
H5) The SimpleDBPolicyEngine constructor is now of the form: (String serverName, ProviderBackend providerBackend) rather than (String resourcePath, String serverName). So it is now backend agnostic (doesn't depend on the file-based resourcePath).
H6) Moved the HadoopGroupResourceAuthorizationProvider and LocalGroupResourceAuthorizationProvider from sentry-provider-policy-db to sentry-provider-file because they are no longer tied to the db-policy. In the future, they could go to sentry-provider-common but for now the ResourceAuthorizationProvider base class uses PolicyFile-specific KV_JOINERS and such.
H7) Modified the HiveAuthzBinding to use the new ProviderBackend/AuthorizationProvider/SimpleDBPolicyEngine model. The defaults are defined such that existing definitions of only the AuthorizationProvider maintains the existing behavior: it will automatically use the SimpleFileProviderBackend and SimpleDBPolicyEngine. I added a test for this as well.
================================================
Notes:
N1) Sorry again that file renames show as delete and new file. I tried "git diff -M" but review board won't accept it.
N2) After this patch, everything in sentry-provider-policy-db is really backend-agnostic, so can be moved to say, sentry-policy-db. Let's do that in a later patch, though.
Diffs
-----
pom.xml 9b73a21
sentry-binding/sentry-binding-hive/pom.xml 16b12dd
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 36d2fd1
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java ab1b4d3
sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/MockUserToGroupMapping.java 881712b
sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestHiveAuthzBindings.java 506abf8
sentry-dist/pom.xml b398ce0
sentry-dist/src/main/assembly/src.xml 60ec346
sentry-provider/pom.xml 2667312
sentry-provider/sentry-provider-common/pom.xml PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/GroupMappingService.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PermissionFactory.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PolicyEngine.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/RoleValidator.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/Roles.java PRE-CREATION
sentry-provider/sentry-provider-common/src/test/java/org/apache/sentry/provider/common/MockGroupMappingServiceProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/pom.xml a3d010a
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/GroupMappingService.java 6af2edf
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupMappingService.java 1bbb125
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupResourceAuthorizationProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupMappingService.java fa3e804
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupResourceAuthorizationProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PermissionFactory.java 44624e7
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyEngine.java ecf954e
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java dd63b85
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/RoleValidator.java 323a18a
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/Roles.java 2c47232
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimplePolicyParser.java 4bc692a
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/AbstractTestSimplePolicyEngine.java aa28bb2
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/MockGroupMappingServiceProvider.java 74956fc
sentry-provider/sentry-provider-policy-db/pom.xml 412c13e
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/AbstractDBRoleValidator.java 19769eb
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/DBWildcardPermission.java b49dd6a
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/HadoopGroupResourceAuthorizationProvider.java 328e990
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/LocalGroupResourceAuthorizationProvider.java 6c43f3c
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/SimpleDBPolicyEngine.java b428e46
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/AbstractTestSimplePolicyEngine.java PRE-CREATION
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/DBPolicyFileBackend.java PRE-CREATION
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestPolicyParsingNegative.java e17eab0
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderGeneralCases.java be36fe8
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderSpecialCases.java 508a566
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineDFS.java 52860b6
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineLocalFS.java 51119d2
sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/AbstractSearchRoleValidator.java b0134bd
sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/SearchWildcardPermission.java 427ed2b
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java e452f71
Diff: https://reviews.apache.org/r/14586/diff/
Testing
-------
Ran the entire test suite.
Thanks,
Gregory Chanan
Re: Review Request 14586: SENTRY-27 Refactor to be able to support different
provider backends (e.g. db vs file)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14586/
-----------------------------------------------------------
(Updated Oct. 14, 2013, 9:03 p.m.)
Review request for sentry and Brock Noland.
Changes
-------
Update for Brock's comments.
Bugs: SENTRY-27
https://issues.apache.org/jira/browse/SENTRY-27
Repository: sentry
Description
-------
This patch breaks the ties between AuthorizationProviders, PolicyEngines, and ProviderBackends. For example, prior to this patch, specifying HadoopGroupResourceAuthorizationProvider ties you to use the SimpleDBPolicyEngine and the file-based policy parser. This makes it annoying to write new Provider Backends (e.g. db vs file) and service policies (e.g. solr vs hive), because you have to duplicate a bunch of code -- e.g. write a HadoopGroupResourceAuthorizationProvider for each combination.
Highlights:
H1) Creates a provider-common module that has classes that are provider related, but backend agnostic. These are:
RoleValidator.java
PermissionFactory.java
ProviderBackend.java
Roles.java
PolicyEngine.java
GroupMappingService.java
MockGroupMappingServiceProvider.java
H2) Creates a ProviderBackend interface of which the old SimplePolicyParser (now SimpleFileProviderBackend) is the only implementation.
H3) Simplifies the SimpleFileProviderBackend by removing the AtomicReference multi-threaded support that wasn't used.
Processing/parsing also no longer occurs in the constructor: this is to break the dependency that the ProviderBackend had on the PolicyEngines, since the PolicyEngine had to supply the roleValidators at construction time.
H4) The AuthorizationProvider constructors are now of the form: (String resource, PolicyEngine policy), rather than (String resource, String serverName). So they are now policy-engine agnostic (doesn't depend on db-policy based serverName).
You'll notice that the HadoopGroupResourceAuthorizationProvider has a resource param even though it's not used; this is to have the same constructor params as the LocalGroupResourceAuthorizationProvider, since the HiveAuthzBinding assumes these have the same constructor.
H5) The SimpleDBPolicyEngine constructor is now of the form: (String serverName, ProviderBackend providerBackend) rather than (String resourcePath, String serverName). So it is now backend agnostic (doesn't depend on the file-based resourcePath).
H6) Moved the HadoopGroupResourceAuthorizationProvider and LocalGroupResourceAuthorizationProvider from sentry-provider-policy-db to sentry-provider-file because they are no longer tied to the db-policy. In the future, they could go to sentry-provider-common but for now the ResourceAuthorizationProvider base class uses PolicyFile-specific KV_JOINERS and such.
H7) Modified the HiveAuthzBinding to use the new ProviderBackend/AuthorizationProvider/SimpleDBPolicyEngine model. The defaults are defined such that existing definitions of only the AuthorizationProvider maintains the existing behavior: it will automatically use the SimpleFileProviderBackend and SimpleDBPolicyEngine. I added a test for this as well.
================================================
Notes:
N1) Sorry again that file renames show as delete and new file. I tried "git diff -M" but review board won't accept it.
N2) After this patch, everything in sentry-provider-policy-db is really backend-agnostic, so can be moved to say, sentry-policy-db. Let's do that in a later patch, though.
Diffs (updated)
-----
pom.xml 9b73a21
sentry-binding/sentry-binding-hive/pom.xml 16b12dd
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 36d2fd1
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java ab1b4d3
sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/MockUserToGroupMapping.java 881712b
sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestHiveAuthzBindings.java 506abf8
sentry-dist/pom.xml b398ce0
sentry-dist/src/main/assembly/src.xml 60ec346
sentry-provider/pom.xml 2667312
sentry-provider/sentry-provider-common/pom.xml PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/GroupMappingService.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PermissionFactory.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PolicyEngine.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/RoleValidator.java PRE-CREATION
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/Roles.java PRE-CREATION
sentry-provider/sentry-provider-common/src/test/java/org/apache/sentry/provider/common/MockGroupMappingServiceProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/pom.xml a3d010a
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/GroupMappingService.java 6af2edf
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupMappingService.java 1bbb125
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupResourceAuthorizationProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupMappingService.java fa3e804
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupResourceAuthorizationProvider.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PermissionFactory.java 44624e7
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyEngine.java ecf954e
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java dd63b85
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/RoleValidator.java 323a18a
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/Roles.java 2c47232
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java PRE-CREATION
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimplePolicyParser.java 4bc692a
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/AbstractTestSimplePolicyEngine.java aa28bb2
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/MockGroupMappingServiceProvider.java 74956fc
sentry-provider/sentry-provider-policy-db/pom.xml 412c13e
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/AbstractDBRoleValidator.java 19769eb
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/DBWildcardPermission.java b49dd6a
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/HadoopGroupResourceAuthorizationProvider.java 328e990
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/LocalGroupResourceAuthorizationProvider.java 6c43f3c
sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/SimpleDBPolicyEngine.java b428e46
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/AbstractTestSimplePolicyEngine.java PRE-CREATION
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/DBPolicyFileBackend.java PRE-CREATION
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestPolicyParsingNegative.java e17eab0
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderGeneralCases.java be36fe8
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderSpecialCases.java 508a566
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineDFS.java 52860b6
sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineLocalFS.java 51119d2
sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/AbstractSearchRoleValidator.java b0134bd
sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/SearchWildcardPermission.java 427ed2b
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java e452f71
Diff: https://reviews.apache.org/r/14586/diff/
Testing
-------
Ran the entire test suite.
Thanks,
Gregory Chanan
Re: Review Request 14586: SENTRY-27 Refactor to be able to support different
provider backends (e.g. db vs file)
Posted by Gregory Chanan <gc...@cloudera.com>.
> On Oct. 14, 2013, 8:27 p.m., Brock Noland wrote:
> > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java, line 159
> > <https://reviews.apache.org/r/14586/diff/2/?file=363644#file363644line159>
> >
> > We should really have a parameters holder for this since resource is not always used but that's not your problem.
Agreed, I filed SENTRY-44 for this.
- Gregory
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14586/#review26980
-----------------------------------------------------------
On Oct. 14, 2013, 9:03 p.m., Gregory Chanan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14586/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2013, 9:03 p.m.)
>
>
> Review request for sentry and Brock Noland.
>
>
> Bugs: SENTRY-27
> https://issues.apache.org/jira/browse/SENTRY-27
>
>
> Repository: sentry
>
>
> Description
> -------
>
> This patch breaks the ties between AuthorizationProviders, PolicyEngines, and ProviderBackends. For example, prior to this patch, specifying HadoopGroupResourceAuthorizationProvider ties you to use the SimpleDBPolicyEngine and the file-based policy parser. This makes it annoying to write new Provider Backends (e.g. db vs file) and service policies (e.g. solr vs hive), because you have to duplicate a bunch of code -- e.g. write a HadoopGroupResourceAuthorizationProvider for each combination.
>
> Highlights:
> H1) Creates a provider-common module that has classes that are provider related, but backend agnostic. These are:
> RoleValidator.java
> PermissionFactory.java
> ProviderBackend.java
> Roles.java
> PolicyEngine.java
> GroupMappingService.java
> MockGroupMappingServiceProvider.java
>
> H2) Creates a ProviderBackend interface of which the old SimplePolicyParser (now SimpleFileProviderBackend) is the only implementation.
>
> H3) Simplifies the SimpleFileProviderBackend by removing the AtomicReference multi-threaded support that wasn't used.
> Processing/parsing also no longer occurs in the constructor: this is to break the dependency that the ProviderBackend had on the PolicyEngines, since the PolicyEngine had to supply the roleValidators at construction time.
>
> H4) The AuthorizationProvider constructors are now of the form: (String resource, PolicyEngine policy), rather than (String resource, String serverName). So they are now policy-engine agnostic (doesn't depend on db-policy based serverName).
> You'll notice that the HadoopGroupResourceAuthorizationProvider has a resource param even though it's not used; this is to have the same constructor params as the LocalGroupResourceAuthorizationProvider, since the HiveAuthzBinding assumes these have the same constructor.
>
> H5) The SimpleDBPolicyEngine constructor is now of the form: (String serverName, ProviderBackend providerBackend) rather than (String resourcePath, String serverName). So it is now backend agnostic (doesn't depend on the file-based resourcePath).
>
> H6) Moved the HadoopGroupResourceAuthorizationProvider and LocalGroupResourceAuthorizationProvider from sentry-provider-policy-db to sentry-provider-file because they are no longer tied to the db-policy. In the future, they could go to sentry-provider-common but for now the ResourceAuthorizationProvider base class uses PolicyFile-specific KV_JOINERS and such.
>
> H7) Modified the HiveAuthzBinding to use the new ProviderBackend/AuthorizationProvider/SimpleDBPolicyEngine model. The defaults are defined such that existing definitions of only the AuthorizationProvider maintains the existing behavior: it will automatically use the SimpleFileProviderBackend and SimpleDBPolicyEngine. I added a test for this as well.
>
>
> ================================================
> Notes:
> N1) Sorry again that file renames show as delete and new file. I tried "git diff -M" but review board won't accept it.
> N2) After this patch, everything in sentry-provider-policy-db is really backend-agnostic, so can be moved to say, sentry-policy-db. Let's do that in a later patch, though.
>
>
> Diffs
> -----
>
> pom.xml 9b73a21
> sentry-binding/sentry-binding-hive/pom.xml 16b12dd
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 36d2fd1
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java ab1b4d3
> sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/MockUserToGroupMapping.java 881712b
> sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestHiveAuthzBindings.java 506abf8
> sentry-dist/pom.xml b398ce0
> sentry-dist/src/main/assembly/src.xml 60ec346
> sentry-provider/pom.xml 2667312
> sentry-provider/sentry-provider-common/pom.xml PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/GroupMappingService.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PermissionFactory.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PolicyEngine.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/RoleValidator.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/Roles.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/test/java/org/apache/sentry/provider/common/MockGroupMappingServiceProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/pom.xml a3d010a
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/GroupMappingService.java 6af2edf
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupMappingService.java 1bbb125
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupResourceAuthorizationProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupMappingService.java fa3e804
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupResourceAuthorizationProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PermissionFactory.java 44624e7
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyEngine.java ecf954e
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java dd63b85
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/RoleValidator.java 323a18a
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/Roles.java 2c47232
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimplePolicyParser.java 4bc692a
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/AbstractTestSimplePolicyEngine.java aa28bb2
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/MockGroupMappingServiceProvider.java 74956fc
> sentry-provider/sentry-provider-policy-db/pom.xml 412c13e
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/AbstractDBRoleValidator.java 19769eb
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/DBWildcardPermission.java b49dd6a
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/HadoopGroupResourceAuthorizationProvider.java 328e990
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/LocalGroupResourceAuthorizationProvider.java 6c43f3c
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/SimpleDBPolicyEngine.java b428e46
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/AbstractTestSimplePolicyEngine.java PRE-CREATION
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/DBPolicyFileBackend.java PRE-CREATION
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestPolicyParsingNegative.java e17eab0
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderGeneralCases.java be36fe8
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderSpecialCases.java 508a566
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineDFS.java 52860b6
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineLocalFS.java 51119d2
> sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/AbstractSearchRoleValidator.java b0134bd
> sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/SearchWildcardPermission.java 427ed2b
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java e452f71
>
> Diff: https://reviews.apache.org/r/14586/diff/
>
>
> Testing
> -------
>
> Ran the entire test suite.
>
>
> Thanks,
>
> Gregory Chanan
>
>
Re: Review Request 14586: SENTRY-27 Refactor to be able to support different
provider backends (e.g. db vs file)
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14586/#review26980
-----------------------------------------------------------
Ship it!
One minor issue. Can you fix and then upload the new patch to the JIRA?
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java
<https://reviews.apache.org/r/14586/#comment52544>
We should really have a parameters holder for this since resource is not always used but that's not your problem.
sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestHiveAuthzBindings.java
<https://reviews.apache.org/r/14586/#comment52545>
We should be catching a specific exception here. I.e a NPE or something similar shouldn't allow this to pass.
- Brock Noland
On Oct. 11, 2013, 1:13 a.m., Gregory Chanan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14586/
> -----------------------------------------------------------
>
> (Updated Oct. 11, 2013, 1:13 a.m.)
>
>
> Review request for sentry and Brock Noland.
>
>
> Bugs: SENTRY-27
> https://issues.apache.org/jira/browse/SENTRY-27
>
>
> Repository: sentry
>
>
> Description
> -------
>
> This patch breaks the ties between AuthorizationProviders, PolicyEngines, and ProviderBackends. For example, prior to this patch, specifying HadoopGroupResourceAuthorizationProvider ties you to use the SimpleDBPolicyEngine and the file-based policy parser. This makes it annoying to write new Provider Backends (e.g. db vs file) and service policies (e.g. solr vs hive), because you have to duplicate a bunch of code -- e.g. write a HadoopGroupResourceAuthorizationProvider for each combination.
>
> Highlights:
> H1) Creates a provider-common module that has classes that are provider related, but backend agnostic. These are:
> RoleValidator.java
> PermissionFactory.java
> ProviderBackend.java
> Roles.java
> PolicyEngine.java
> GroupMappingService.java
> MockGroupMappingServiceProvider.java
>
> H2) Creates a ProviderBackend interface of which the old SimplePolicyParser (now SimpleFileProviderBackend) is the only implementation.
>
> H3) Simplifies the SimpleFileProviderBackend by removing the AtomicReference multi-threaded support that wasn't used.
> Processing/parsing also no longer occurs in the constructor: this is to break the dependency that the ProviderBackend had on the PolicyEngines, since the PolicyEngine had to supply the roleValidators at construction time.
>
> H4) The AuthorizationProvider constructors are now of the form: (String resource, PolicyEngine policy), rather than (String resource, String serverName). So they are now policy-engine agnostic (doesn't depend on db-policy based serverName).
> You'll notice that the HadoopGroupResourceAuthorizationProvider has a resource param even though it's not used; this is to have the same constructor params as the LocalGroupResourceAuthorizationProvider, since the HiveAuthzBinding assumes these have the same constructor.
>
> H5) The SimpleDBPolicyEngine constructor is now of the form: (String serverName, ProviderBackend providerBackend) rather than (String resourcePath, String serverName). So it is now backend agnostic (doesn't depend on the file-based resourcePath).
>
> H6) Moved the HadoopGroupResourceAuthorizationProvider and LocalGroupResourceAuthorizationProvider from sentry-provider-policy-db to sentry-provider-file because they are no longer tied to the db-policy. In the future, they could go to sentry-provider-common but for now the ResourceAuthorizationProvider base class uses PolicyFile-specific KV_JOINERS and such.
>
> H7) Modified the HiveAuthzBinding to use the new ProviderBackend/AuthorizationProvider/SimpleDBPolicyEngine model. The defaults are defined such that existing definitions of only the AuthorizationProvider maintains the existing behavior: it will automatically use the SimpleFileProviderBackend and SimpleDBPolicyEngine. I added a test for this as well.
>
>
> ================================================
> Notes:
> N1) Sorry again that file renames show as delete and new file. I tried "git diff -M" but review board won't accept it.
> N2) After this patch, everything in sentry-provider-policy-db is really backend-agnostic, so can be moved to say, sentry-policy-db. Let's do that in a later patch, though.
>
>
> Diffs
> -----
>
> pom.xml 9b73a21
> sentry-binding/sentry-binding-hive/pom.xml 16b12dd
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 36d2fd1
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java ab1b4d3
> sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/MockUserToGroupMapping.java 881712b
> sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestHiveAuthzBindings.java 506abf8
> sentry-dist/pom.xml b398ce0
> sentry-dist/src/main/assembly/src.xml 60ec346
> sentry-provider/pom.xml 2667312
> sentry-provider/sentry-provider-common/pom.xml PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/GroupMappingService.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PermissionFactory.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/PolicyEngine.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/RoleValidator.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/Roles.java PRE-CREATION
> sentry-provider/sentry-provider-common/src/test/java/org/apache/sentry/provider/common/MockGroupMappingServiceProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/pom.xml a3d010a
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/GroupMappingService.java 6af2edf
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupMappingService.java 1bbb125
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/HadoopGroupResourceAuthorizationProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupMappingService.java fa3e804
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/LocalGroupResourceAuthorizationProvider.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PermissionFactory.java 44624e7
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyEngine.java ecf954e
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java dd63b85
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/RoleValidator.java 323a18a
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/Roles.java 2c47232
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java PRE-CREATION
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimplePolicyParser.java 4bc692a
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/AbstractTestSimplePolicyEngine.java aa28bb2
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/MockGroupMappingServiceProvider.java 74956fc
> sentry-provider/sentry-provider-policy-db/pom.xml 412c13e
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/AbstractDBRoleValidator.java 19769eb
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/DBWildcardPermission.java b49dd6a
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/HadoopGroupResourceAuthorizationProvider.java 328e990
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/LocalGroupResourceAuthorizationProvider.java 6c43f3c
> sentry-provider/sentry-provider-policy-db/src/main/java/org/apache/sentry/provider/db/SimpleDBPolicyEngine.java b428e46
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/AbstractTestSimplePolicyEngine.java PRE-CREATION
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/DBPolicyFileBackend.java PRE-CREATION
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestPolicyParsingNegative.java e17eab0
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderGeneralCases.java be36fe8
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestResourceAuthorizationProviderSpecialCases.java 508a566
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineDFS.java 52860b6
> sentry-provider/sentry-provider-policy-db/src/test/java/org/apache/sentry/provider/db/TestSimpleDBPolicyEngineLocalFS.java 51119d2
> sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/AbstractSearchRoleValidator.java b0134bd
> sentry-provider/sentry-provider-policy-search/src/main/java/org/apache/sentry/provider/search/SearchWildcardPermission.java 427ed2b
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java e452f71
>
> Diff: https://reviews.apache.org/r/14586/diff/
>
>
> Testing
> -------
>
> Ran the entire test suite.
>
>
> Thanks,
>
> Gregory Chanan
>
>