You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Prasad Mujumdar <pr...@cloudera.com> on 2014/02/21 23:43:24 UTC
Review Request 18380: SENTRY-3: Create a diagnostics tool for configuration
validation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18380/
-----------------------------------------------------------
Review request for sentry, Brock Noland, Shreepadma Venugopalan, and Sravya Tirukkovalur.
Bugs: SENTRY-3
https://issues.apache.org/jira/browse/SENTRY-3
Repository: sentry
Description
-------
Tool for offline diagnostics for Hive:
- Validate the sentry policy file
The policy file syntax/semantic errors and warnings are collected and added to the exception. The tool extracts this information and print it for the end user. The sentry configuration and policy file locations are read from the hive configuration, and can be overridden using command line arguments. This is useful for testing correction/changes without having to modify the existing setup.
- List permissions for a given user (reverse permission look up)
The authorization provider and Policy engine interfaces are extended to support reverse lookup. The tool can be used to list out permissions for a given user which is useful for analyzing errors or testing.
- Mock compilation of the given query and retrieve the missing privileges
This is supported in online as well as offline mode. The offline mode invokes embedded hive driver to compile a query and shows the missing privilege if the authorization fails. This doesn't require connection to hiveserver, however does require the metadata. Hence it's more useful for dev/testing env.
The online mode submits a query to HiveServer2. The authorization errors are saved in a config property by hive bindings, and retrieved by the tool. It adds a config property which makes sentry to throw a mock error even when the authorization successful. This ensures that the query is never executed. It's a workaround since HiveServer2 doesn't support compile-only RPC request.
Diffs
-----
bin/sentry PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java cac4864
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 45d5d3b
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryConfigTool.java PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java b7d79d6
sentry-core/sentry-core-common/pom.xml a14f129
sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/SentryConfigurationException.java PRE-CREATION
sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 693de1b
sentry-policy/sentry-policy-db/src/main/java/org/apache/sentry/policy/db/SimpleDBPolicyEngine.java 1d72f87
sentry-policy/sentry-policy-search/src/main/java/org/apache/sentry/policy/search/SimpleSearchPolicyEngine.java 21711ef
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/AuthorizationProvider.java 1244755
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java f48eafe
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java 415a509
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java 205d012
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java f432915
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/TestGetGroupMapping.java a4d4bb3
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/Context.java 66cd2d1
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestConfigTool.java PRE-CREATION
Diff: https://reviews.apache.org/r/18380/diff/
Testing
-------
Added a unit test to cover the validation cases. Manually tested similar options using the tool script.
Thanks,
Prasad Mujumdar
Re: Review Request 18380: SENTRY-3: Create a diagnostics tool for
configuration validation
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18380/#review35214
-----------------------------------------------------------
bin/sentry
<https://reviews.apache.org/r/18380/#comment65668>
sure. will make that change.
- Prasad Mujumdar
On Feb. 21, 2014, 10:43 p.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18380/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2014, 10:43 p.m.)
>
>
> Review request for sentry, Brock Noland, Shreepadma Venugopalan, and Sravya Tirukkovalur.
>
>
> Bugs: SENTRY-3
> https://issues.apache.org/jira/browse/SENTRY-3
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tool for offline diagnostics for Hive:
> - Validate the sentry policy file
> The policy file syntax/semantic errors and warnings are collected and added to the exception. The tool extracts this information and print it for the end user. The sentry configuration and policy file locations are read from the hive configuration, and can be overridden using command line arguments. This is useful for testing correction/changes without having to modify the existing setup.
>
> - List permissions for a given user (reverse permission look up)
> The authorization provider and Policy engine interfaces are extended to support reverse lookup. The tool can be used to list out permissions for a given user which is useful for analyzing errors or testing.
> - Mock compilation of the given query and retrieve the missing privileges
> This is supported in online as well as offline mode. The offline mode invokes embedded hive driver to compile a query and shows the missing privilege if the authorization fails. This doesn't require connection to hiveserver, however does require the metadata. Hence it's more useful for dev/testing env.
> The online mode submits a query to HiveServer2. The authorization errors are saved in a config property by hive bindings, and retrieved by the tool. It adds a config property which makes sentry to throw a mock error even when the authorization successful. This ensures that the query is never executed. It's a workaround since HiveServer2 doesn't support compile-only RPC request.
>
>
> Diffs
> -----
>
> bin/sentry PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java cac4864
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 45d5d3b
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryConfigTool.java PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java b7d79d6
> sentry-core/sentry-core-common/pom.xml a14f129
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/SentryConfigurationException.java PRE-CREATION
> sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 693de1b
> sentry-policy/sentry-policy-db/src/main/java/org/apache/sentry/policy/db/SimpleDBPolicyEngine.java 1d72f87
> sentry-policy/sentry-policy-search/src/main/java/org/apache/sentry/policy/search/SimpleSearchPolicyEngine.java 21711ef
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/AuthorizationProvider.java 1244755
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java f48eafe
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java 415a509
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java 205d012
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java f432915
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/TestGetGroupMapping.java a4d4bb3
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/Context.java 66cd2d1
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestConfigTool.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/18380/diff/
>
>
> Testing
> -------
>
> Added a unit test to cover the validation cases. Manually tested similar options using the tool script.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 18380: SENTRY-3: Create a diagnostics tool for
configuration validation
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18380/#review35209
-----------------------------------------------------------
LGTM! The only big item I think we should address is the --sentry-config-tool option as describe below. Other than that I am OK with this.
bin/sentry
<https://reviews.apache.org/r/18380/#comment65661>
We'll want to merge this with the other sentry tool for SPS. Thus can we change
--sentry-config-tool
to
--command sentry-config-tool
or
--command config-tool
That way when we merge the two tools we won't have conflicting options such as --sentry-config-tool and --sentry-service we can just have --command [config-tool, service].
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
<https://reviews.apache.org/r/18380/#comment65662>
"Mock "
to
" Mock"
?
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java
<https://reviews.apache.org/r/18380/#comment65663>
remove TODO
- Brock Noland
On Feb. 21, 2014, 10:43 p.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18380/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2014, 10:43 p.m.)
>
>
> Review request for sentry, Brock Noland, Shreepadma Venugopalan, and Sravya Tirukkovalur.
>
>
> Bugs: SENTRY-3
> https://issues.apache.org/jira/browse/SENTRY-3
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tool for offline diagnostics for Hive:
> - Validate the sentry policy file
> The policy file syntax/semantic errors and warnings are collected and added to the exception. The tool extracts this information and print it for the end user. The sentry configuration and policy file locations are read from the hive configuration, and can be overridden using command line arguments. This is useful for testing correction/changes without having to modify the existing setup.
>
> - List permissions for a given user (reverse permission look up)
> The authorization provider and Policy engine interfaces are extended to support reverse lookup. The tool can be used to list out permissions for a given user which is useful for analyzing errors or testing.
> - Mock compilation of the given query and retrieve the missing privileges
> This is supported in online as well as offline mode. The offline mode invokes embedded hive driver to compile a query and shows the missing privilege if the authorization fails. This doesn't require connection to hiveserver, however does require the metadata. Hence it's more useful for dev/testing env.
> The online mode submits a query to HiveServer2. The authorization errors are saved in a config property by hive bindings, and retrieved by the tool. It adds a config property which makes sentry to throw a mock error even when the authorization successful. This ensures that the query is never executed. It's a workaround since HiveServer2 doesn't support compile-only RPC request.
>
>
> Diffs
> -----
>
> bin/sentry PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java cac4864
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 45d5d3b
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryConfigTool.java PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java b7d79d6
> sentry-core/sentry-core-common/pom.xml a14f129
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/SentryConfigurationException.java PRE-CREATION
> sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 693de1b
> sentry-policy/sentry-policy-db/src/main/java/org/apache/sentry/policy/db/SimpleDBPolicyEngine.java 1d72f87
> sentry-policy/sentry-policy-search/src/main/java/org/apache/sentry/policy/search/SimpleSearchPolicyEngine.java 21711ef
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/AuthorizationProvider.java 1244755
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java f48eafe
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java 415a509
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java 205d012
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java f432915
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/TestGetGroupMapping.java a4d4bb3
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/Context.java 66cd2d1
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestConfigTool.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/18380/diff/
>
>
> Testing
> -------
>
> Added a unit test to cover the validation cases. Manually tested similar options using the tool script.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 18380: SENTRY-3: Create a diagnostics tool for
configuration validation
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18380/#review35323
-----------------------------------------------------------
Ship it!
Ship It!
- Brock Noland
On Feb. 22, 2014, 12:24 a.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18380/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2014, 12:24 a.m.)
>
>
> Review request for sentry, Brock Noland, Shreepadma Venugopalan, and Sravya Tirukkovalur.
>
>
> Bugs: SENTRY-3
> https://issues.apache.org/jira/browse/SENTRY-3
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tool for offline diagnostics for Hive:
> - Validate the sentry policy file
> The policy file syntax/semantic errors and warnings are collected and added to the exception. The tool extracts this information and print it for the end user. The sentry configuration and policy file locations are read from the hive configuration, and can be overridden using command line arguments. This is useful for testing correction/changes without having to modify the existing setup.
>
> - List permissions for a given user (reverse permission look up)
> The authorization provider and Policy engine interfaces are extended to support reverse lookup. The tool can be used to list out permissions for a given user which is useful for analyzing errors or testing.
> - Mock compilation of the given query and retrieve the missing privileges
> This is supported in online as well as offline mode. The offline mode invokes embedded hive driver to compile a query and shows the missing privilege if the authorization fails. This doesn't require connection to hiveserver, however does require the metadata. Hence it's more useful for dev/testing env.
> The online mode submits a query to HiveServer2. The authorization errors are saved in a config property by hive bindings, and retrieved by the tool. It adds a config property which makes sentry to throw a mock error even when the authorization successful. This ensures that the query is never executed. It's a workaround since HiveServer2 doesn't support compile-only RPC request.
>
>
> Diffs
> -----
>
> bin/config-tool.sh PRE-CREATION
> bin/sentry PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java cac4864
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 45d5d3b
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryConfigTool.java PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java b7d79d6
> sentry-core/sentry-core-common/pom.xml a14f129
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/SentryConfigurationException.java PRE-CREATION
> sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 693de1b
> sentry-policy/sentry-policy-db/src/main/java/org/apache/sentry/policy/db/SimpleDBPolicyEngine.java 1d72f87
> sentry-policy/sentry-policy-search/src/main/java/org/apache/sentry/policy/search/SimpleSearchPolicyEngine.java 21711ef
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/AuthorizationProvider.java 1244755
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java f48eafe
> sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java 415a509
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java 205d012
> sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java f432915
> sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/TestGetGroupMapping.java a4d4bb3
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/Context.java 66cd2d1
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestConfigTool.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/18380/diff/
>
>
> Testing
> -------
>
> Added a unit test to cover the validation cases. Manually tested similar options using the tool script.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 18380: SENTRY-3: Create a diagnostics tool for
configuration validation
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18380/
-----------------------------------------------------------
(Updated Feb. 22, 2014, 12:24 a.m.)
Review request for sentry, Brock Noland, Shreepadma Venugopalan, and Sravya Tirukkovalur.
Changes
-------
Changes per review feedback.
Bugs: SENTRY-3
https://issues.apache.org/jira/browse/SENTRY-3
Repository: sentry
Description
-------
Tool for offline diagnostics for Hive:
- Validate the sentry policy file
The policy file syntax/semantic errors and warnings are collected and added to the exception. The tool extracts this information and print it for the end user. The sentry configuration and policy file locations are read from the hive configuration, and can be overridden using command line arguments. This is useful for testing correction/changes without having to modify the existing setup.
- List permissions for a given user (reverse permission look up)
The authorization provider and Policy engine interfaces are extended to support reverse lookup. The tool can be used to list out permissions for a given user which is useful for analyzing errors or testing.
- Mock compilation of the given query and retrieve the missing privileges
This is supported in online as well as offline mode. The offline mode invokes embedded hive driver to compile a query and shows the missing privilege if the authorization fails. This doesn't require connection to hiveserver, however does require the metadata. Hence it's more useful for dev/testing env.
The online mode submits a query to HiveServer2. The authorization errors are saved in a config property by hive bindings, and retrieved by the tool. It adds a config property which makes sentry to throw a mock error even when the authorization successful. This ensures that the query is never executed. It's a workaround since HiveServer2 doesn't support compile-only RPC request.
Diffs (updated)
-----
bin/config-tool.sh PRE-CREATION
bin/sentry PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java cac4864
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 45d5d3b
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryConfigTool.java PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java b7d79d6
sentry-core/sentry-core-common/pom.xml a14f129
sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/SentryConfigurationException.java PRE-CREATION
sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 693de1b
sentry-policy/sentry-policy-db/src/main/java/org/apache/sentry/policy/db/SimpleDBPolicyEngine.java 1d72f87
sentry-policy/sentry-policy-search/src/main/java/org/apache/sentry/policy/search/SimpleSearchPolicyEngine.java 21711ef
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/AuthorizationProvider.java 1244755
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/NoAuthorizationProvider.java f48eafe
sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java 415a509
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/ResourceAuthorizationProvider.java 205d012
sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java f432915
sentry-provider/sentry-provider-file/src/test/java/org/apache/sentry/provider/file/TestGetGroupMapping.java a4d4bb3
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/Context.java 66cd2d1
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestConfigTool.java PRE-CREATION
Diff: https://reviews.apache.org/r/18380/diff/
Testing
-------
Added a unit test to cover the validation cases. Manually tested similar options using the tool script.
Thanks,
Prasad Mujumdar