You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Phabricator (Updated) (JIRA)" <ji...@apache.org> on 2012/03/15 22:14:40 UTC
[jira] [Updated] (HIVE-2809) StorageHandler authorization providers
[ https://issues.apache.org/jira/browse/HIVE-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2809:
------------------------------
Attachment: HIVE-2809.D1953.5.patch
enis updated the revision "HIVE-2809 [jira] StorageHandler authorization providers".
Reviewers: JIRA
Updated the patch for trunk, and fixed a possible NPE in HdfsAP.
REVISION DETAIL
https://reviews.facebook.net/D1953
AFFECTED FILES
contrib/src/java/org/apache/hadoop/hive/contrib/metastore/hooks/TestURLHook.java
contrib/src/test/queries/clientnegative/url_hook.q
contrib/src/test/results/clientnegative/url_hook.q.out
contrib/src/test/results/clientpositive/fileformat_base64.q.out
contrib/src/test/results/clientpositive/serde_s3.q.out
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseAuthorizationProvider.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/test/results/hbase_binary_storage_queries.q.out
hbase-handler/src/test/results/hbase_queries.q.out
hbase-handler/src/test/results/hbase_stats.q.out
hbase-handler/src/test/results/hbase_stats2.q.out
metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
ql/src/java/org/apache/hadoop/hive/ql/hooks/Entity.java
ql/src/java/org/apache/hadoop/hive/ql/hooks/ReadEntity.java
ql/src/java/org/apache/hadoop/hive/ql/hooks/WriteEntity.java
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/DefaultHiveAuthorizationProvider.java
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/HdfsAuthorizationProvider.java
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/HiveAuthorizationProviderBase.java
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageDelegationAuthorizationProvider.java
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/AuthTestUtils.java
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/TestHdfsAuthorizationProvider.java
ql/src/test/queries/clientnegative/authorization_fail_5.q
ql/src/test/queries/clientnegative/authorization_part.q
ql/src/test/queries/clientpositive/alter_rename_partition_authorization.q
ql/src/test/queries/clientpositive/exim_23_import_part_authsuccess.q
ql/src/test/results/clientnegative/addpart1.q.out
ql/src/test/results/clientnegative/authorization_fail_5.q.out
ql/src/test/results/clientnegative/authorization_part.q.out
ql/src/test/results/clientnegative/database_drop_not_empty.q.out
ql/src/test/results/clientnegative/database_drop_not_empty_restrict.q.out
ql/src/test/results/clientnegative/drop_partition_failure.q.out
ql/src/test/results/clientnegative/exim_01_nonpart_over_loaded.q.out
ql/src/test/results/clientnegative/exim_02_all_part_over_overlap.q.out
ql/src/test/results/clientnegative/exim_03_nonpart_noncompat_colschema.q.out
ql/src/test/results/clientnegative/exim_04_nonpart_noncompat_colnumber.q.out
ql/src/test/results/clientnegative/exim_05_nonpart_noncompat_coltype.q.out
ql/src/test/results/clientnegative/exim_06_nonpart_noncompat_storage.q.out
ql/src/test/results/clientnegative/exim_07_nonpart_noncompat_ifof.q.out
ql/src/test/results/clientnegative/exim_08_nonpart_noncompat_serde.q.out
ql/src/test/results/clientnegative/exim_09_nonpart_noncompat_serdeparam.q.out
ql/src/test/results/clientnegative/exim_10_nonpart_noncompat_bucketing.q.out
ql/src/test/results/clientnegative/exim_11_nonpart_noncompat_sorting.q.out
ql/src/test/results/clientnegative/exim_13_nonnative_import.q.out
ql/src/test/results/clientnegative/exim_14_nonpart_part.q.out
ql/src/test/results/clientnegative/exim_15_part_nonpart.q.out
ql/src/test/results/clientnegative/exim_16_part_noncompat_schema.q.out
ql/src/test/results/clientnegative/exim_17_part_spec_underspec.q.out
ql/src/test/results/clientnegative/exim_18_part_spec_missing.q.out
ql/src/test/results/clientnegative/exim_19_external_over_existing.q.out
ql/src/test/results/clientnegative/exim_20_managed_location_over_existing.q.out
ql/src/test/results/clientnegative/exim_21_part_managed_external.q.out
ql/src/test/results/clientnegative/exim_23_import_exist_authfail.q.out
ql/src/test/results/clientnegative/exim_24_import_part_authfail.q.out
ql/src/test/results/clientnegative/exim_25_import_nonexist_authfail.q.out
ql/src/test/results/clientnegative/protectmode_part_no_drop.q.out
ql/src/test/results/clientnegative/protectmode_tbl2.q.out
ql/src/test/results/clientnegative/protectmode_tbl3.q.out
ql/src/test/results/clientnegative/protectmode_tbl4.q.out
ql/src/test/results/clientnegative/protectmode_tbl5.q.out
ql/src/test/results/clientnegative/protectmode_tbl_no_drop.q.out
ql/src/test/results/clientpositive/add_part_exist.q.out
ql/src/test/results/clientpositive/alter1.q.out
ql/src/test/results/clientpositive/alter2.q.out
ql/src/test/results/clientpositive/alter3.q.out
ql/src/test/results/clientpositive/alter4.q.out
ql/src/test/results/clientpositive/alter5.q.out
ql/src/test/results/clientpositive/alter_index.q.out
ql/src/test/results/clientpositive/alter_merge_2.q.out
ql/src/test/results/clientpositive/alter_merge_stats.q.out
ql/src/test/results/clientpositive/alter_partition_format_loc.q.out
ql/src/test/results/clientpositive/alter_rename_partition.q.out
ql/src/test/results/clientpositive/alter_rename_partition_authorization.q.out
ql/src/test/results/clientpositive/alter_table_serde.q.out
ql/src/test/results/clientpositive/alter_view_rename.q.out
ql/src/test/results/clientpositive/archive_corrupt.q.out
ql/src/test/results/clientpositive/authorization_5.q.out
ql/src/test/results/clientpositive/authorization_7.q.out
ql/src/test/results/clientpositive/autogen_colalias.q.out
ql/src/test/results/clientpositive/ba_table1.q.out
ql/src/test/results/clientpositive/ba_table2.q.out
ql/src/test/results/clientpositive/ba_table_udfs.q.out
ql/src/test/results/clientpositive/ba_table_union.q.out
ql/src/test/results/clientpositive/binary_table_bincolserde.q.out
ql/src/test/results/clientpositive/binary_table_colserde.q.out
ql/src/test/results/clientpositive/bucket_groupby.q.out
ql/src/test/results/clientpositive/columnarserde_create_shortcut.q.out
ql/src/test/results/clientpositive/combine2.q.out
ql/src/test/results/clientpositive/combine3.q.out
ql/src/test/results/clientpositive/create_1.q.out
ql/src/test/results/clientpositive/create_default_prop.q.out
ql/src/test/results/clientpositive/create_escape.q.out
ql/src/test/results/clientpositive/create_insert_outputformat.q.out
ql/src/test/results/clientpositive/create_like.q.out
ql/src/test/results/clientpositive/create_like_view.q.out
ql/src/test/results/clientpositive/create_nested_type.q.out
ql/src/test/results/clientpositive/create_or_replace_view.q.out
ql/src/test/results/clientpositive/create_view.q.out
ql/src/test/results/clientpositive/create_view_partitioned.q.out
ql/src/test/results/clientpositive/ctas.q.out
ql/src/test/results/clientpositive/database.q.out
ql/src/test/results/clientpositive/database_location.q.out
ql/src/test/results/clientpositive/database_properties.q.out
ql/src/test/results/clientpositive/ddltime.q.out
ql/src/test/results/clientpositive/default_partition_name.q.out
ql/src/test/results/clientpositive/describe_formatted_view_partitioned.q.out
ql/src/test/results/clientpositive/describe_table.q.out
ql/src/test/results/clientpositive/drop_multi_partitions.q.out
ql/src/test/results/clientpositive/exim_00_nonpart_empty.q.out
ql/src/test/results/clientpositive/exim_01_nonpart.q.out
ql/src/test/results/clientpositive/exim_02_00_part_empty.q.out
ql/src/test/results/clientpositive/exim_02_part.q.out
ql/src/test/results/clientpositive/exim_03_nonpart_over_compat.q.out
ql/src/test/results/clientpositive/exim_04_all_part.q.out
ql/src/test/results/clientpositive/exim_04_evolved_parts.q.out
ql/src/test/results/clientpositive/exim_05_some_part.q.out
ql/src/test/results/clientpositive/exim_06_one_part.q.out
ql/src/test/results/clientpositive/exim_07_all_part_over_nonoverlap.q.out
ql/src/test/results/clientpositive/exim_08_nonpart_rename.q.out
ql/src/test/results/clientpositive/exim_09_part_spec_nonoverlap.q.out
ql/src/test/results/clientpositive/exim_10_external_managed.q.out
ql/src/test/results/clientpositive/exim_11_managed_external.q.out
ql/src/test/results/clientpositive/exim_12_external_location.q.out
ql/src/test/results/clientpositive/exim_13_managed_location.q.out
ql/src/test/results/clientpositive/exim_14_managed_location_over_existing.q.out
ql/src/test/results/clientpositive/exim_15_external_part.q.out
ql/src/test/results/clientpositive/exim_16_part_external.q.out
ql/src/test/results/clientpositive/exim_17_part_managed.q.out
ql/src/test/results/clientpositive/exim_18_part_external.q.out
ql/src/test/results/clientpositive/exim_19_00_part_external_location.q.out
ql/src/test/results/clientpositive/exim_19_part_external_location.q.out
ql/src/test/results/clientpositive/exim_20_part_managed_location.q.out
ql/src/test/results/clientpositive/exim_22_import_exist_authsuccess.q.out
ql/src/test/results/clientpositive/exim_23_import_part_authsuccess.q.out
ql/src/test/results/clientpositive/exim_24_import_nonexist_authsuccess.q.out
ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
ql/src/test/results/clientpositive/fileformat_text.q.out
ql/src/test/results/clientpositive/index_creation.q.out
ql/src/test/results/clientpositive/inoutdriver.q.out
ql/src/test/results/clientpositive/input1.q.out
ql/src/test/results/clientpositive/input10.q.out
ql/src/test/results/clientpositive/input15.q.out
ql/src/test/results/clientpositive/input2.q.out
ql/src/test/results/clientpositive/input3.q.out
ql/src/test/results/clientpositive/input46.q.out
ql/src/test/results/clientpositive/input_part10.q.out
ql/src/test/results/clientpositive/inputddl2.q.out
ql/src/test/results/clientpositive/inputddl3.q.out
ql/src/test/results/clientpositive/inputddl4.q.out
ql/src/test/results/clientpositive/inputddl5.q.out
ql/src/test/results/clientpositive/inputddl6.q.out
ql/src/test/results/clientpositive/inputddl7.q.out
ql/src/test/results/clientpositive/inputddl8.q.out
ql/src/test/results/clientpositive/insert_into6.q.out
ql/src/test/results/clientpositive/join_thrift.q.out
ql/src/test/results/clientpositive/load_dyn_part1.q.out
ql/src/test/results/clientpositive/load_dyn_part10.q.out
ql/src/test/results/clientpositive/load_dyn_part11.q.out
ql/src/test/results/clientpositive/load_dyn_part12.q.out
ql/src/test/results/clientpositive/load_dyn_part13.q.out
ql/src/test/results/clientpositive/load_dyn_part14.q.out
ql/src/test/results/clientpositive/load_dyn_part15.q.out
ql/src/test/results/clientpositive/load_dyn_part2.q.out
ql/src/test/results/clientpositive/load_dyn_part3.q.out
ql/src/test/results/clientpositive/load_dyn_part4.q.out
ql/src/test/results/clientpositive/load_dyn_part5.q.out
ql/src/test/results/clientpositive/load_dyn_part6.q.out
ql/src/test/results/clientpositive/load_dyn_part7.q.out
ql/src/test/results/clientpositive/load_dyn_part8.q.out
ql/src/test/results/clientpositive/load_dyn_part9.q.out
ql/src/test/results/clientpositive/load_fs.q.out
ql/src/test/results/clientpositive/merge3.q.out
ql/src/test/results/clientpositive/merge4.q.out
ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out
ql/src/test/results/clientpositive/mi.q.out
ql/src/test/results/clientpositive/multi_sahooks.q.out
ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
ql/src/test/results/clientpositive/partition_schema1.q.out
ql/src/test/results/clientpositive/partition_special_char.q.out
ql/src/test/results/clientpositive/print_header.q.out
ql/src/test/results/clientpositive/protectmode.q.out
ql/src/test/results/clientpositive/protectmode2.q.out
ql/src/test/results/clientpositive/rcfile_bigdata.q.out
ql/src/test/results/clientpositive/rcfile_columnar.q.out
ql/src/test/results/clientpositive/rcfile_default_format.q.out
ql/src/test/results/clientpositive/rename_column.q.out
ql/src/test/results/clientpositive/show_partitions.q.out
ql/src/test/results/clientpositive/show_tables.q.out
ql/src/test/results/clientpositive/showparts.q.out
ql/src/test/results/clientpositive/split_sample.q.out
ql/src/test/results/clientpositive/stats0.q.out
ql/src/test/results/clientpositive/stats1.q.out
ql/src/test/results/clientpositive/stats10.q.out
ql/src/test/results/clientpositive/stats12.q.out
ql/src/test/results/clientpositive/stats13.q.out
ql/src/test/results/clientpositive/stats14.q.out
ql/src/test/results/clientpositive/stats15.q.out
ql/src/test/results/clientpositive/stats16.q.out
ql/src/test/results/clientpositive/stats2.q.out
ql/src/test/results/clientpositive/stats3.q.out
ql/src/test/results/clientpositive/stats4.q.out
ql/src/test/results/clientpositive/stats5.q.out
ql/src/test/results/clientpositive/stats6.q.out
ql/src/test/results/clientpositive/stats7.q.out
ql/src/test/results/clientpositive/stats8.q.out
ql/src/test/results/clientpositive/stats9.q.out
ql/src/test/results/clientpositive/tablename_with_select.q.out
ql/src/test/results/clientpositive/udf_map_keys.q.out
ql/src/test/results/clientpositive/udf_map_values.q.out
ql/src/test/results/clientpositive/udf_printf.q.out
ql/src/test/results/clientpositive/udf_sort_array.q.out
ql/src/test/results/clientpositive/updateAccessTime.q.out
> StorageHandler authorization providers
> --------------------------------------
>
> Key: HIVE-2809
> URL: https://issues.apache.org/jira/browse/HIVE-2809
> Project: Hive
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Attachments: HIVE-2809.D1953.1.patch, HIVE-2809.D1953.2.patch, HIVE-2809.D1953.3.patch, HIVE-2809.D1953.4.patch, HIVE-2809.D1953.5.patch
>
>
> In this issue, we would like to discuss the possibility of supplementing the Hive authorization model with authorization at the storage level. As discussed in HIVE-1943, Hive should also check for operation permissions in hdfs and hbase, since otherwise, data and metadata can be in an inconsistent state or be orphaned. Going a step further, some of the setups might not need the full featured auth model by Hive, but want to rely on managing the permissions at the data layer. In this model, the metadata operations are checked first from hdfs/hbase and it is allowed only if they are allowed at the data layer. The semantics are documented at https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Design.
> So, the goals of this issue are:
> - Port storage handler specific authorization providers, and the StorageDelegationAuthorizationProvider from HCATALOG-245 and HCATALOG-260 to Hive.
> - Keep current Hive's default authorization provider, and enable user to use this and/or the storage one. auth providers are already configurable.
> - Move the manual checks that had to be performed about authorization in Hcat to Hive, specifically:
> -- CREATE DATABASE/TABLE, ADD PARTITION statements does not call
> HiveAuthorizationProvider.authorize() with the candidate objects, which means that
> we cannot do checks against defined LOCATION.
> -- HiveOperation does not define sufficient Privileges for most of the operations,
> especially database operations.
> -- For some of the operations, Hive SemanticAnalyzer does not add the changed
> object as a WriteEntity or ReadEntity.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira