You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Oleksiy Sayankin (Jira)" <ji...@apache.org> on 2020/02/25 10:40:00 UTC
[jira] [Comment Edited] (HIVE-22919)
StorageBasedAuthorizationProvider does not allow create databases after
changing hive.metastore.warehouse.dir
[ https://issues.apache.org/jira/browse/HIVE-22919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041814#comment-17041814 ]
Oleksiy Sayankin edited comment on HIVE-22919 at 2/25/20 10:39 AM:
-------------------------------------------------------------------
*FIXED*
*ROOT-CAUSE*
The root-cause of the issue does not relate to Storage Base Authorization, it is about correct update of Hive variable {{hive.metastore.warehouse.dir}}. As one can see from the exception:
{code}
FAILED: HiveException org.apache.hadoop.security.AccessControlException: User testuser1(user id 5001) does not have access to hdfs:/tmp/m2/m3.db
{code}
Hive wants to create new database at {{hdfs:/tmp/m2}} despite the operator {{SET}} that was executed before
{code}
SET hive.metastore.warehouse.dir=/tmp/m3;
{code}
This happens because {{StorageBasedAuthorizationProvider}} has an instance of {{Warehouse}} object that has value for {{hive.metastore.warehouse.dir}}. When a user updates the {{hive.metastore.warehouse.dir}} in {{Configuration}} instance, this action does not force {{Warehouse}} object to refresh the value of {{hive.metastore.warehouse.dir}} and hence it has the old one.
*SOLUTION*
Add {{isWarehouseChanged()}} method to check whether {{hive.metastore.warehouse.dir}} has been changed and recreate {{Warehouse}} in {{StorageBasedAuthorizationProvider}} if yes.
*EFFECTS*
{{StorageBasedAuthorizationProvider}} initialization.
was (Author: osayankin):
*FIXED*
*ROOT-CAUSE*
The root-cause of the issue does not relate to Storage Base Authorization, it is about correct update of Hive variable {{hive.metastore.warehouse.dir}}. As one can see from the exception:
{code}
FAILED: HiveException org.apache.hadoop.security.AccessControlException: User testuser1(user id 5001) does not have access to hdfs:/tmp/m2/m3.db
{code}
Hive wants to create new database at {{hdfs:/tmp/m2}} despite the operator {{SET}} that was executed before
{code}
SET hive.metastore.warehouse.dir=/tmp/m3;
{code}
This happens because {{StorageBasedAuthorizationProvider}} has an instance of {{Warehouse}} object that has value for {{hive.metastore.warehouse.dir}}. When a user updates the {{hive.metastore.warehouse.dir}} in {{HiveConf}} instance, this action does not force {{Warehouse}} object to refresh the value of {{hive.metastore.warehouse.dir}} and hence it has the old one.
*SOLUTION*
Add {{isWarehouseChanged()}} method to check whether {{hive.metastore.warehouse.dir}} has been changed and recreate {{Warehouse}} in {{StorageBasedAuthorizationProvider}} if yes.
*EFFECTS*
{{StorageBasedAuthorizationProvider}} initialization.
> StorageBasedAuthorizationProvider does not allow create databases after changing hive.metastore.warehouse.dir
> -------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-22919
> URL: https://issues.apache.org/jira/browse/HIVE-22919
> Project: Hive
> Issue Type: Bug
> Reporter: Oleksiy Sayankin
> Assignee: Oleksiy Sayankin
> Priority: Major
> Attachments: HIVE-22919.1.patch, HIVE-22919.2.patch, HIVE-22919.3.patch
>
>
> *ENVIRONMENT:*
> Hive-2.3
> *STEPS TO REPRODUCE:*
> 1. Configure Storage Based Authorization:
> {code:xml}<property>
> <name>hive.security.authorization.enabled</name>
> <value>true</value>
> </property>
> <property>
> <name>hive.security.metastore.authorization.manager</name>
> <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
> </property>
> <property>
> <name>hive.security.authorization.manager</name>
> <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
> </property>
> <property>
> <name>hive.security.metastore.authenticator.manager</name>
> <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value>
> </property>
> <property>
> <name>hive.metastore.pre.event.listeners</name>
> <value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value>
> </property>{code}
> 2. Create a few directories, change owners and permissions to it:
> {code:java}hadoop fs -mkdir /tmp/m1
> hadoop fs -mkdir /tmp/m2
> hadoop fs -mkdir /tmp/m3
> hadoop fs -chown testuser1:testuser1 /tmp/m[1,3]
> hadoop fs -chmod 700 /tmp/m[1-3]{code}
> 3. Check permissions:
> {code:java}[test@node2 ~]$ hadoop fs -ls /tmp|grep m[1-3]
> drwx------ - testuser1 testuser1 0 2020-02-11 10:25 /tmp/m1
> drwx------ - test test 0 2020-02-11 10:25 /tmp/m2
> drwx------ - testuser1 testuser1 1 2020-02-11 10:36 /tmp/m3
> [test@node2 ~]$
> {code}
> 4. Loggin into Hive CLI using embedded Hive Metastore as *"testuser1"* user, with *"hive.metastore.warehouse.dir"* set to *"/tmp/m1"*:
> {code:java}
> sudo -u testuser1 hive --hiveconf hive.metastore.uris= --hiveconf hive.metastore.warehouse.dir=/tmp/m1
> {code}
> 5. Perform the next steps:
> {code:sql}-- 1. Check "hive.metastore.warehouse.dir" value:
> SET hive.metastore.warehouse.dir;
> -- 2. Set "hive.metastore.warehouse.dir" to the path, to which "testuser1" user does not have an access:
> SET hive.metastore.warehouse.dir=/tmp/m2;
> -- 3. Try to create a database:
> CREATE DATABASE m2;
> -- 4. Set "hive.metastore.warehouse.dir" to the path, to which "testuser1" user has an access:
> SET hive.metastore.warehouse.dir=/tmp/m3;
> -- 5. Try to create a database:
> CREATE DATABASE m3;
> {code}
> *ACTUAL RESULT:*
> Query 5 fails with an exception below. It does not handle "hive.metastore.warehouse.dir" proprty:
> {code:java}
> hive> -- 5. Try to create a database:
> hive> CREATE DATABASE m3;
> FAILED: HiveException org.apache.hadoop.security.AccessControlException: User testuser1(user id 5001) does not have access to hdfs:/tmp/m2/m3.db
> hive>
> {code}
> *EXPECTED RESULT:*
> Query 5 creates a database;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)