You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Alex Holmes <gr...@gmail.com> on 2011/08/16 23:55:29 UTC
Hive 0.7.1 authorization woes
Hi all,
I've been struggling with getting Hive authorization to work for a few
hours, and I really hope someone can help me. I installed Hive 0.7.1
on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
configured Hive to enable authorization:
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
I kept all the other Hive security configs with their default settings.
I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
metastore and the Hive CLI are all running as the same user (the HDFS
superuser). Here are the sequence of steps that are causing me issues.
Without authorization everything works perfectly (creating, loading, selecting).
I've also tried creating and loading the table without authorization, granting
the select privilege at various levels (global, table, database), turning on
auth and performing the select, resulting in the same exception.
Any help with this would be greatly appreciated!
Thanks,
Alex
--
[hduser@aholmes-desktop ~]$ hive
Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
hive> set hive.security.authorization.enabled=false;
hive> grant all to user hduser;
OK
Time taken: 0.233 seconds
hive> set hive.security.authorization.enabled=true;
hive> CREATE TABLE pokes3 (foo INT, bar STRING);
FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
get_privilege_set failed: unknown result)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: get_privilege_set failed:
unknown result
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: get_privilege_set
failed: unknown result
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
... 14 more
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
Hi,
hive> CREATE TABLE pokes2 (foo INT, bar STRING);
OK
hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes2;
OK
hive> grant select on table pokes2 to user hduser;
OK
hive> set hive.security.authorization.enabled=true;
hive> show grant user hduser on table pokes2;
OK
database default
table pokes2
principalName hduser
principalType USER
privilege Select
grantTime 1314318185
grantor hduser
Time taken: 0.041 seconds
hive> select * from pokes2;
FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
get_privilege_set failed: unknown result)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: get_privilege_set failed:
unknown result
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserDBAndTable(DefaultHiveAuthorizationProvider.java:259)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:159)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:531)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: get_privilege_set
failed: unknown result
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
... 15 more
On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <he...@gmail.com> wrote:
> this is what i have tried with a remote metastore:
>
> > set hive.security.authorization.enabled=false;
> hive>
> >
> >
> > drop table src2;
> OK
> Time taken: 1.002 seconds
> hive> create table src2 (key int, value string);
> OK
> Time taken: 0.03 seconds
> hive>
> >
> >
> > set hive.security.authorization.enabled=true;
> hive> grant select on table src2 to user heyongqiang;
> OK
> Time taken: 0.113 seconds
> hive> select * from src2;
> OK
> Time taken: 0.188 seconds
> hive> show grant user heyongqiang on table src2;
> OK
>
> database default
> table src2
> principalName heyongqiang
> principalType USER
> privilege Select
> grantTime Wed Aug 24 15:03:51 PDT 2011
> grantor heyongqiang
>
> can u do a show grant?
>
> (But with remote metastore, i think hive should not return empty list
> instead of null for list_privileges etc.)
>
>
>
> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <gr...@gmail.com> wrote:
>> Authorization works for me with the local metastore. The remote
>> metastore works with authorization turned off, but as soon as I turn
>> it on and issue any commands I get these exceptions on the hive
>> client.
>>
>> Could you also try the remote metastore please? I'm pretty sure that
>> authorization does not work with it at all.
>>
>> Thanks,
>> Alex
>>
>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
>>> I am using local metastore, and can not reproduce the problem.
>>>
>>> what message did you get when running local metastore?
>>>
>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>> Thanks for opening a ticket.
>>>>
>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>> that the bug is only related to global grants).
>>>>
>>>> hive> set hive.security.authorization.enabled=false;
>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>> OK
>>>> Time taken: 1.245 seconds
>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>>>> Copying data from file:/app/hadoop/hive1.in
>>>> Copying file: file:/app/hadoop/hive1.in
>>>> Loading data to table default.pokes
>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>> OK
>>>> Time taken: 0.33 seconds
>>>> hive> select * from pokes;
>>>> OK
>>>> 1 a
>>>> 2 b
>>>> 3 c
>>>> Time taken: 0.095 seconds
>>>> hive> grant select on table pokes to user hduser;
>>>> OK
>>>> Time taken: 0.251 seconds
>>>> hive> set hive.security.authorization.enabled=true;
>>>> hive> select * from pokes;
>>>> FAILED: Hive Internal Error:
>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>> get_privilege_set failed: unknown result)
>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>> unknown result
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>> ...
>>>>
>>>> mysql> select * from TBL_PRIVS;
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>> | 1 | 1314219701 | 0 | hduser | USER |
>>>> hduser | USER | Select | 1 |
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>
>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>
>>>> Authorization failed:No privilege 'Create' found for outputs {
>>>> database:default}. Use show grant to get more details.
>>>>
>>>> Whereas I just get an exception (as you can see above). Were you also
>>>> running with the remote metastore? I get these meaningful messages
>>>> with the local metastore (and authorization on), but with the remote
>>>> metastore with authorization turned on, I always get exceptions.
>>>>
>>>> Many thanks,
>>>> Alex
>>>>
>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>
>>>>> thanks for reporting this one!
>>>>>
>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>> I created the mysql database (with the simple create database command)
>>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>>> some grant information and what I see in the database:
>>>>>>
>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>> hive> grant all to user hduser;
>>>>>> OK
>>>>>> Time taken: 0.334 seconds
>>>>>> hive> show grant user hduser;
>>>>>> OK
>>>>>>
>>>>>> principalName hduser
>>>>>> principalType USER
>>>>>> privilege All
>>>>>> grantTime 1314191500
>>>>>> grantor hduser
>>>>>> Time taken: 0.046 seconds
>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>> FAILED: Hive Internal Error:
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>> get_privilege_set failed: unknown result)
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>> unknown result
>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>> ...
>>>>>>
>>>>>> mysql> use hive;
>>>>>> Database changed
>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>>> | hduser | USER | All |
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> 1 row in set (0.00 sec)
>>>>>>
>>>>>>
>>>>>> Thanks for your help,
>>>>>> Alex
>>>>>>
>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>>> do a show grant?
>>>>>>>
>>>>>>> thanks
>>>>>>> yongqiang
>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>>> configured Hive to enable authorization:
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>>> <value>true</value>
>>>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>>>
>>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>>
>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>> hive> grant all to user hduser;
>>>>>>>> OK
>>>>>>>> Time taken: 0.233 seconds
>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>> unknown result
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>> failed: unknown result
>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>> ... 14 more
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
HDFS, the Hive metastore and the hive client are all running as "hduser".
On Thu, Aug 25, 2011 at 8:22 PM, yongqiang he <he...@gmail.com> wrote:
> what is your unix name on that machine? can u do a whoami?
>
> On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes <gr...@gmail.com> wrote:
>> Here's the hive-site.xml file (I use the same file for both the client
>> and remote metastore). We're using mysql as the metastore DB.
>>
>>
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> <configuration>
>> <property>
>> <name>hive.security.authorization.enabled</name>
>> <value>true</value>
>> </property>
>> <property>
>> <name>hive.metastore.local</name>
>> <value>false</value>
>> </property>
>> <property>
>> <name>hive.metastore.uris</name>
>> <value>thrift://localhost:9083</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionURL</name>
>> <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionDriverName</name>
>> <value>com.mysql.jdbc.Driver</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionUserName</name>
>> <value>hive</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionPassword</name>
>> <value>secret</value>
>> </property>
>> </configuration>
>>
>>
>>
>> On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <he...@gmail.com> wrote:
>>> this is what i have tried with a remote metastore:
>>>
>>> > set hive.security.authorization.enabled=false;
>>> hive>
>>> >
>>> >
>>> > drop table src2;
>>> OK
>>> Time taken: 1.002 seconds
>>> hive> create table src2 (key int, value string);
>>> OK
>>> Time taken: 0.03 seconds
>>> hive>
>>> >
>>> >
>>> > set hive.security.authorization.enabled=true;
>>> hive> grant select on table src2 to user heyongqiang;
>>> OK
>>> Time taken: 0.113 seconds
>>> hive> select * from src2;
>>> OK
>>> Time taken: 0.188 seconds
>>> hive> show grant user heyongqiang on table src2;
>>> OK
>>>
>>> database default
>>> table src2
>>> principalName heyongqiang
>>> principalType USER
>>> privilege Select
>>> grantTime Wed Aug 24 15:03:51 PDT 2011
>>> grantor heyongqiang
>>>
>>> can u do a show grant?
>>>
>>> (But with remote metastore, i think hive should not return empty list
>>> instead of null for list_privileges etc.)
>>>
>>>
>>>
>>> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>> Authorization works for me with the local metastore. The remote
>>>> metastore works with authorization turned off, but as soon as I turn
>>>> it on and issue any commands I get these exceptions on the hive
>>>> client.
>>>>
>>>> Could you also try the remote metastore please? I'm pretty sure that
>>>> authorization does not work with it at all.
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
>>>>> I am using local metastore, and can not reproduce the problem.
>>>>>
>>>>> what message did you get when running local metastore?
>>>>>
>>>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>> Thanks for opening a ticket.
>>>>>>
>>>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>>>> that the bug is only related to global grants).
>>>>>>
>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>> OK
>>>>>> Time taken: 1.245 seconds
>>>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>>>>>> Copying data from file:/app/hadoop/hive1.in
>>>>>> Copying file: file:/app/hadoop/hive1.in
>>>>>> Loading data to table default.pokes
>>>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>>>> OK
>>>>>> Time taken: 0.33 seconds
>>>>>> hive> select * from pokes;
>>>>>> OK
>>>>>> 1 a
>>>>>> 2 b
>>>>>> 3 c
>>>>>> Time taken: 0.095 seconds
>>>>>> hive> grant select on table pokes to user hduser;
>>>>>> OK
>>>>>> Time taken: 0.251 seconds
>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>> hive> select * from pokes;
>>>>>> FAILED: Hive Internal Error:
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>> get_privilege_set failed: unknown result)
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>> unknown result
>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>> ...
>>>>>>
>>>>>> mysql> select * from TBL_PRIVS;
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>> | 1 | 1314219701 | 0 | hduser | USER |
>>>>>> hduser | USER | Select | 1 |
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>>
>>>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>>>
>>>>>> Authorization failed:No privilege 'Create' found for outputs {
>>>>>> database:default}. Use show grant to get more details.
>>>>>>
>>>>>> Whereas I just get an exception (as you can see above). Were you also
>>>>>> running with the remote metastore? I get these meaningful messages
>>>>>> with the local metastore (and authorization on), but with the remote
>>>>>> metastore with authorization turned on, I always get exceptions.
>>>>>>
>>>>>> Many thanks,
>>>>>> Alex
>>>>>>
>>>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>>>
>>>>>>> thanks for reporting this one!
>>>>>>>
>>>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>>> I created the mysql database (with the simple create database command)
>>>>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>>>>> some grant information and what I see in the database:
>>>>>>>>
>>>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>>>> hive> grant all to user hduser;
>>>>>>>> OK
>>>>>>>> Time taken: 0.334 seconds
>>>>>>>> hive> show grant user hduser;
>>>>>>>> OK
>>>>>>>>
>>>>>>>> principalName hduser
>>>>>>>> principalType USER
>>>>>>>> privilege All
>>>>>>>> grantTime 1314191500
>>>>>>>> grantor hduser
>>>>>>>> Time taken: 0.046 seconds
>>>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>> unknown result
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>> ...
>>>>>>>>
>>>>>>>> mysql> use hive;
>>>>>>>> Database changed
>>>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>>>>> | hduser | USER | All |
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> 1 row in set (0.00 sec)
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for your help,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>>>>> do a show grant?
>>>>>>>>>
>>>>>>>>> thanks
>>>>>>>>> yongqiang
>>>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>>>>> configured Hive to enable authorization:
>>>>>>>>>>
>>>>>>>>>> <property>
>>>>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>>>>> <value>true</value>
>>>>>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>>>>>
>>>>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>>>>
>>>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>>>> hive> grant all to user hduser;
>>>>>>>>>> OK
>>>>>>>>>> Time taken: 0.233 seconds
>>>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>>>> unknown result
>>>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>>>> failed: unknown result
>>>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>>>> ... 14 more
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by yongqiang he <he...@gmail.com>.
what is your unix name on that machine? can u do a whoami?
On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes <gr...@gmail.com> wrote:
> Here's the hive-site.xml file (I use the same file for both the client
> and remote metastore). We're using mysql as the metastore DB.
>
>
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <configuration>
> <property>
> <name>hive.security.authorization.enabled</name>
> <value>true</value>
> </property>
> <property>
> <name>hive.metastore.local</name>
> <value>false</value>
> </property>
> <property>
> <name>hive.metastore.uris</name>
> <value>thrift://localhost:9083</value>
> </property>
> <property>
> <name>javax.jdo.option.ConnectionURL</name>
> <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
> </property>
> <property>
> <name>javax.jdo.option.ConnectionDriverName</name>
> <value>com.mysql.jdbc.Driver</value>
> </property>
> <property>
> <name>javax.jdo.option.ConnectionUserName</name>
> <value>hive</value>
> </property>
> <property>
> <name>javax.jdo.option.ConnectionPassword</name>
> <value>secret</value>
> </property>
> </configuration>
>
>
>
> On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <he...@gmail.com> wrote:
>> this is what i have tried with a remote metastore:
>>
>> > set hive.security.authorization.enabled=false;
>> hive>
>> >
>> >
>> > drop table src2;
>> OK
>> Time taken: 1.002 seconds
>> hive> create table src2 (key int, value string);
>> OK
>> Time taken: 0.03 seconds
>> hive>
>> >
>> >
>> > set hive.security.authorization.enabled=true;
>> hive> grant select on table src2 to user heyongqiang;
>> OK
>> Time taken: 0.113 seconds
>> hive> select * from src2;
>> OK
>> Time taken: 0.188 seconds
>> hive> show grant user heyongqiang on table src2;
>> OK
>>
>> database default
>> table src2
>> principalName heyongqiang
>> principalType USER
>> privilege Select
>> grantTime Wed Aug 24 15:03:51 PDT 2011
>> grantor heyongqiang
>>
>> can u do a show grant?
>>
>> (But with remote metastore, i think hive should not return empty list
>> instead of null for list_privileges etc.)
>>
>>
>>
>> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <gr...@gmail.com> wrote:
>>> Authorization works for me with the local metastore. The remote
>>> metastore works with authorization turned off, but as soon as I turn
>>> it on and issue any commands I get these exceptions on the hive
>>> client.
>>>
>>> Could you also try the remote metastore please? I'm pretty sure that
>>> authorization does not work with it at all.
>>>
>>> Thanks,
>>> Alex
>>>
>>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
>>>> I am using local metastore, and can not reproduce the problem.
>>>>
>>>> what message did you get when running local metastore?
>>>>
>>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>> Thanks for opening a ticket.
>>>>>
>>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>>> that the bug is only related to global grants).
>>>>>
>>>>> hive> set hive.security.authorization.enabled=false;
>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>> OK
>>>>> Time taken: 1.245 seconds
>>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>>>>> Copying data from file:/app/hadoop/hive1.in
>>>>> Copying file: file:/app/hadoop/hive1.in
>>>>> Loading data to table default.pokes
>>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>>> OK
>>>>> Time taken: 0.33 seconds
>>>>> hive> select * from pokes;
>>>>> OK
>>>>> 1 a
>>>>> 2 b
>>>>> 3 c
>>>>> Time taken: 0.095 seconds
>>>>> hive> grant select on table pokes to user hduser;
>>>>> OK
>>>>> Time taken: 0.251 seconds
>>>>> hive> set hive.security.authorization.enabled=true;
>>>>> hive> select * from pokes;
>>>>> FAILED: Hive Internal Error:
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>> get_privilege_set failed: unknown result)
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>> unknown result
>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>> ...
>>>>>
>>>>> mysql> select * from TBL_PRIVS;
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>> | 1 | 1314219701 | 0 | hduser | USER |
>>>>> hduser | USER | Select | 1 |
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>
>>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>>
>>>>> Authorization failed:No privilege 'Create' found for outputs {
>>>>> database:default}. Use show grant to get more details.
>>>>>
>>>>> Whereas I just get an exception (as you can see above). Were you also
>>>>> running with the remote metastore? I get these meaningful messages
>>>>> with the local metastore (and authorization on), but with the remote
>>>>> metastore with authorization turned on, I always get exceptions.
>>>>>
>>>>> Many thanks,
>>>>> Alex
>>>>>
>>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>>
>>>>>> thanks for reporting this one!
>>>>>>
>>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>> I created the mysql database (with the simple create database command)
>>>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>>>> some grant information and what I see in the database:
>>>>>>>
>>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>>> hive> grant all to user hduser;
>>>>>>> OK
>>>>>>> Time taken: 0.334 seconds
>>>>>>> hive> show grant user hduser;
>>>>>>> OK
>>>>>>>
>>>>>>> principalName hduser
>>>>>>> principalType USER
>>>>>>> privilege All
>>>>>>> grantTime 1314191500
>>>>>>> grantor hduser
>>>>>>> Time taken: 0.046 seconds
>>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>>> FAILED: Hive Internal Error:
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>> get_privilege_set failed: unknown result)
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>> unknown result
>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>> ...
>>>>>>>
>>>>>>> mysql> use hive;
>>>>>>> Database changed
>>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>>>> | hduser | USER | All |
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> 1 row in set (0.00 sec)
>>>>>>>
>>>>>>>
>>>>>>> Thanks for your help,
>>>>>>> Alex
>>>>>>>
>>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>>>> do a show grant?
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> yongqiang
>>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>>>> configured Hive to enable authorization:
>>>>>>>>>
>>>>>>>>> <property>
>>>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>>>> <value>true</value>
>>>>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>>>>
>>>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>>>
>>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>>> hive> grant all to user hduser;
>>>>>>>>> OK
>>>>>>>>> Time taken: 0.233 seconds
>>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>>> unknown result
>>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>>> failed: unknown result
>>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>>> ... 14 more
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
Here's the hive-site.xml file (I use the same file for both the client
and remote metastore). We're using mysql as the metastore DB.
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.local</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://localhost:9083</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>secret</value>
</property>
</configuration>
On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <he...@gmail.com> wrote:
> this is what i have tried with a remote metastore:
>
> > set hive.security.authorization.enabled=false;
> hive>
> >
> >
> > drop table src2;
> OK
> Time taken: 1.002 seconds
> hive> create table src2 (key int, value string);
> OK
> Time taken: 0.03 seconds
> hive>
> >
> >
> > set hive.security.authorization.enabled=true;
> hive> grant select on table src2 to user heyongqiang;
> OK
> Time taken: 0.113 seconds
> hive> select * from src2;
> OK
> Time taken: 0.188 seconds
> hive> show grant user heyongqiang on table src2;
> OK
>
> database default
> table src2
> principalName heyongqiang
> principalType USER
> privilege Select
> grantTime Wed Aug 24 15:03:51 PDT 2011
> grantor heyongqiang
>
> can u do a show grant?
>
> (But with remote metastore, i think hive should not return empty list
> instead of null for list_privileges etc.)
>
>
>
> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <gr...@gmail.com> wrote:
>> Authorization works for me with the local metastore. The remote
>> metastore works with authorization turned off, but as soon as I turn
>> it on and issue any commands I get these exceptions on the hive
>> client.
>>
>> Could you also try the remote metastore please? I'm pretty sure that
>> authorization does not work with it at all.
>>
>> Thanks,
>> Alex
>>
>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
>>> I am using local metastore, and can not reproduce the problem.
>>>
>>> what message did you get when running local metastore?
>>>
>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>> Thanks for opening a ticket.
>>>>
>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>> that the bug is only related to global grants).
>>>>
>>>> hive> set hive.security.authorization.enabled=false;
>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>> OK
>>>> Time taken: 1.245 seconds
>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>>>> Copying data from file:/app/hadoop/hive1.in
>>>> Copying file: file:/app/hadoop/hive1.in
>>>> Loading data to table default.pokes
>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>> OK
>>>> Time taken: 0.33 seconds
>>>> hive> select * from pokes;
>>>> OK
>>>> 1 a
>>>> 2 b
>>>> 3 c
>>>> Time taken: 0.095 seconds
>>>> hive> grant select on table pokes to user hduser;
>>>> OK
>>>> Time taken: 0.251 seconds
>>>> hive> set hive.security.authorization.enabled=true;
>>>> hive> select * from pokes;
>>>> FAILED: Hive Internal Error:
>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>> get_privilege_set failed: unknown result)
>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>> unknown result
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>> ...
>>>>
>>>> mysql> select * from TBL_PRIVS;
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>> | 1 | 1314219701 | 0 | hduser | USER |
>>>> hduser | USER | Select | 1 |
>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>
>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>
>>>> Authorization failed:No privilege 'Create' found for outputs {
>>>> database:default}. Use show grant to get more details.
>>>>
>>>> Whereas I just get an exception (as you can see above). Were you also
>>>> running with the remote metastore? I get these meaningful messages
>>>> with the local metastore (and authorization on), but with the remote
>>>> metastore with authorization turned on, I always get exceptions.
>>>>
>>>> Many thanks,
>>>> Alex
>>>>
>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>
>>>>> thanks for reporting this one!
>>>>>
>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>> I created the mysql database (with the simple create database command)
>>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>>> some grant information and what I see in the database:
>>>>>>
>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>> hive> grant all to user hduser;
>>>>>> OK
>>>>>> Time taken: 0.334 seconds
>>>>>> hive> show grant user hduser;
>>>>>> OK
>>>>>>
>>>>>> principalName hduser
>>>>>> principalType USER
>>>>>> privilege All
>>>>>> grantTime 1314191500
>>>>>> grantor hduser
>>>>>> Time taken: 0.046 seconds
>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>> FAILED: Hive Internal Error:
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>> get_privilege_set failed: unknown result)
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>> unknown result
>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>> ...
>>>>>>
>>>>>> mysql> use hive;
>>>>>> Database changed
>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>>> | hduser | USER | All |
>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>> 1 row in set (0.00 sec)
>>>>>>
>>>>>>
>>>>>> Thanks for your help,
>>>>>> Alex
>>>>>>
>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>>> do a show grant?
>>>>>>>
>>>>>>> thanks
>>>>>>> yongqiang
>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>>> configured Hive to enable authorization:
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>>> <value>true</value>
>>>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>>>
>>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>>
>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>> hive> grant all to user hduser;
>>>>>>>> OK
>>>>>>>> Time taken: 0.233 seconds
>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>> unknown result
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>> failed: unknown result
>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>> ... 14 more
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by yongqiang he <he...@gmail.com>.
this is what i have tried with a remote metastore:
> set hive.security.authorization.enabled=false;
hive>
>
>
> drop table src2;
OK
Time taken: 1.002 seconds
hive> create table src2 (key int, value string);
OK
Time taken: 0.03 seconds
hive>
>
>
> set hive.security.authorization.enabled=true;
hive> grant select on table src2 to user heyongqiang;
OK
Time taken: 0.113 seconds
hive> select * from src2;
OK
Time taken: 0.188 seconds
hive> show grant user heyongqiang on table src2;
OK
database default
table src2
principalName heyongqiang
principalType USER
privilege Select
grantTime Wed Aug 24 15:03:51 PDT 2011
grantor heyongqiang
can u do a show grant?
(But with remote metastore, i think hive should not return empty list
instead of null for list_privileges etc.)
On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <gr...@gmail.com> wrote:
> Authorization works for me with the local metastore. The remote
> metastore works with authorization turned off, but as soon as I turn
> it on and issue any commands I get these exceptions on the hive
> client.
>
> Could you also try the remote metastore please? I'm pretty sure that
> authorization does not work with it at all.
>
> Thanks,
> Alex
>
> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
>> I am using local metastore, and can not reproduce the problem.
>>
>> what message did you get when running local metastore?
>>
>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>>> Thanks for opening a ticket.
>>>
>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>> that the bug is only related to global grants).
>>>
>>> hive> set hive.security.authorization.enabled=false;
>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>> OK
>>> Time taken: 1.245 seconds
>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>>> Copying data from file:/app/hadoop/hive1.in
>>> Copying file: file:/app/hadoop/hive1.in
>>> Loading data to table default.pokes
>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>> OK
>>> Time taken: 0.33 seconds
>>> hive> select * from pokes;
>>> OK
>>> 1 a
>>> 2 b
>>> 3 c
>>> Time taken: 0.095 seconds
>>> hive> grant select on table pokes to user hduser;
>>> OK
>>> Time taken: 0.251 seconds
>>> hive> set hive.security.authorization.enabled=true;
>>> hive> select * from pokes;
>>> FAILED: Hive Internal Error:
>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>> get_privilege_set failed: unknown result)
>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>> unknown result
>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>> ...
>>>
>>> mysql> select * from TBL_PRIVS;
>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>> | 1 | 1314219701 | 0 | hduser | USER |
>>> hduser | USER | Select | 1 |
>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>
>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>
>>> Authorization failed:No privilege 'Create' found for outputs {
>>> database:default}. Use show grant to get more details.
>>>
>>> Whereas I just get an exception (as you can see above). Were you also
>>> running with the remote metastore? I get these meaningful messages
>>> with the local metastore (and authorization on), but with the remote
>>> metastore with authorization turned on, I always get exceptions.
>>>
>>> Many thanks,
>>> Alex
>>>
>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>
>>>> thanks for reporting this one!
>>>>
>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>>> I created the mysql database (with the simple create database command)
>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>> some grant information and what I see in the database:
>>>>>
>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>> hive> grant all to user hduser;
>>>>> OK
>>>>> Time taken: 0.334 seconds
>>>>> hive> show grant user hduser;
>>>>> OK
>>>>>
>>>>> principalName hduser
>>>>> principalType USER
>>>>> privilege All
>>>>> grantTime 1314191500
>>>>> grantor hduser
>>>>> Time taken: 0.046 seconds
>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>> FAILED: Hive Internal Error:
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>> get_privilege_set failed: unknown result)
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>> unknown result
>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>> ...
>>>>>
>>>>> mysql> use hive;
>>>>> Database changed
>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>> | hduser | USER | All |
>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>> 1 row in set (0.00 sec)
>>>>>
>>>>>
>>>>> Thanks for your help,
>>>>> Alex
>>>>>
>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>> do a show grant?
>>>>>>
>>>>>> thanks
>>>>>> yongqiang
>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>> configured Hive to enable authorization:
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>> <value>true</value>
>>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>>> </property>
>>>>>>>
>>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>>
>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>
>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Alex
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>> hive> grant all to user hduser;
>>>>>>> OK
>>>>>>> Time taken: 0.233 seconds
>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>> FAILED: Hive Internal Error:
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>> get_privilege_set failed: unknown result)
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>> unknown result
>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>> failed: unknown result
>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>> ... 14 more
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
Authorization works for me with the local metastore. The remote
metastore works with authorization turned off, but as soon as I turn
it on and issue any commands I get these exceptions on the hive
client.
Could you also try the remote metastore please? I'm pretty sure that
authorization does not work with it at all.
Thanks,
Alex
On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <he...@gmail.com> wrote:
> I am using local metastore, and can not reproduce the problem.
>
> what message did you get when running local metastore?
>
> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
>> Thanks for opening a ticket.
>>
>> Table-level grants aren't working for me either (HIVE-2405 suggests
>> that the bug is only related to global grants).
>>
>> hive> set hive.security.authorization.enabled=false;
>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>> OK
>> Time taken: 1.245 seconds
>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
>> Copying data from file:/app/hadoop/hive1.in
>> Copying file: file:/app/hadoop/hive1.in
>> Loading data to table default.pokes
>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>> OK
>> Time taken: 0.33 seconds
>> hive> select * from pokes;
>> OK
>> 1 a
>> 2 b
>> 3 c
>> Time taken: 0.095 seconds
>> hive> grant select on table pokes to user hduser;
>> OK
>> Time taken: 0.251 seconds
>> hive> set hive.security.authorization.enabled=true;
>> hive> select * from pokes;
>> FAILED: Hive Internal Error:
>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>> get_privilege_set failed: unknown result)
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>> unknown result
>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>> ...
>>
>> mysql> select * from TBL_PRIVS;
>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>> | 1 | 1314219701 | 0 | hduser | USER |
>> hduser | USER | Select | 1 |
>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>
>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>
>> Authorization failed:No privilege 'Create' found for outputs {
>> database:default}. Use show grant to get more details.
>>
>> Whereas I just get an exception (as you can see above). Were you also
>> running with the remote metastore? I get these meaningful messages
>> with the local metastore (and authorization on), but with the remote
>> metastore with authorization turned on, I always get exceptions.
>>
>> Many thanks,
>> Alex
>>
>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>
>>> thanks for reporting this one!
>>>
>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>>> I created the mysql database (with the simple create database command)
>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>> some grant information and what I see in the database:
>>>>
>>>> [hduser@aholmes-desktop conf]$ hive
>>>> hive> grant all to user hduser;
>>>> OK
>>>> Time taken: 0.334 seconds
>>>> hive> show grant user hduser;
>>>> OK
>>>>
>>>> principalName hduser
>>>> principalType USER
>>>> privilege All
>>>> grantTime 1314191500
>>>> grantor hduser
>>>> Time taken: 0.046 seconds
>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>> FAILED: Hive Internal Error:
>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>> get_privilege_set failed: unknown result)
>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>> unknown result
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>> ...
>>>>
>>>> mysql> use hive;
>>>> Database changed
>>>> mysql> select * from GLOBAL_PRIVS;
>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>> | hduser | USER | All |
>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>> 1 row in set (0.00 sec)
>>>>
>>>>
>>>> Thanks for your help,
>>>> Alex
>>>>
>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>> do a show grant?
>>>>>
>>>>> thanks
>>>>> yongqiang
>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>> configured Hive to enable authorization:
>>>>>>
>>>>>> <property>
>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>> <value>true</value>
>>>>>> <description>enable or disable the hive client authorization</description>
>>>>>> </property>
>>>>>>
>>>>>> I kept all the other Hive security configs with their default settings.
>>>>>>
>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>>> I've also tried creating and loading the table without authorization, granting
>>>>>> the select privilege at various levels (global, table, database), turning on
>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>
>>>>>> Any help with this would be greatly appreciated!
>>>>>>
>>>>>> Thanks,
>>>>>> Alex
>>>>>>
>>>>>> --
>>>>>>
>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>> hive> grant all to user hduser;
>>>>>> OK
>>>>>> Time taken: 0.233 seconds
>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>> FAILED: Hive Internal Error:
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>> get_privilege_set failed: unknown result)
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>> unknown result
>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>> failed: unknown result
>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>> ... 14 more
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by yongqiang he <he...@gmail.com>.
I am using local metastore, and can not reproduce the problem.
what message did you get when running local metastore?
On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <gr...@gmail.com> wrote:
> Thanks for opening a ticket.
>
> Table-level grants aren't working for me either (HIVE-2405 suggests
> that the bug is only related to global grants).
>
> hive> set hive.security.authorization.enabled=false;
> hive> CREATE TABLE pokes (foo INT, bar STRING);
> OK
> Time taken: 1.245 seconds
> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
> Copying data from file:/app/hadoop/hive1.in
> Copying file: file:/app/hadoop/hive1.in
> Loading data to table default.pokes
> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
> OK
> Time taken: 0.33 seconds
> hive> select * from pokes;
> OK
> 1 a
> 2 b
> 3 c
> Time taken: 0.095 seconds
> hive> grant select on table pokes to user hduser;
> OK
> Time taken: 0.251 seconds
> hive> set hive.security.authorization.enabled=true;
> hive> select * from pokes;
> FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
> get_privilege_set failed: unknown result)
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.thrift.TApplicationException: get_privilege_set failed:
> unknown result
> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
> ...
>
> mysql> select * from TBL_PRIVS;
> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
> | 1 | 1314219701 | 0 | hduser | USER |
> hduser | USER | Select | 1 |
> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>
> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>
> Authorization failed:No privilege 'Create' found for outputs {
> database:default}. Use show grant to get more details.
>
> Whereas I just get an exception (as you can see above). Were you also
> running with the remote metastore? I get these meaningful messages
> with the local metastore (and authorization on), but with the remote
> metastore with authorization turned on, I always get exceptions.
>
> Many thanks,
> Alex
>
> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
>> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
>> https://issues.apache.org/jira/browse/HIVE-2405
>>
>> thanks for reporting this one!
>>
>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>>> I created the mysql database (with the simple create database command)
>>> and the remote metastore seemed to creat the mysql tables. Here's
>>> some grant information and what I see in the database:
>>>
>>> [hduser@aholmes-desktop conf]$ hive
>>> hive> grant all to user hduser;
>>> OK
>>> Time taken: 0.334 seconds
>>> hive> show grant user hduser;
>>> OK
>>>
>>> principalName hduser
>>> principalType USER
>>> privilege All
>>> grantTime 1314191500
>>> grantor hduser
>>> Time taken: 0.046 seconds
>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>> FAILED: Hive Internal Error:
>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>> get_privilege_set failed: unknown result)
>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>> unknown result
>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>> ...
>>>
>>> mysql> use hive;
>>> Database changed
>>> mysql> select * from GLOBAL_PRIVS;
>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>> | 1 | 1314191500 | 0 | hduser | USER
>>> | hduser | USER | All |
>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>> 1 row in set (0.00 sec)
>>>
>>>
>>> Thanks for your help,
>>> Alex
>>>
>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>>> Have you created the metastore mysql tables for authorization? Can u
>>>> do a show grant?
>>>>
>>>> thanks
>>>> yongqiang
>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>>> Hi all,
>>>>>
>>>>> I've been struggling with getting Hive authorization to work for a few
>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>> configured Hive to enable authorization:
>>>>>
>>>>> <property>
>>>>> <name>hive.security.authorization.enabled</name>
>>>>> <value>true</value>
>>>>> <description>enable or disable the hive client authorization</description>
>>>>> </property>
>>>>>
>>>>> I kept all the other Hive security configs with their default settings.
>>>>>
>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>>> I've also tried creating and loading the table without authorization, granting
>>>>> the select privilege at various levels (global, table, database), turning on
>>>>> auth and performing the select, resulting in the same exception.
>>>>>
>>>>> Any help with this would be greatly appreciated!
>>>>>
>>>>> Thanks,
>>>>> Alex
>>>>>
>>>>> --
>>>>>
>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>> hive> set hive.security.authorization.enabled=false;
>>>>> hive> grant all to user hduser;
>>>>> OK
>>>>> Time taken: 0.233 seconds
>>>>> hive> set hive.security.authorization.enabled=true;
>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>> FAILED: Hive Internal Error:
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>> get_privilege_set failed: unknown result)
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>> unknown result
>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>> failed: unknown result
>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>> ... 14 more
>>>>>
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
Thanks for opening a ticket.
Table-level grants aren't working for me either (HIVE-2405 suggests
that the bug is only related to global grants).
hive> set hive.security.authorization.enabled=false;
hive> CREATE TABLE pokes (foo INT, bar STRING);
OK
Time taken: 1.245 seconds
hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
Copying data from file:/app/hadoop/hive1.in
Copying file: file:/app/hadoop/hive1.in
Loading data to table default.pokes
Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
OK
Time taken: 0.33 seconds
hive> select * from pokes;
OK
1 a
2 b
3 c
Time taken: 0.095 seconds
hive> grant select on table pokes to user hduser;
OK
Time taken: 0.251 seconds
hive> set hive.security.authorization.enabled=true;
hive> select * from pokes;
FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
get_privilege_set failed: unknown result)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: get_privilege_set failed:
unknown result
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
...
mysql> select * from TBL_PRIVS;
+--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
| TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
+--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
| 1 | 1314219701 | 0 | hduser | USER |
hduser | USER | Select | 1 |
+--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
Also, I noticed in HIVE-2405 that you get a meaningful error message:
Authorization failed:No privilege 'Create' found for outputs {
database:default}. Use show grant to get more details.
Whereas I just get an exception (as you can see above). Were you also
running with the remote metastore? I get these meaningful messages
with the local metastore (and authorization on), but with the remote
metastore with authorization turned on, I always get exceptions.
Many thanks,
Alex
On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <he...@gmail.com> wrote:
> This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
> https://issues.apache.org/jira/browse/HIVE-2405
>
> thanks for reporting this one!
>
> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
>> I created the mysql database (with the simple create database command)
>> and the remote metastore seemed to creat the mysql tables. Here's
>> some grant information and what I see in the database:
>>
>> [hduser@aholmes-desktop conf]$ hive
>> hive> grant all to user hduser;
>> OK
>> Time taken: 0.334 seconds
>> hive> show grant user hduser;
>> OK
>>
>> principalName hduser
>> principalType USER
>> privilege All
>> grantTime 1314191500
>> grantor hduser
>> Time taken: 0.046 seconds
>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>> FAILED: Hive Internal Error:
>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>> get_privilege_set failed: unknown result)
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>> unknown result
>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>> ...
>>
>> mysql> use hive;
>> Database changed
>> mysql> select * from GLOBAL_PRIVS;
>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>> | 1 | 1314191500 | 0 | hduser | USER
>> | hduser | USER | All |
>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>> 1 row in set (0.00 sec)
>>
>>
>> Thanks for your help,
>> Alex
>>
>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>>> Have you created the metastore mysql tables for authorization? Can u
>>> do a show grant?
>>>
>>> thanks
>>> yongqiang
>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>>> Hi all,
>>>>
>>>> I've been struggling with getting Hive authorization to work for a few
>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>> configured Hive to enable authorization:
>>>>
>>>> <property>
>>>> <name>hive.security.authorization.enabled</name>
>>>> <value>true</value>
>>>> <description>enable or disable the hive client authorization</description>
>>>> </property>
>>>>
>>>> I kept all the other Hive security configs with their default settings.
>>>>
>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>> superuser). Here are the sequence of steps that are causing me issues.
>>>> Without authorization everything works perfectly (creating, loading, selecting).
>>>> I've also tried creating and loading the table without authorization, granting
>>>> the select privilege at various levels (global, table, database), turning on
>>>> auth and performing the select, resulting in the same exception.
>>>>
>>>> Any help with this would be greatly appreciated!
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>> --
>>>>
>>>> [hduser@aholmes-desktop ~]$ hive
>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>> hive> set hive.security.authorization.enabled=false;
>>>> hive> grant all to user hduser;
>>>> OK
>>>> Time taken: 0.233 seconds
>>>> hive> set hive.security.authorization.enabled=true;
>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>> FAILED: Hive Internal Error:
>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>> get_privilege_set failed: unknown result)
>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>> unknown result
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>> failed: unknown result
>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>> ... 14 more
>>>>
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by yongqiang he <he...@gmail.com>.
This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
https://issues.apache.org/jira/browse/HIVE-2405
thanks for reporting this one!
On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <gr...@gmail.com> wrote:
> I created the mysql database (with the simple create database command)
> and the remote metastore seemed to creat the mysql tables. Here's
> some grant information and what I see in the database:
>
> [hduser@aholmes-desktop conf]$ hive
> hive> grant all to user hduser;
> OK
> Time taken: 0.334 seconds
> hive> show grant user hduser;
> OK
>
> principalName hduser
> principalType USER
> privilege All
> grantTime 1314191500
> grantor hduser
> Time taken: 0.046 seconds
> hive> CREATE TABLE pokes (foo INT, bar STRING);
> FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
> get_privilege_set failed: unknown result)
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.thrift.TApplicationException: get_privilege_set failed:
> unknown result
> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
> ...
>
> mysql> use hive;
> Database changed
> mysql> select * from GLOBAL_PRIVS;
> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
> | 1 | 1314191500 | 0 | hduser | USER
> | hduser | USER | All |
> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
> 1 row in set (0.00 sec)
>
>
> Thanks for your help,
> Alex
>
> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
>> Have you created the metastore mysql tables for authorization? Can u
>> do a show grant?
>>
>> thanks
>> yongqiang
>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>>> Hi all,
>>>
>>> I've been struggling with getting Hive authorization to work for a few
>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>> configured Hive to enable authorization:
>>>
>>> <property>
>>> <name>hive.security.authorization.enabled</name>
>>> <value>true</value>
>>> <description>enable or disable the hive client authorization</description>
>>> </property>
>>>
>>> I kept all the other Hive security configs with their default settings.
>>>
>>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>> superuser). Here are the sequence of steps that are causing me issues.
>>> Without authorization everything works perfectly (creating, loading, selecting).
>>> I've also tried creating and loading the table without authorization, granting
>>> the select privilege at various levels (global, table, database), turning on
>>> auth and performing the select, resulting in the same exception.
>>>
>>> Any help with this would be greatly appreciated!
>>>
>>> Thanks,
>>> Alex
>>>
>>> --
>>>
>>> [hduser@aholmes-desktop ~]$ hive
>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>> hive> set hive.security.authorization.enabled=false;
>>> hive> grant all to user hduser;
>>> OK
>>> Time taken: 0.233 seconds
>>> hive> set hive.security.authorization.enabled=true;
>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>> FAILED: Hive Internal Error:
>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>> get_privilege_set failed: unknown result)
>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>> unknown result
>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>> failed: unknown result
>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>> ... 14 more
>>>
>>
>
Re: Hive 0.7.1 authorization woes
Posted by Alex Holmes <gr...@gmail.com>.
I created the mysql database (with the simple create database command)
and the remote metastore seemed to creat the mysql tables. Here's
some grant information and what I see in the database:
[hduser@aholmes-desktop conf]$ hive
hive> grant all to user hduser;
OK
Time taken: 0.334 seconds
hive> show grant user hduser;
OK
principalName hduser
principalType USER
privilege All
grantTime 1314191500
grantor hduser
Time taken: 0.046 seconds
hive> CREATE TABLE pokes (foo INT, bar STRING);
FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
get_privilege_set failed: unknown result)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: get_privilege_set failed:
unknown result
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
...
mysql> use hive;
Database changed
mysql> select * from GLOBAL_PRIVS;
+---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
| USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
| PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
+---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
| 1 | 1314191500 | 0 | hduser | USER
| hduser | USER | All |
+---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
1 row in set (0.00 sec)
Thanks for your help,
Alex
On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <he...@gmail.com> wrote:
> Have you created the metastore mysql tables for authorization? Can u
> do a show grant?
>
> thanks
> yongqiang
> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
>> Hi all,
>>
>> I've been struggling with getting Hive authorization to work for a few
>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>> configured Hive to enable authorization:
>>
>> <property>
>> <name>hive.security.authorization.enabled</name>
>> <value>true</value>
>> <description>enable or disable the hive client authorization</description>
>> </property>
>>
>> I kept all the other Hive security configs with their default settings.
>>
>> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
>> metastore and the Hive CLI are all running as the same user (the HDFS
>> superuser). Here are the sequence of steps that are causing me issues.
>> Without authorization everything works perfectly (creating, loading, selecting).
>> I've also tried creating and loading the table without authorization, granting
>> the select privilege at various levels (global, table, database), turning on
>> auth and performing the select, resulting in the same exception.
>>
>> Any help with this would be greatly appreciated!
>>
>> Thanks,
>> Alex
>>
>> --
>>
>> [hduser@aholmes-desktop ~]$ hive
>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>> hive> set hive.security.authorization.enabled=false;
>> hive> grant all to user hduser;
>> OK
>> Time taken: 0.233 seconds
>> hive> set hive.security.authorization.enabled=true;
>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>> FAILED: Hive Internal Error:
>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>> get_privilege_set failed: unknown result)
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>> unknown result
>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>> failed: unknown result
>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>> ... 14 more
>>
>
Re: Hive 0.7.1 authorization woes
Posted by yongqiang he <he...@gmail.com>.
Have you created the metastore mysql tables for authorization? Can u
do a show grant?
thanks
yongqiang
On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <gr...@gmail.com> wrote:
> Hi all,
>
> I've been struggling with getting Hive authorization to work for a few
> hours, and I really hope someone can help me. I installed Hive 0.7.1
> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
> configured Hive to enable authorization:
>
> <property>
> <name>hive.security.authorization.enabled</name>
> <value>true</value>
> <description>enable or disable the hive client authorization</description>
> </property>
>
> I kept all the other Hive security configs with their default settings.
>
> I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
> metastore and the Hive CLI are all running as the same user (the HDFS
> superuser). Here are the sequence of steps that are causing me issues.
> Without authorization everything works perfectly (creating, loading, selecting).
> I've also tried creating and loading the table without authorization, granting
> the select privilege at various levels (global, table, database), turning on
> auth and performing the select, resulting in the same exception.
>
> Any help with this would be greatly appreciated!
>
> Thanks,
> Alex
>
> --
>
> [hduser@aholmes-desktop ~]$ hive
> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
> hive> set hive.security.authorization.enabled=false;
> hive> grant all to user hduser;
> OK
> Time taken: 0.233 seconds
> hive> set hive.security.authorization.enabled=true;
> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
> FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
> get_privilege_set failed: unknown result)
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.thrift.TApplicationException: get_privilege_set failed:
> unknown result
> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
> at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
> failed: unknown result
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
> at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
> ... 14 more
>