You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sanjay Subramanian <Sa...@wizecommerce.com> on 2013/03/26 03:01:38 UTC
HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Ben Kim <be...@gmail.com>.
Hi
apoloze for bringing up old post(email) but does *hive.metastore.execute.setugi
*property work properly?
I'm using Hive 0.11 with Hadoop 2.0.0
after setting hive.metastore.execute.setugi to true and logging into
*ben*system account,
if i create a table with "*create table pokes(foo int, bar string)*" it
creates /hive/warehouse/pokes with owner set to hive
and if I "*drop table pokes*", I get "OK" message but from the server side
i get following error
2013-06-07 20:32:27,594 ERROR metastore.HiveMetaStore
> (HiveMetaStore.java:deleteTableData(1220)) - Failed to delete table
> directory: hdfs://server01/hive/warehouse/ben Got exception:
> org.apache.hadoop.security.AccessControlException Permission denied: user=
> *hive*, access=ALL, inode="/hive/warehouse/pokes":ben:hadoop:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkSubAccess(FSPermissionChecker.java:191)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4716)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:2816)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInt(FSNamesystem.java:2777)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2764)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:621)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:408)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44968)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
>
so even with the configuration, hive is still trying to create and remove
table with hive user instead of ben.
If anyone had similar issue please share with us.
Thankyou alot!
*
*
**
*
*
On Wed, Mar 27, 2013 at 7:46 AM, Sanjay Subramanian <
Sanjay.Subramanian@wizecommerce.com> wrote:
> Ok I solved this
> The default setting *hive.metastore.execute.setugi* in Hive is FALSE
> Adding this to the hive-site.xml solved it
>
> <property>
> <name>hive.metastore.execute.setugi</name>
> <value>true</value>
> <description>In unsecure mode, setting this property to true will cause
> the metastore to execute DFS operations using the client's reported user
> and group permissions. Note that this property must be set on both the
> client and server sides. Further note that its best effort. If client sets
> its to true and server sets it to false, client setting will be
> ignored.</description>
> </property>
>
>
> From: Sanjay Subramanian <sa...@wizecommerce.com>
> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> Date: Monday, March 25, 2013 7:01 PM
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Subject: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
>
> Steps to recreate the use case:
>
> - Log in as *sasubramanian* to Linux Box
> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
> - Go to HDFS /user/hive/warehouse/
>
> Name Type Size Replication Block Size Modification Time
> Permission Owner Group
> name dir 2013-03-25 18:57 rwxr-xr-x *
> hive* supergroup
>
> I want the table top be created as Owner = sasubramanian
> How can I do that ?
>
> Thanks
> sanjay
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
--
*Benjamin Kim*
*benkimkimben at gmail*
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
I am using HiveService interface create partitions…I will send you the errors from the logger tomorrow
Thanks
sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 8:08 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
if you can share the error when you add partitions after enabling the property that will give more hints
On Tue, Mar 26, 2013 at 8:38 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Can you enable this property ?
also can you run "show grant user userid" ?
there was a bug for this https://issues.apache.org/jira/browse/HIVE-2405 fixed in 0.8.0 so hopefully you will not hit this.
if enable this property and grant permissions to create table to the user, it means user can add partitions.
from language manual
* CREATE - Allows users to create objects. For a database, this means users can create tables, and for a table, this means users can create partitions .. you can refer the entire table at https://cwiki.apache.org/Hive/languagemanual-auth.html
On Tue, Mar 26, 2013 at 8:24 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
I am using
Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
My metastore is MySQL
My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions if I keep that as true
<property>
<name>hive.security.authorization.enabled</name>
<value>false</value>
<description>enable or disable the hive client authorization</description>
</property>
I have not defined a property=hive.security.authorization.manager
Thanks
sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:43 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
YARN should not play any role in any create table statement. It just creates a directory with DFSClient. Normally it tries to create it with the cli userid in my experience.
which version of hive are you using?
which is your metastore?
Can you check for the following values?
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
<description>the hive client authorization manager class name.
The user defined authorization class should implement interface
org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
</description>
</property>
On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run several hive tables and scripts ; The same test Create Table script gives the correct owner name
I added location but that did not help
hive -e "CREATE TABLE name (id INT, name STRING) LOCATION '/user/hive/warehouse/name';"
Thanks
Sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:13 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Forgot to add,
if you want full filesystem level security on HDFS then you will need to enable kerberos based security.
On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Sanjay,
can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To create it by the individual users you need to provide by the location clause.
On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Hive CLI works fine for "ALTER TABLE" but get HiveServerException
using ThriftHive.Client
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hive-site.xml setting
---------------------
hive.security.authorization.enabled = true
Script
------
ALTER TABLE myTable ADD PARTITION (partition1='some_value1' , partition2='some_value2') LOCATION '/path/to/directory/on/hdfs/containing/data'
I can execute this script using Hive CLI but ThriftHive.Client gives this error
HiveServerException(message:Query returned non-zero code: 403, cause: null, errorCode:403, SQLState:null) at
org.apache.hadoop.hive.service.ThriftHive$execute_result.read(ThriftHive.java:1333) at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at
org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:106) at
org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:93) at
com.wizecommerce.utils.hive.service.HiveServiceBAction.runActionStr(HiveServiceBAction.java:46) at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.addPartitions(OutpdirHeaderAddPartition.java:105) at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.main(OutpdirHeaderAddPartition.java:69) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152) at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hi
I added the following to hive-site.xml
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
I did not add hive.security.authorization.manager because I am currently using Cloudera Manager 4.1.2 to install all components and looks like they don't have a Hcatalog distribution that I need for the hive.security.authorization.manager param to work correctly (after pointing the HIVE_AUX_JARS to this Hcatalog jar location)
:-) I can do a tarball install on my sandbox if needed later but at this moment I am trying to stabilize the steps for Cloudera CDH4.1.2 install so that I can tell the OPS guys to follow the steps…they will find it difficult to manually install on N node clusters in production and I think that also will be highly error prone
Anyway the add partitions works from the Hive CLI and Beeswax BUT fails when I use the HiveService interface (This is a oozie workflow created in Hue and submitted to Hadoop Yarn Cluster)
HiveServerException(message:Query returned non-zero code: 403, cause: null, errorCode:403, SQLState:null) at
org.apache.hadoop.hive.service.ThriftHive$execute_result.read(ThriftHive.java:1333) at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at
org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:106) at
org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:93) at
com.wizecommerce.utils.hive.service.HiveServiceBAction.runActionStr(HiveServiceBAction.java:46) at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.addPartitions(OutpdirHeaderAddPartition.java:105) at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.main(OutpdirHeaderAddPartition.java:69) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152) at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Thanks
sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 8:08 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
if you can share the error when you add partitions after enabling the property that will give more hints
On Tue, Mar 26, 2013 at 8:38 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Can you enable this property ?
also can you run "show grant user userid" ?
there was a bug for this https://issues.apache.org/jira/browse/HIVE-2405 fixed in 0.8.0 so hopefully you will not hit this.
if enable this property and grant permissions to create table to the user, it means user can add partitions.
from language manual
* CREATE - Allows users to create objects. For a database, this means users can create tables, and for a table, this means users can create partitions .. you can refer the entire table at https://cwiki.apache.org/Hive/languagemanual-auth.html
On Tue, Mar 26, 2013 at 8:24 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
I am using
Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
My metastore is MySQL
My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions if I keep that as true
<property>
<name>hive.security.authorization.enabled</name>
<value>false</value>
<description>enable or disable the hive client authorization</description>
</property>
I have not defined a property=hive.security.authorization.manager
Thanks
sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:43 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
YARN should not play any role in any create table statement. It just creates a directory with DFSClient. Normally it tries to create it with the cli userid in my experience.
which version of hive are you using?
which is your metastore?
Can you check for the following values?
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
<description>the hive client authorization manager class name.
The user defined authorization class should implement interface
org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
</description>
</property>
On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run several hive tables and scripts ; The same test Create Table script gives the correct owner name
I added location but that did not help
hive -e "CREATE TABLE name (id INT, name STRING) LOCATION '/user/hive/warehouse/name';"
Thanks
Sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:13 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Forgot to add,
if you want full filesystem level security on HDFS then you will need to enable kerberos based security.
On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Sanjay,
can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To create it by the individual users you need to provide by the location clause.
On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Nitin Pawar <ni...@gmail.com>.
if you can share the error when you add partitions after enabling the
property that will give more hints
On Tue, Mar 26, 2013 at 8:38 AM, Nitin Pawar <ni...@gmail.com>wrote:
> Can you enable this property ?
> also can you run "show grant user userid" ?
>
> there was a bug for this https://issues.apache.org/jira/browse/HIVE-2405fixed in 0.8.0 so hopefully you will not hit this.
>
> if enable this property and grant permissions to create table to the user,
> it means user can add partitions.
> from language manual
> CREATE - Allows users to create objects. For a database, this means users
> can create tables, and for a table, this means users can create partitions
> .. you can refer the entire table at
> https://cwiki.apache.org/Hive/languagemanual-auth.html
>
>
>
> On Tue, Mar 26, 2013 at 8:24 AM, Sanjay Subramanian <
> Sanjay.Subramanian@wizecommerce.com> wrote:
>
>> I am using
>> Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
>> My metastore is MySQL
>>
>> My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions if I keep that as true
>>
>> <property>
>> <name>hive.security.authorization.enabled</name>
>> <value>*false*</value>
>> <description>enable or disable the hive client authorization</description>
>> </property>
>>
>> I have not defined a property=hive.security.authorization.manager
>>
>> Thanks
>> sanjay
>>
>> From: Nitin Pawar <ni...@gmail.com>
>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>> Date: Monday, March 25, 2013 7:43 PM
>>
>> To: "user@hive.apache.org" <us...@hive.apache.org>
>> Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as
>> Owner ?
>>
>> YARN should not play any role in any create table statement. It just
>> creates a directory with DFSClient. Normally it tries to create it with the
>> cli userid in my experience.
>>
>> which version of hive are you using?
>> which is your metastore?
>>
>> Can you check for the following values?
>>
>> <property>
>> <name>hive.security.authorization.enabled</name>
>> <value>true</value>
>> <description>enable or disable the hive client authorization</description>
>> </property>
>>
>> <property>
>> <name>hive.security.authorization.manager</name>
>> <value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
>> <description>the hive client authorization manager class name.
>> The user defined authorization class should implement interface
>> org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
>> </description>
>> </property>
>>
>>
>>
>> On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <
>> Sanjay.Subramanian@wizecommerce.com> wrote:
>>
>>> Hi Nitin
>>> I notice this peculiarity in Yarn and Hive
>>> I have another earlier cluster with MRv1 where I have created and run
>>> several hive tables and scripts ; The same test Create Table script gives
>>> the correct owner name
>>> I added location but that did not help
>>> hive -e "CREATE TABLE name (id INT, name STRING) LOCATION
>>> '/user/hive/warehouse/name';"
>>>
>>> Thanks
>>> Sanjay
>>>
>>> From: Nitin Pawar <ni...@gmail.com>
>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>> Date: Monday, March 25, 2013 7:13 PM
>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>> Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as
>>> Owner ?
>>>
>>> Forgot to add,
>>> if you want full filesystem level security on HDFS then you will need to
>>> enable kerberos based security.
>>>
>>>
>>> On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>wrote:
>>>
>>>> Sanjay,
>>>>
>>>> can you try adding 'LOCATION' clause to your create statement.
>>>> By default the hive warehouse directory is writable by all the user. To
>>>> create it by the individual users you need to provide by the location
>>>> clause.
>>>>
>>>>
>>>> On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <
>>>> Sanjay.Subramanian@wizecommerce.com> wrote:
>>>>
>>>>> Steps to recreate the use case:
>>>>>
>>>>> - Log in as *sasubramanian* to Linux Box
>>>>> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
>>>>> - Go to HDFS /user/hive/warehouse/
>>>>>
>>>>> Name Type Size Replication Block Size Modification Time
>>>>> Permission Owner Group
>>>>> name dir 2013-03-25 18:57 rwxr-xr-x
>>>>> *hive* supergroup
>>>>>
>>>>> I want the table top be created as Owner = sasubramanian
>>>>> How can I do that ?
>>>>>
>>>>> Thanks
>>>>> sanjay
>>>>>
>>>>> CONFIDENTIALITY NOTICE
>>>>> ======================
>>>>> This email message and any attachments are for the exclusive use of
>>>>> the intended recipient(s) and may contain confidential and privileged
>>>>> information. Any unauthorized review, use, disclosure or distribution is
>>>>> prohibited. If you are not the intended recipient, please contact the
>>>>> sender by reply email and destroy all copies of the original message along
>>>>> with any attachments, from your computer system. If you are the intended
>>>>> recipient, please be advised that the content of this message is subject to
>>>>> access, review and disclosure by the sender's Email System Administrator.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>> CONFIDENTIALITY NOTICE
>>> ======================
>>> This email message and any attachments are for the exclusive use of the
>>> intended recipient(s) and may contain confidential and privileged
>>> information. Any unauthorized review, use, disclosure or distribution is
>>> prohibited. If you are not the intended recipient, please contact the
>>> sender by reply email and destroy all copies of the original message along
>>> with any attachments, from your computer system. If you are the intended
>>> recipient, please be advised that the content of this message is subject to
>>> access, review and disclosure by the sender's Email System Administrator.
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>> CONFIDENTIALITY NOTICE
>> ======================
>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>
>
>
> --
> Nitin Pawar
>
--
Nitin Pawar
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Nitin Pawar <ni...@gmail.com>.
Can you enable this property ?
also can you run "show grant user userid" ?
there was a bug for this
https://issues.apache.org/jira/browse/HIVE-2405fixed in 0.8.0 so
hopefully you will not hit this.
if enable this property and grant permissions to create table to the user,
it means user can add partitions.
from language manual
CREATE - Allows users to create objects. For a database, this means users
can create tables, and for a table, this means users can create partitions
.. you can refer the entire table at
https://cwiki.apache.org/Hive/languagemanual-auth.html
On Tue, Mar 26, 2013 at 8:24 AM, Sanjay Subramanian <
Sanjay.Subramanian@wizecommerce.com> wrote:
> I am using
> Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
> My metastore is MySQL
>
> My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions if I keep that as true
>
> <property>
> <name>hive.security.authorization.enabled</name>
> <value>*false*</value>
> <description>enable or disable the hive client authorization</description>
> </property>
>
> I have not defined a property=hive.security.authorization.manager
>
> Thanks
> sanjay
>
> From: Nitin Pawar <ni...@gmail.com>
> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> Date: Monday, March 25, 2013 7:43 PM
>
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as
> Owner ?
>
> YARN should not play any role in any create table statement. It just
> creates a directory with DFSClient. Normally it tries to create it with the
> cli userid in my experience.
>
> which version of hive are you using?
> which is your metastore?
>
> Can you check for the following values?
>
> <property>
> <name>hive.security.authorization.enabled</name>
> <value>true</value>
> <description>enable or disable the hive client authorization</description>
> </property>
>
> <property>
> <name>hive.security.authorization.manager</name>
> <value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
> <description>the hive client authorization manager class name.
> The user defined authorization class should implement interface
> org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
> </description>
> </property>
>
>
>
> On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <
> Sanjay.Subramanian@wizecommerce.com> wrote:
>
>> Hi Nitin
>> I notice this peculiarity in Yarn and Hive
>> I have another earlier cluster with MRv1 where I have created and run
>> several hive tables and scripts ; The same test Create Table script gives
>> the correct owner name
>> I added location but that did not help
>> hive -e "CREATE TABLE name (id INT, name STRING) LOCATION
>> '/user/hive/warehouse/name';"
>>
>> Thanks
>> Sanjay
>>
>> From: Nitin Pawar <ni...@gmail.com>
>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>> Date: Monday, March 25, 2013 7:13 PM
>> To: "user@hive.apache.org" <us...@hive.apache.org>
>> Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as
>> Owner ?
>>
>> Forgot to add,
>> if you want full filesystem level security on HDFS then you will need to
>> enable kerberos based security.
>>
>>
>> On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> Sanjay,
>>>
>>> can you try adding 'LOCATION' clause to your create statement.
>>> By default the hive warehouse directory is writable by all the user. To
>>> create it by the individual users you need to provide by the location
>>> clause.
>>>
>>>
>>> On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <
>>> Sanjay.Subramanian@wizecommerce.com> wrote:
>>>
>>>> Steps to recreate the use case:
>>>>
>>>> - Log in as *sasubramanian* to Linux Box
>>>> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
>>>> - Go to HDFS /user/hive/warehouse/
>>>>
>>>> Name Type Size Replication Block Size Modification Time
>>>> Permission Owner Group
>>>> name dir 2013-03-25 18:57 rwxr-xr-x
>>>> *hive* supergroup
>>>>
>>>> I want the table top be created as Owner = sasubramanian
>>>> How can I do that ?
>>>>
>>>> Thanks
>>>> sanjay
>>>>
>>>> CONFIDENTIALITY NOTICE
>>>> ======================
>>>> This email message and any attachments are for the exclusive use of the
>>>> intended recipient(s) and may contain confidential and privileged
>>>> information. Any unauthorized review, use, disclosure or distribution is
>>>> prohibited. If you are not the intended recipient, please contact the
>>>> sender by reply email and destroy all copies of the original message along
>>>> with any attachments, from your computer system. If you are the intended
>>>> recipient, please be advised that the content of this message is subject to
>>>> access, review and disclosure by the sender's Email System Administrator.
>>>>
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>> CONFIDENTIALITY NOTICE
>> ======================
>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>
>
>
> --
> Nitin Pawar
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
--
Nitin Pawar
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
I am using
Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
My metastore is MySQL
My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions if I keep that as true
<property>
<name>hive.security.authorization.enabled</name>
<value>false</value>
<description>enable or disable the hive client authorization</description>
</property>
I have not defined a property=hive.security.authorization.manager
Thanks
sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:43 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
YARN should not play any role in any create table statement. It just creates a directory with DFSClient. Normally it tries to create it with the cli userid in my experience.
which version of hive are you using?
which is your metastore?
Can you check for the following values?
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
<description>the hive client authorization manager class name.
The user defined authorization class should implement interface
org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
</description>
</property>
On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run several hive tables and scripts ; The same test Create Table script gives the correct owner name
I added location but that did not help
hive -e "CREATE TABLE name (id INT, name STRING) LOCATION '/user/hive/warehouse/name';"
Thanks
Sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:13 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Forgot to add,
if you want full filesystem level security on HDFS then you will need to enable kerberos based security.
On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Sanjay,
can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To create it by the individual users you need to provide by the location clause.
On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Nitin Pawar <ni...@gmail.com>.
YARN should not play any role in any create table statement. It just
creates a directory with DFSClient. Normally it tries to create it with the
cli userid in my experience.
which version of hive are you using?
which is your metastore?
Can you check for the following values?
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
<description>the hive client authorization manager class name.
The user defined authorization class should implement interface
org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
</description>
</property>
On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <
Sanjay.Subramanian@wizecommerce.com> wrote:
> Hi Nitin
> I notice this peculiarity in Yarn and Hive
> I have another earlier cluster with MRv1 where I have created and run
> several hive tables and scripts ; The same test Create Table script gives
> the correct owner name
> I added location but that did not help
> hive -e "CREATE TABLE name (id INT, name STRING) LOCATION
> '/user/hive/warehouse/name';"
>
> Thanks
> Sanjay
>
> From: Nitin Pawar <ni...@gmail.com>
> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> Date: Monday, March 25, 2013 7:13 PM
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as
> Owner ?
>
> Forgot to add,
> if you want full filesystem level security on HDFS then you will need to
> enable kerberos based security.
>
>
> On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> Sanjay,
>>
>> can you try adding 'LOCATION' clause to your create statement.
>> By default the hive warehouse directory is writable by all the user. To
>> create it by the individual users you need to provide by the location
>> clause.
>>
>>
>> On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <
>> Sanjay.Subramanian@wizecommerce.com> wrote:
>>
>>> Steps to recreate the use case:
>>>
>>> - Log in as *sasubramanian* to Linux Box
>>> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
>>> - Go to HDFS /user/hive/warehouse/
>>>
>>> Name Type Size Replication Block Size Modification Time
>>> Permission Owner Group
>>> name dir 2013-03-25 18:57 rwxr-xr-x
>>> *hive* supergroup
>>>
>>> I want the table top be created as Owner = sasubramanian
>>> How can I do that ?
>>>
>>> Thanks
>>> sanjay
>>>
>>> CONFIDENTIALITY NOTICE
>>> ======================
>>> This email message and any attachments are for the exclusive use of the
>>> intended recipient(s) and may contain confidential and privileged
>>> information. Any unauthorized review, use, disclosure or distribution is
>>> prohibited. If you are not the intended recipient, please contact the
>>> sender by reply email and destroy all copies of the original message along
>>> with any attachments, from your computer system. If you are the intended
>>> recipient, please be advised that the content of this message is subject to
>>> access, review and disclosure by the sender's Email System Administrator.
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
>
> --
> Nitin Pawar
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
--
Nitin Pawar
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run several hive tables and scripts ; The same test Create Table script gives the correct owner name
I added location but that did not help
hive -e "CREATE TABLE name (id INT, name STRING) LOCATION '/user/hive/warehouse/name';"
Thanks
Sanjay
From: Nitin Pawar <ni...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:13 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Forgot to add,
if you want full filesystem level security on HDFS then you will need to enable kerberos based security.
On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>> wrote:
Sanjay,
can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To create it by the individual users you need to provide by the location clause.
On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <Sa...@wizecommerce.com>> wrote:
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar
--
Nitin Pawar
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Nitin Pawar <ni...@gmail.com>.
Forgot to add,
if you want full filesystem level security on HDFS then you will need to
enable kerberos based security.
On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <ni...@gmail.com>wrote:
> Sanjay,
>
> can you try adding 'LOCATION' clause to your create statement.
> By default the hive warehouse directory is writable by all the user. To
> create it by the individual users you need to provide by the location
> clause.
>
>
> On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <
> Sanjay.Subramanian@wizecommerce.com> wrote:
>
>> Steps to recreate the use case:
>>
>> - Log in as *sasubramanian* to Linux Box
>> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
>> - Go to HDFS /user/hive/warehouse/
>>
>> Name Type Size Replication Block Size Modification Time
>> Permission Owner Group
>> name dir 2013-03-25 18:57 rwxr-xr-x
>> *hive* supergroup
>>
>> I want the table top be created as Owner = sasubramanian
>> How can I do that ?
>>
>> Thanks
>> sanjay
>>
>> CONFIDENTIALITY NOTICE
>> ======================
>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>
>
>
> --
> Nitin Pawar
>
--
Nitin Pawar
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Posted by Nitin Pawar <ni...@gmail.com>.
Sanjay,
can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To
create it by the individual users you need to provide by the location
clause.
On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <
Sanjay.Subramanian@wizecommerce.com> wrote:
> Steps to recreate the use case:
>
> - Log in as *sasubramanian* to Linux Box
> - Execute hive -e "CREATE TABLE name (id INT, name STRING);"
> - Go to HDFS /user/hive/warehouse/
>
> Name Type Size Replication Block Size Modification Time
> Permission Owner Group
> name dir 2013-03-25 18:57 rwxr-xr-x *
> hive* supergroup
>
> I want the table top be created as Owner = sasubramanian
> How can I do that ?
>
> Thanks
> sanjay
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
--
Nitin Pawar
Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner
?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Ok I solved this
The default setting hive.metastore.execute.setugi in Hive is FALSE
Adding this to the hive-site.xml solved it
<property>
<name>hive.metastore.execute.setugi</name>
<value>true</value>
<description>In unsecure mode, setting this property to true will cause the metastore to execute DFS operations using the client's reported user and group permissions. Note that this property must be set on both the client and server sides. Further note that its best effort. If client sets its to true and server sets it to false, client setting will be ignored.</description>
</property>
From: Sanjay Subramanian <sa...@wizecommerce.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, March 25, 2013 7:01 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Steps to recreate the use case:
- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT, name STRING);"
- Go to HDFS /user/hive/warehouse/
Name Type Size Replication Block Size Modification Time Permission Owner Group
name dir 2013-03-25 18:57 rwxr-xr-x hive supergroup
I want the table top be created as Owner = sasubramanian
How can I do that ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.