You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Alexandre Linte (JIRA)" <ji...@apache.org> on 2016/11/04 10:28:59 UTC
[jira] [Created] (HIVE-15128) "hive.exec.stagingdir" produces an
"AccessControlException" when working with temp tables
Alexandre Linte created HIVE-15128:
--------------------------------------
Summary: "hive.exec.stagingdir" produces an "AccessControlException" when working with temp tables
Key: HIVE-15128
URL: https://issues.apache.org/jira/browse/HIVE-15128
Project: Hive
Issue Type: Bug
Components: Beeline, CLI, Hive
Affects Versions: 2.1.0
Environment: The property "hive.exec.stagingdir" is producing an AccessControlException error, only when working with a temporary table and with an INSERT INTO TABLE statement.
The configuration used in the hive-site.xml is:
{noformat}
<property>
<name>hive.exec.stagingdir</name>
<value>/user/${user.name}/.hiveStaging/.staging</value>
<description>In Hive >= 0.14, set to ${hive.exec.scratchdir}/${user.name}/.staging</description>
</property>
{noformat}
Trying to insert a value into a temporary table will produce the following output:
{noformat}
hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 16.09 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
Failed with exception org.apache.hadoop.security.AccessControlException: User does not belong to hadoop
at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. org.apache.hadoop.security.AccessControlException: User does not belong to hadoop
at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
{noformat}
But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No AccessControlException error here and erverything is properly inserted.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 6.74 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
chgrp: changing ownership of 'hdfs://sandbox/tmp/hive/shfs3453/edab56b7-b1e1-4f87-bdfd-eeece41f6fc9/_tmp_space.db/874bfdbd-7bab-4e0c-85ac-ec4552ea560b': User does not belong to hadoop
OK
Time taken: 8.224 seconds
{noformat}
If the previous query is done one more time, the warning will not be displayed.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180831_40350dc3-1f66-45e2-8304-90b00933aead
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 13.00 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
OK
Time taken: 14.161 seconds
{noformat}
Note: if the warning is displayed one time, an INSERT INTO TABLE will work.
Reporter: Alexandre Linte
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)