You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Vihang Karajgaonkar (Jira)" <ji...@apache.org> on 2020/12/23 18:32:00 UTC
[jira] [Resolved] (IMPALA-2211) Impala reports missing WRITE access
on dir in creating an external table
[ https://issues.apache.org/jira/browse/IMPALA-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vihang Karajgaonkar resolved IMPALA-2211.
-----------------------------------------
Resolution: Won't Fix
Closing this as won't fix since the error is coming from HMS which tries to create a directory. Also, my understanding is that in case of external table HMS will only try to create the directory if it doesn't exist already. So this may be some setup issue as well.
> Impala reports missing WRITE access on dir in creating an external table
> ------------------------------------------------------------------------
>
> Key: IMPALA-2211
> URL: https://issues.apache.org/jira/browse/IMPALA-2211
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.2
> Reporter: Vamsee K. Yarlagadda
> Priority: Minor
> Labels: ramp-up
>
> It looks like Impala complains about missing write permissions on a HDFS directory when trying to create an external table through Impyla.
> {code}
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py", line 152, in execute
> configuration=configuration)
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py", line 166, in execute_async
> self._execute_async(op)
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py", line 172, in _execute_async
> operation_fn()
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py", line 164, in op
> configuration)
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py", line 134, in wrapper
> return func(*args, **kwargs)
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py", line 240, in execute_statement
> err_if_rpc_not_ok(resp)
> File "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py", line 81, in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> HiveServer2Error:
> ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore:
> CAUSED BY: MetaException: Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=impala, access=WRITE, inode="/user/systest/tpcds_10_text":systest:supergroup:drwxr-xr-x
> at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:257)
> at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:238)
> at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:216)
> at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:145)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:138)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6599)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6581)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6533)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4337)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4307)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4280)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:853)
> at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:321)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:601)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)
> {code}
> My main worry is if i am creating an external table (pointing to an HDFS location), why does impala need WRITE access during table creations step. Ideally it should just have READ access right?
> DDL statement being invoked
> {code}
> create external table call_center(
> cc_call_center_sk int
> , cc_call_center_id string
> , cc_rec_start_date string
> , cc_rec_end_date string
> , cc_closed_date_sk int
> , cc_open_date_sk int
> , cc_name string
> , cc_class string
> , cc_employees int
> , cc_sq_ft int
> , cc_hours string
> , cc_manager string
> , cc_mkt_id int
> , cc_mkt_class string
> , cc_mkt_desc string
> , cc_market_manager string
> , cc_division int
> , cc_division_name string
> , cc_company int
> , cc_company_name string
> , cc_street_number string
> , cc_street_name string
> , cc_street_type string
> , cc_suite_number string
> , cc_city string
> , cc_county string
> , cc_state string
> , cc_zip string
> , cc_country string
> , cc_gmt_offset decimal(5,2)
> , cc_tax_percentage decimal(5,2)
> )
> row format delimited fields terminated by '|'
> location '/user/systest/tpcds_10_text/call_center';
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)