You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "LiPenglin (Jira)" <ji...@apache.org> on 2022/06/02 16:33:00 UTC
[jira] [Commented] (IMPALA-11330) Handle missing Iceberg data/metadata gracefully
[ https://issues.apache.org/jira/browse/IMPALA-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545549#comment-17545549 ]
LiPenglin commented on IMPALA-11330:
------------------------------------
I confirm that when I delete the metadata directory directly, the drop table will throw an exception and data cannot be deleted by impala unless manually deleted.
{code:java}
Query: drop table ice_tbl
ERROR: NotFoundException: Failed to open input stream for file: hdfs://localhost:20500/test-warehouse/tdb.db/ice_tbl/metadata/00001-15f86494-2607-4e5a-8178-7ea33821f04c.metadata.json{code}
> Handle missing Iceberg data/metadata gracefully
> -----------------------------------------------
>
> Key: IMPALA-11330
> URL: https://issues.apache.org/jira/browse/IMPALA-11330
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Tamas Mate
> Priority: Major
> Labels: impala-iceberg
>
> In case the data/metadata directory is not available for an Iceberg table queries are failing with NotFoundException, see bellow. This affects DROP TABLE as well, which means that an Iceberg table can get stuck in the system if the administrator moves the data.
> {code:none}
> ERROR: NotFoundException: Failed to open input stream for file: hdfs://localhost:20500/test-warehouse/test2/metadata/00001-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> CAUSED BY: FileNotFoundException: File does not exist: /test-warehouse/test2/metadata/00001-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:87)
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:77)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:159)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2035)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:737)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:454)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> CAUSED BY: RemoteException: File does not exist: /test-warehouse/test2/metadata/00001-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:87)
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:77)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:159)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2035)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:737)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:454)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org