You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zoltán Rátkai (Jira)" <ji...@apache.org> on 2022/11/23 14:02:00 UTC
[jira] [Resolved] (HIVE-26679) [Hive] Drops archive partitions error
[ https://issues.apache.org/jira/browse/HIVE-26679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zoltán Rátkai resolved HIVE-26679.
----------------------------------
Resolution: Fixed
The issue is in TEZ:
https://issues.apache.org/jira/browse/TEZ-4415?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=17564569
> [Hive] Drops archive partitions error
> -------------------------------------
>
> Key: HIVE-26679
> URL: https://issues.apache.org/jira/browse/HIVE-26679
> Project: Hive
> Issue Type: Bug
> Reporter: Wang Jiangkun
> Assignee: Zoltán Rátkai
> Priority: Blocker
> Attachments: image-2022-11-10-16-58-45-250.png
>
>
> When the partition table is converted to an archive partition, the drop partition times are Error. it seems there is an issue occurs, which is
> {code:java}
> set hive.archive.enabled=true;
> alter table tb1 archive partition(city="nanjing");{code}
> {code:java}
> 2022-10-27 16:55:41,872 ERROR hive.ql.exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Got exception: java.io.IOException Invalid path for the Har Filesystem. No index file in har://hdfs-nameservice/hive/tb1/city=nanjing/data.har
> at org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2364)
> at org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2345)
> at org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:3900)
> at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3860)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:368)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2130)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1801)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1206)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1194)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: MetaException(message:Got exception: java.io.IOException Invalid path for the Har Filesystem. No index file in har://hdfs-nameservice/hive/tb1/city=nanjing/data.har)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result.read(ThriftHiveMetastore.java)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_partitions_req(ThriftHiveMetastore.java:2081)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_partitions_req(ThriftHiveMetastore.java:2068)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropPartitions(HiveMetaStoreClient.java:1008)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:173)
> at com.sun.proxy.$Proxy29.dropPartitions(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2332)
> at com.sun.proxy.$Proxy29.dropPartitions(Unknown Source)
> at org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2358)
> ... 23 more 2022-10-09T10:36:32,271 INFO [573c4ce0-f73c-439b-829d{code}
> After the archiving operation is performed, the corresponding partition metadata information has changed
> {code:java}
> Partition Value: [nanjing]
> Database: default
> Table: tb1
> CreateTime: Mon Oct 31 08:53:47 CST 2022
> LastAccessTime: UNKNOWN
> Location: har://hdfs-nameservice/hive/tb1/city=nanjing/data.har
> Partition Parameters:
> archiving_level 1
> is_archived true
> numFiles 2
> original_location hdfs://nameservice/hive/tb1/city=nanjing
> totalSize 32
> transient_lastDdlTime 1667177954
> {code}
> When the drop partition is performed, an error occurs because the original_location : hdfs://nameservice/hive/tb1/city=nanjing is deleted first, and then the directory does not exist when the Location: har://hdfs-nameservice/hive/tb1/city=nanjing/data.har is deleted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)