You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ayush Saxena (Jira)" <ji...@apache.org> on 2024/09/10 17:24:00 UTC
[jira] [Resolved] (HIVE-28515) Iceberg: Concurrent queries fail during commit with ValidationException
[ https://issues.apache.org/jira/browse/HIVE-28515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ayush Saxena resolved HIVE-28515.
---------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
> Iceberg: Concurrent queries fail during commit with ValidationException
> -----------------------------------------------------------------------
>
> Key: HIVE-28515
> URL: https://issues.apache.org/jira/browse/HIVE-28515
> Project: Hive
> Issue Type: Bug
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> {noformat}
> Caused by: org.apache.iceberg.exceptions.ValidationException: Cannot commit, missing data files: [file:/Users/ayushsaxena/code/hive/iceberg/iceberg-handler/target/tmp/hive7073916777566968859/external/customers/data/00000-0-data-ayushsaxena_20240909232021_99fd025f-1e27-4541-ab3e-77c6f9905eb7-job_17259492220180_0001-6-00001.parquet]
> at org.apache.iceberg.MergingSnapshotProducer.validateDataFilesExist(MergingSnapshotProducer.java:751)
> at org.apache.iceberg.BaseRowDelta.validate(BaseRowDelta.java:116)
> at org.apache.iceberg.SnapshotProducer.apply(SnapshotProducer.java:233)
> at org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:384)
> at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
> at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
> at org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:382)
> at org.apache.iceberg.mr.hive.HiveIcebergOutputCommitter.commitWrite(HiveIcebergOutputCommitter.java:580)
> at org.apache.iceberg.mr.hive.HiveIcebergOutputCommitter.commitTable(HiveIcebergOutputCommitter.java:494)
> at org.apache.iceberg.mr.hive.HiveIcebergOutputCommitter.lambda$commitJobs$4(HiveIcebergOutputCommitter.java:291){noformat}
> Queries failing with {{ValidationException}} during commit even with retry strategy configured with {{write_conflict}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)