[jira] [Assigned] (IMPALA-13501) Conflicting commits to Iceberg tables leave uncommitted orphan files

["Noemi Pap-Takacs (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noemi Pap-Takacs reassigned IMPALA-13501:
-----------------------------------------

    Assignee:     (was: Noemi Pap-Takacs)

> Conflicting commits to Iceberg tables leave uncommitted orphan files
> --------------------------------------------------------------------
>
>                 Key: IMPALA-13501
>                 URL: https://issues.apache.org/jira/browse/IMPALA-13501
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>            Reporter: Noemi Pap-Takacs
>            Priority: Major
>              Labels: impala-iceberg
>
> Iceberg supports multiple writers with optimistic concurrency. Each writer can write new files which are then added to the table after a validation check to ensure that the commit does not conflict with other modifications made during the execution.
> When there was a conflicting change and the newly written files cannot be committed, there are 2 ways to proceed: the commit can be retried and rebased on top of the latest snapshot. If this cannot resolve the conflict, the change cannot be committed and the files become orphan files in the file system.
> It would be nice to remove the remaining files from an unsuccessful commit in one step. Deleting orphan files later as a table maintenance step is also a possible resolution.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org