[jira] [Assigned] (HUDI-8808) Fix concurrent execution of appending rollback blocks in the same file group

["Varun Madan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Varun Madan reassigned HUDI-8808:
---------------------------------

    Assignee: Y Ethan Guo

> Fix concurrent execution of appending rollback blocks in the same file group
> ----------------------------------------------------------------------------
>
>                 Key: HUDI-8808
>                 URL: https://issues.apache.org/jira/browse/HUDI-8808
>             Project: Apache Hudi
>          Issue Type: Sub-task
>            Reporter: Y Ethan Guo
>            Assignee: Y Ethan Guo
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.0.1
>
>   Original Estimate: 3h
>  Remaining Estimate: 3h
>
> When there are multiple log files generated in the same file group in a inflight deltacommit, rollback of such a deltacommit can fail, because (1) the rollback plan contains multiple rollback requests targeting the log files to roll back in the same file group (2) concurrent execution of these rollback requests in Spark executors cause creation of new rollback log files in parallel, which involves determining the new log version concurrently, leading to the same new log version to be used in multiple executors, causing marker creation to fail (i.e., multiple executors try to create the marker on the same file name, and subsequent marker creation requests fail due to the fact that marker already exists).
> Note that this is for Hudi 0.x format only (backwards compatible writer).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)