You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Prashant Wason (Jira)" <ji...@apache.org> on 2023/10/05 04:02:00 UTC

[jira] [Closed] (HUDI-6153) Change the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks

     [ https://issues.apache.org/jira/browse/HUDI-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prashant Wason closed HUDI-6153.
--------------------------------
    Fix Version/s: 0.14.0
                       (was: 0.14.1)
       Resolution: Fixed

> Change the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks
> ---------------------------------------------------------------------------------------------
>
>                 Key: HUDI-6153
>                 URL: https://issues.apache.org/jira/browse/HUDI-6153
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: metadata
>            Reporter: Prashant Wason
>            Assignee: Prashant Wason
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.14.0
>
>
> When rolling back completed commits for indexes like record-index, the list of all keys removed from the dataset is required. This information cannot be available during rollback processing in MDT since the files have already been deleted during the rollback inflight processing. 
> Hence, the current MDT rollback mechanism of adding -files, -col_stats entries does not work for record index.
> This PR changes the rollback mechanism to actually rollback deltacommits on the MDT. This makes the rollback handing faster and keeps the MDT in sync with dataset.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)