[jira] [Closed] (HUDI-5465) Fix compaction and rollback handling in MDT for multi-writer scenarios in DT

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan closed HUDI-5465.
-------------------------------------
    Resolution: Duplicate

> Fix compaction and rollback handling in MDT for multi-writer scenarios in DT
> ----------------------------------------------------------------------------
>
>                 Key: HUDI-5465
>                 URL: https://issues.apache.org/jira/browse/HUDI-5465
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: metadata
>            Reporter: sivabalan narayanan
>            Assignee: Ethan Guo
>            Priority: Blocker
>             Fix For: 0.13.0
>
>
> lets say c50 is latest DC in MDT. c49 from DT comes through(multi-writer). triggers compaction in MDT(since ignoring c49 there are no other pending instants in DT). new base instant time is c50. and we add 49.deltacommit to MDT. and during the process we crash. 
> rollback for 49 kicks in DT. When applying rollback of 49 to MDT, we detect 49 has already been compacted since last compacted time is 50 and {*}fail the rollback when we try to apply to MDT{*}.
>  
> We need to fix this entire flow for rollbacks and compaction related multi-writer scenarios. 
>  
> Impact: 
> writes to MDT might fail at some point and users have to disable MDT and make progress



--
This message was sent by Atlassian Jira
(v8.20.10#820010)