You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/12/22 16:47:00 UTC

[jira] [Created] (HUDI-5465) Fix compaction and rollback handling in MDT for multi-writer scenarios in DT

sivabalan narayanan created HUDI-5465:
-----------------------------------------

             Summary: Fix compaction and rollback handling in MDT for multi-writer scenarios in DT
                 Key: HUDI-5465
                 URL: https://issues.apache.org/jira/browse/HUDI-5465
             Project: Apache Hudi
          Issue Type: Bug
          Components: metadata
            Reporter: sivabalan narayanan


lets say c50 is latest DC in MDT. c49 from DT comes through(multi-writer). triggers compaction in MDT(since ignoring c49 there are no other pending instants in DT). new base instant time is c50. and we add 49.deltacommit to MDT. and during the process we crash. 
rollback for 49 kicks in DT. When applying rollback of 49 to MDT, we detect 49 has already been compacted since last compacted time is 50 and {*}fail the rollback when we try to apply to MDT{*}.

 

We need to fix this entire flow for rollbacks and compaction related multi-writer scenarios. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)