You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2023/01/12 00:33:00 UTC

[jira] [Closed] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT

     [ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo closed HUDI-5434.
---------------------------
    Resolution: Fixed

> Fix archival in MDT to not rely on rollbacks/clean in DT
> --------------------------------------------------------
>
>                 Key: HUDI-5434
>                 URL: https://issues.apache.org/jira/browse/HUDI-5434
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: metadata
>            Reporter: sivabalan narayanan
>            Assignee: Ethan Guo
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.13.0
>
>
> as of now, archival in MDT is guarded until first entry in DT's active timeline. but DT could contain rollback that could date back few days or even weeks. So, we need to fix that to check for first write action in DT (commit, delta commit, replace commit) and then guard MDT archival based on that. 
>  
> Impact:
> could result in huge no of entries in active timeline in MDT. might hamper perf or throttling in cloud stores.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)