[jira] [Closed] (HUDI-5950) Fix pending instants determination for MDT compaction

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan closed HUDI-5950.
-------------------------------------
      Assignee: sivabalan narayanan
    Resolution: Fixed

> Fix pending instants determination for MDT compaction
> -----------------------------------------------------
>
>                 Key: HUDI-5950
>                 URL: https://issues.apache.org/jira/browse/HUDI-5950
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: metadata
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.13.1, 0.12.3
>
>
> Currently the logic to deduce pending instants in MDT is as below
>  * a = we get latest completed delta commit from MDT.
>  * Find any inflights in DT timeline *before* \{a}
> and if we don't find any such inflights, we will go ahead and may be compact MDT.
> But what incase the latest delta commit in MDT succeeded in MDT, but failed in DT. so, it could potentially result in triggering compaction in MDT which should not happen. 
>  
> So, the right fix is 
>  * a = we get latest completed delta commit from MDT.
>  * Find any inflights in DT timeline *before or equals* to \{a}
> This should take care of not triggering compaction in MDT when here are inflights in DT which is committed to MDT. 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)