[jira] [Updated] (HUDI-5950) Fix pending instants determination for MDT compaction

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-5950:
---------------------------------
    Labels: pull-request-available  (was: )

> Fix pending instants determination for MDT compaction
> -----------------------------------------------------
>
>                 Key: HUDI-5950
>                 URL: https://issues.apache.org/jira/browse/HUDI-5950
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: metadata
>            Reporter: sivabalan narayanan
>            Priority: Major
>              Labels: pull-request-available
>
> Currently the logic to deduce pending instants in MDT is as below
>  * a = we get latest completed delta commit from MDT.
>  * Find any inflights in DT timeline *before* \{a}
> and if we don't find any such inflights, we will go ahead and may be compact MDT.
> But what incase the latest delta commit in MDT succeeded in MDT, but failed in DT. so, it could potentially result in triggering compaction in MDT which should not happen. 
>  
> So, the right fix is 
>  * a = we get latest completed delta commit from MDT.
>  * Find any inflights in DT timeline *before or equals* to \{a}
> This should take care of not triggering compaction in MDT when here are inflights in DT which is committed to MDT. 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)