You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2023/03/17 19:45:00 UTC
[jira] [Created] (HUDI-5950) Fix pending instants determination for MDT compaction
sivabalan narayanan created HUDI-5950:
-----------------------------------------
Summary: Fix pending instants determination for MDT compaction
Key: HUDI-5950
URL: https://issues.apache.org/jira/browse/HUDI-5950
Project: Apache Hudi
Issue Type: Bug
Components: metadata
Reporter: sivabalan narayanan
Currently the logic to deduce pending instants in MDT is as below
* a = we get latest completed delta commit from MDT.
* Find any inflights in DT timeline *before* \{a}
and if we don't find any such inflights, we will go ahead and may be compact MDT.
But what incase the latest delta commit in MDT succeeded in MDT, but failed in DT. so, it could potentially result in triggering compaction in MDT which should not happen.
So, the right fix is
* a = we get latest completed delta commit from MDT.
* Find any inflights in DT timeline *before or equals* to \{a}
This should take care of not triggering compaction in MDT when here are inflights in DT which is committed to MDT.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)