You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2023/02/16 05:28:00 UTC

[jira] [Updated] (HUDI-5358) Fix flaky tests in TestCleanerInsertAndCleanByCommits

     [ https://issues.apache.org/jira/browse/HUDI-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo updated HUDI-5358:
----------------------------
    Fix Version/s: 0.12.2
                       (was: 0.13.0)

> Fix flaky tests in TestCleanerInsertAndCleanByCommits
> -----------------------------------------------------
>
>                 Key: HUDI-5358
>                 URL: https://issues.apache.org/jira/browse/HUDI-5358
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Ethan Guo
>            Assignee: Ethan Guo
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.12.2
>
>
> In the tests, the {{KEEP_LATEST_COMMITS}} cleaner policy is used. This policy first figures out the earliest commit to retain based on the config of the number of retained commits ({{{}hoodie.cleaner.commits.retained{}}}). Then, for each file group, one more version before the earliest commit to retain is also kept from cleaning. The commit for the version can be different among file groups. 
> However, the current validation logic only statically picks the one commit before the earliest commit to retain in the Hudi timeline for all file groups, which does not match the {{KEEP_LATEST_COMMITS}} cleaner policy.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)