You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2022/03/06 14:05:00 UTC

[jira] [Commented] (HUDI-3406) Rollback incorrectly relying on FS listing instead of Commit Metadata

    [ https://issues.apache.org/jira/browse/HUDI-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501968#comment-17501968 ] 

Raymond Xu commented on HUDI-3406:
----------------------------------

[~shivnarayan] 2nd thought on the logic: is it actually better to always use fs listing for rollback operation? 1st using commit metadata saves fs load but i don't think perf is a big concern given rollback is not a frequent event. 2nd fs listing serves a better source of truth for files to be cleaned up. can we make a call here? cc [~x1q1j1] [~alexey.kudinkin] 

> Rollback incorrectly relying on FS listing instead of Commit Metadata
> ---------------------------------------------------------------------
>
>                 Key: HUDI-3406
>                 URL: https://issues.apache.org/jira/browse/HUDI-3406
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Alexey Kudinkin
>            Assignee: Forward Xu
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.11.0
>
>
> [ListingBasedRollbackHelper|https://github.com/apache/hudi/blob/973087f385f0e42bd42e2d57008a756e182041ff/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/ListingBasedRollbackHelper.java#L135] is relying on FS listing (`getBaseAndLogFilesToBeDeleted`) to determine which files should be deleted instead of relying on Commit Metadata for successfully completed commits.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)