You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Alexey Kudinkin (Jira)" <ji...@apache.org> on 2022/01/28 23:09:00 UTC

[jira] [Commented] (HUDI-3180) Include only files belonging to completed commits while bootstrapping metadata table

    [ https://issues.apache.org/jira/browse/HUDI-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484010#comment-17484010 ] 

Alexey Kudinkin commented on HUDI-3180:
---------------------------------------

[~shivnarayan] this is not working correctly for MOR tables – it's not filtering out uncommitted Log Files that are produced by pending operaitons.

> Include only files belonging to completed commits while bootstrapping metadata table
> ------------------------------------------------------------------------------------
>
>                 Key: HUDI-3180
>                 URL: https://issues.apache.org/jira/browse/HUDI-3180
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: writer-core
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Major
>              Labels: pull-request-available, sev:critical
>             Fix For: 0.10.1
>
>
> metadata table bootstrap does not filter for completed commit files, but all files based on fs.list call. We should filter out files that are part of an ongoing commit while doing bootstrap. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)