You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "ZiyueGuan (Jira)" <ji...@apache.org> on 2021/11/16 09:45:00 UTC

[jira] [Assigned] (HUDI-2771) Handle FileNotExist exception in parquet Utils

     [ https://issues.apache.org/jira/browse/HUDI-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ZiyueGuan reassigned HUDI-2771:
-------------------------------

    Assignee: ZiyueGuan

> Handle FileNotExist exception in parquet Utils
> ----------------------------------------------
>
>                 Key: HUDI-2771
>                 URL: https://issues.apache.org/jira/browse/HUDI-2771
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: ZiyueGuan
>            Assignee: ZiyueGuan
>            Priority: Major
>
> As spark may use speculation. there may be two base file in one file group with same instant time. In such case, we check if parquet is valid by reading footer. However, a cleaning process may happen during this process, which lead to a FileNotFoundException. We may catch such exception and treat it as a invalid parquet. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)