You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "ZiyueGuan (Jira)" <ji...@apache.org> on 2021/11/16 09:40:00 UTC

[jira] [Created] (HUDI-2771) Handle FileNotExist exception in parquet Utils

ZiyueGuan created HUDI-2771:
-------------------------------

             Summary: Handle FileNotExist exception in parquet Utils
                 Key: HUDI-2771
                 URL: https://issues.apache.org/jira/browse/HUDI-2771
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: ZiyueGuan


As spark may use speculation. there may be two base file in one file group with same instant time. In such case, we check if parquet is valid by reading footer. However, a cleaning process may happen during this process, which lead to a FileNotFoundException. We may catch such exception and treat it as a invalid parquet. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)