You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Harini Kannan (Jira)" <ji...@apache.org> on 2019/09/26 18:38:00 UTC

[jira] [Created] (ARROW-6713) [Python] Getting "ArrowIOError: Corrupted file, smaller than file footer" when reading large number of parquet files through ParquetDataset()

Harini Kannan created ARROW-6713:
------------------------------------

             Summary: [Python] Getting "ArrowIOError: Corrupted file, smaller than file footer" when reading large number of parquet files through ParquetDataset()
                 Key: ARROW-6713
                 URL: https://issues.apache.org/jira/browse/ARROW-6713
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Harini Kannan
         Attachments: Screen Shot 2019-09-26 at 2.30.49 PM.png

When trying to read a large number of parquet files (> 600) into ParquetDataset(), getting the error: 

ArrowIOError: Corrupted file, smaller than file footer.

 

Note:

-This works fine for small number of (10-20) parquet files.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)