You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@arrow.apache.org by "Uwe L. Korn (JIRA)" <ji...@apache.org> on 2016/11/01 12:45:58 UTC

[jira] [Commented] (ARROW-357) Default Parquet chunk_size of 64k is too small

    [ https://issues.apache.org/jira/browse/ARROW-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625339#comment-15625339 ] 

Uwe L. Korn commented on ARROW-357:
-----------------------------------

PR: https://github.com/apache/arrow/pull/192

> Default Parquet chunk_size of 64k is too small
> ----------------------------------------------
>
>                 Key: ARROW-357
>                 URL: https://issues.apache.org/jira/browse/ARROW-357
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.1.0
>            Reporter: Uwe L. Korn
>            Assignee: Uwe L. Korn
>
> The current default size of RowGroups when writing Parquet files is 64k rows. This produces a lot of RowGroups given large files which are then really ineffective to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)