You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sergio Peña (JIRA)" <ji...@apache.org> on 2015/02/12 18:40:12 UTC

[jira] [Commented] (HIVE-9670) Avoid reading file footers in ParquetRecordReaderWrapper

    [ https://issues.apache.org/jira/browse/HIVE-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318600#comment-14318600 ] 

Sergio Peña commented on HIVE-9670:
-----------------------------------

This fix needs parquet 1.6 release. We should wait until its official version is released.

> Avoid reading file footers in ParquetRecordReaderWrapper
> --------------------------------------------------------
>
>                 Key: HIVE-9670
>                 URL: https://issues.apache.org/jira/browse/HIVE-9670
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergio Peña
>            Assignee: Sergio Peña
>
> ParquetRecordReaderWrapper is reading the file footer to create the splits, but then when calling the realReader.initialize(), the file footer is read again by parquet.
> The issue PARQUET-139 did work to avoid reading the footers in parquet-avro. We should implement the same idea in Hive, and update the parquet library to the latest stable version from upstream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)