[jira] [Resolved] (PARQUET-751) DictionaryFilter patch broke column projection

["Ryan Blue (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PARQUET-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan Blue resolved PARQUET-751.
-------------------------------
    Resolution: Fixed

> DictionaryFilter patch broke column projection
> ----------------------------------------------
>
>                 Key: PARQUET-751
>                 URL: https://issues.apache.org/jira/browse/PARQUET-751
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.9.0
>            Reporter: Ryan Blue
>            Assignee: Ryan Blue
>             Fix For: 1.9.0
>
>
> The {{DictionaryFilter}} patch included a change to reuse a {{ParquetFileReader}} rather than creating new ones (and opening new file streams). But by reusing a reader, the projection columns are no longer set. The result is that all columns are read from disk, but only the requested columns are materialized. The fix is to set the requested schema on the file reader when it is fetched from the read context inside {{InternalParquetRecordReader}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)