You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Ryan Blue (JIRA)" <ji...@apache.org> on 2016/10/14 23:08:20 UTC

[jira] [Created] (PARQUET-751) DictionaryFilter patch broke column projection

Ryan Blue created PARQUET-751:
---------------------------------

             Summary: DictionaryFilter patch broke column projection
                 Key: PARQUET-751
                 URL: https://issues.apache.org/jira/browse/PARQUET-751
             Project: Parquet
          Issue Type: Bug
          Components: parquet-mr
    Affects Versions: 1.9.0
            Reporter: Ryan Blue
            Assignee: Ryan Blue
             Fix For: 1.9.0


The {{DictionaryFilter}} patch included a change to reuse a {{ParquetFileReader}} rather than creating new ones (and opening new file streams). But by reusing a reader, the projection columns are no longer set. The result is that all columns are read from disk, but only the requested columns are materialized. The fix is to set the requested schema on the file reader when it is fetched from the read context inside {{InternalParquetRecordReader}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)