[jira] [Resolved] (PARQUET-548) Add Java metadata for PageEncodingStats

["Ryan Blue (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PARQUET-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan Blue resolved PARQUET-548.
-------------------------------
       Resolution: Fixed
    Fix Version/s: 1.9.0

Merged #332. Thanks for reviewing, [~julienledem]!

> Add Java metadata for PageEncodingStats
> ---------------------------------------
>
>                 Key: PARQUET-548
>                 URL: https://issues.apache.org/jira/browse/PARQUET-548
>             Project: Parquet
>          Issue Type: New Feature
>          Components: parquet-mr
>            Reporter: Ryan Blue
>            Assignee: Ryan Blue
>             Fix For: 1.9.0
>
>
> PARQUET-384 needs to determine whether an entire column chunk is dictionary-encoded, but it is difficult to detect that case based on the set of encodings for a column. For 1.0, this can be done by checking for a PLAIN page because both dictionary pages and dictionary-encoded pages use PLAIN_DICTIONARY and RLE/BIT_PACKING is only used for repetition and definition levels. But for 2.0, dictionary pages might be using PLAIN and there is no way to tell if a column has fallen back.
> [PageEncodingStats|https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L449] were added to the format to solve this problem, so we just need to implement them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)