You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Ryan Blue (JIRA)" <ji...@apache.org> on 2018/05/24 18:24:00 UTC

[jira] [Created] (PARQUET-1309) Parquet Java uses incorrect stats and dictionary filter properties

Ryan Blue created PARQUET-1309:
----------------------------------

             Summary: Parquet Java uses incorrect stats and dictionary filter properties
                 Key: PARQUET-1309
                 URL: https://issues.apache.org/jira/browse/PARQUET-1309
             Project: Parquet
          Issue Type: Bug
          Components: parquet-mr
            Reporter: Ryan Blue
             Fix For: 1.10.1


In SPARK-24251, we found that the changes to use HadoopReadOptions accidentally switched the [properties that enable stats and dictionary filters|https://github.com/apache/parquet-mr/blob/8bbc6cb95fd9b4b9e86c924ca1e40fd555ecac1d/parquet-hadoop/src/main/java/org/apache/parquet/HadoopReadOptions.java#L83]. Both are enabled by default so it is unlikely that anyone will need to turn them off and there is an easy work-around, but we should fix the properties for 1.10.0. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on 1.8.x).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)