You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Thomas Omans (JIRA)" <ji...@apache.org> on 2015/05/19 23:43:00 UTC

[jira] [Created] (SPARK-7743) Upgrade parquet dependency

Thomas Omans created SPARK-7743:
-----------------------------------

             Summary: Upgrade parquet dependency
                 Key: SPARK-7743
                 URL: https://issues.apache.org/jira/browse/SPARK-7743
             Project: Spark
          Issue Type: Bug
            Reporter: Thomas Omans


There are many outstanding issues with the parquet format that have been resolved between the version depended on by spark (1.6.0rc3 as of spark 1.3.1) and the most recent parquet release (1.6.0).

Some of these are things include not supporting schema migration when using parquet with avro, not supporting summary metadata in the parquet footers causing null pointer exceptions reading, and many others.

See https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-160 for the full list of fixes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org