[jira] [Commented] (SPARK-7743) Upgrade parquet dependency

["Cheng Lian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551579#comment-14551579 ] 

Cheng Lian commented on SPARK-7743:
-----------------------------------

We probably want to upgrade to the newly release 1.7.0, which is identical to 1.6.0 except for some renaming:

{quote}
This release is identical to the 1.6.0-incubating release, except that the packages and artifacts have been renamed to org.apache.parquet. There are no behavioral changes to make transitioning to the new names easier. 
{quote}

> Upgrade parquet dependency
> --------------------------
>
>                 Key: SPARK-7743
>                 URL: https://issues.apache.org/jira/browse/SPARK-7743
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Thomas Omans
>
> There are many outstanding issues with the parquet format that have been resolved between the version depended on by spark (1.6.0rc3 as of spark 1.3.1) and the most recent parquet release (1.6.0).
> Some of these are things include not supporting schema migration when using parquet with avro, not supporting summary metadata in the parquet footers causing null pointer exceptions reading, and many others.
> See https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-160 for the full list of fixes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org