You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2015/10/09 01:19:26 UTC
[jira] [Resolved] (SPARK-8848) Write Parquet LISTs and MAPs
conforming to Parquet format spec
[ https://issues.apache.org/jira/browse/SPARK-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Lian resolved SPARK-8848.
-------------------------------
Resolution: Fixed
Fix Version/s: 1.6.0
Issue resolved by pull request 8988
[https://github.com/apache/spark/pull/8988]
> Write Parquet LISTs and MAPs conforming to Parquet format spec
> --------------------------------------------------------------
>
> Key: SPARK-8848
> URL: https://issues.apache.org/jira/browse/SPARK-8848
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 1.0.2, 1.1.1, 1.2.2, 1.3.1, 1.4.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
> Fix For: 1.6.0
>
>
> [Parquet format PR #17|https://github.com/apache/parquet-format/pull/17] standardized structures of Parquet complex types (LIST & MAP). Spark SQL should follow this spec and write Parquet data conforming to the standard.
> Note that although currently Parquet files written by Spark SQL is non-standard (because Parquet format spec wasn't clear about this part when Spark SQL Parquet support was authored), it's still compatible with the most recent Parquet format spec, because the format we use is covered by the backwards-compatibility rules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org