You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/07/06 21:06:04 UTC
[jira] [Commented] (SPARK-6123) Parquet reader should use the
schema of every file to create converter
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615490#comment-14615490 ]
Apache Spark commented on SPARK-6123:
-------------------------------------
User 'liancheng' has created a pull request for this issue:
https://github.com/apache/spark/pull/7231
> Parquet reader should use the schema of every file to create converter
> ----------------------------------------------------------------------
>
> Key: SPARK-6123
> URL: https://issues.apache.org/jira/browse/SPARK-6123
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Yin Huai
> Priority: Critical
>
> For two parquet files for the same table having an array column, if values of the array in one file was created when containsNull was true and those in another file was created when containsNull was false, the containsNull in the merged schema will be true and we cannot correctly read data from the table created with containsNull=false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org