You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/05/16 01:02:00 UTC

[jira] [Updated] (SPARK-24289) Unable to Infer schema

     [ https://issues.apache.org/jira/browse/SPARK-24289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-24289:
---------------------------------
    Fix Version/s:     (was: 2.2.1)

> Unable to Infer schema
> ----------------------
>
>                 Key: SPARK-24289
>                 URL: https://issues.apache.org/jira/browse/SPARK-24289
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.2.1
>            Reporter: Muraliraja
>            Priority: Major
>
> When we run a spark job and when the datatypes during schema inference change and the same is written as parquet files, then when we try to read the data, it fails or mergeSchema fails.
> A better way would be to upcast the datatypes to the highest types via a spark option.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org