[jira] [Resolved] (SPARK-4856) Null & empty string should not be considered as StringType at begining in Json schema inferring

["Michael Armbrust (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Armbrust resolved SPARK-4856.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.3.0

Issue resolved by pull request 3708
[https://github.com/apache/spark/pull/3708]

> Null & empty string should not be considered as StringType at begining in Json schema inferring
> -----------------------------------------------------------------------------------------------
>
>                 Key: SPARK-4856
>                 URL: https://issues.apache.org/jira/browse/SPARK-4856
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Cheng Hao
>             Fix For: 1.3.0
>
>
> We have data like:
> {noformat}
> TestSQLContext.sparkContext.parallelize(
>   """{"ip":"27.31.100.29","headers":{"Host":"1.abc.com","Charset":"UTF-8"}}""" ::
>   """{"ip":"27.31.100.29","headers":{}}""" ::
>   """{"ip":"27.31.100.29","headers":""}""" :: Nil)
> {noformat}
> As empty string (the "headers") will be considered as String, and it ignores the real nested data type (struct type "headers" in line 1), and then we will get the "headers" (in line 1) as String Type, which is not our expectation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org