You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "roncenzhao (JIRA)" <ji...@apache.org> on 2017/03/19 13:45:41 UTC
[jira] [Commented] (SPARK-20017) Functions "str_to_map" and
"explode" throws NPE exceptioin
[ https://issues.apache.org/jira/browse/SPARK-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931691#comment-15931691 ]
roncenzhao commented on SPARK-20017:
------------------------------------
The root cause of this exception is:
The `StringToMap`'s `dataType` is `MapType(StringType, StringType, valueContainsNull = false)` and the result of `str_to_map('')` is `"" -> null`, so in the function `explode`, it will throw NPE exception.
I think we should change the `dataType` of `StringToMap` to `MapType(StringType, StringType)`, the default value of `valueContainsNull` is `true`.
> Functions "str_to_map" and "explode" throws NPE exceptioin
> ----------------------------------------------------------
>
> Key: SPARK-20017
> URL: https://issues.apache.org/jira/browse/SPARK-20017
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: roncenzhao
> Attachments: screenshot-1.png
>
>
> ```
> val sqlDf = spark.sql("select k,v from (select str_to_map('') as map_col from range(2)) tbl lateral view explode(map_col) as k,v")
> sqlDf.show
> ```
> The sql throws NPE exception.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org