You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Alex Kushnir (JIRA)" <ji...@apache.org> on 2019/06/27 14:30:00 UTC
[jira] [Updated] (SPARK-28186) array_contains returns null instead
of false when one of the items in the array is null
[ https://issues.apache.org/jira/browse/SPARK-28186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Kushnir updated SPARK-28186:
---------------------------------
Description:
If array of items contains a null item then array_contains returns true if item is found but if item is not found it returns null instead of false
Seq(
(1, Seq("a", "b", "c")),
(2, Seq("a", "b", null, "c"))
).toDF("id", "vals").createOrReplaceTempView("tbl")
spark.sql("select id, vals, array_contains(vals, 'a') as has_a, array_contains(vals, 'd') as has_d from tbl").show
+----+---------++----------+
|id|vals|has_a|has_d|
+----+---------++----------+
|1|[a, b, c]|true|false|
|2|[a, b,, c]|true|null|
+----+---------++----------+
was:
If array of items contains a null item when array_contains returns true if item is found but if item is not found it returns null instead of false
Seq(
(1, Seq("a", "b", "c")),
(2, Seq("a", "b", null, "c"))
).toDF("id", "vals").createOrReplaceTempView("tbl")
spark.sql("select id, vals, array_contains(vals, 'a') as has_a, array_contains(vals, 'd') as has_d from tbl").show
+---+----------+-----+-----+
| id| vals |has_a|has_d|
+---+----------+-----+-----+
| 1| [a, b, c]| true|false|
| 2|[a, b,, c]| true| null|
+---+----------+-----+-----+
> array_contains returns null instead of false when one of the items in the array is null
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-28186
> URL: https://issues.apache.org/jira/browse/SPARK-28186
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Alex Kushnir
> Priority: Major
>
> If array of items contains a null item then array_contains returns true if item is found but if item is not found it returns null instead of false
> Seq(
> (1, Seq("a", "b", "c")),
> (2, Seq("a", "b", null, "c"))
> ).toDF("id", "vals").createOrReplaceTempView("tbl")
> spark.sql("select id, vals, array_contains(vals, 'a') as has_a, array_contains(vals, 'd') as has_d from tbl").show
> +----+---------++----------+
> |id|vals|has_a|has_d|
> +----+---------++----------+
> |1|[a, b, c]|true|false|
> |2|[a, b,, c]|true|null|
> +----+---------++----------+
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org