You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dilip Biswal (JIRA)" <ji...@apache.org> on 2018/09/25 08:37:00 UTC
[jira] [Updated] (SPARK-25522) Improve type promotion for input
arguments of elementAt function
[ https://issues.apache.org/jira/browse/SPARK-25522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dilip Biswal updated SPARK-25522:
---------------------------------
Description:
In ElementAt, when first argument is MapType, we should coerce the key type and the second argument based on findTightestCommonType. This is not happening currently.
Also, when the first argument is ArrayType, the second argument should be an integer type or a smaller integral type that can be safely casted to an integer type. Currently we may do an unsafe cast.
was:
In ArrayPosition, we currently cast the right hand side expression to match the element type of the left hand side Array. This may result in down casting and may return wrong result or questionable result.
Example :
spark-sql> select array_position(array(1), 1.34);
1
spark-sql> select array_position(array(1), 'foo');
null
We should safely coerce both left and right hand side expressions.
> Improve type promotion for input arguments of elementAt function
> -----------------------------------------------------------------
>
> Key: SPARK-25522
> URL: https://issues.apache.org/jira/browse/SPARK-25522
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Dilip Biswal
> Assignee: Dilip Biswal
> Priority: Major
> Fix For: 2.4.0
>
>
> In ElementAt, when first argument is MapType, we should coerce the key type and the second argument based on findTightestCommonType. This is not happening currently.
> Also, when the first argument is ArrayType, the second argument should be an integer type or a smaller integral type that can be safely casted to an integer type. Currently we may do an unsafe cast.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org