You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2016/07/23 01:54:20 UTC
[jira] [Comment Edited] (SPARK-16646) LEAST doesn't accept numeric
arguments with different data types
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389273#comment-15389273 ]
Hyukjin Kwon edited comment on SPARK-16646 at 7/23/16 1:53 AM:
---------------------------------------------------------------
It seems basically comparison between numbers and decimal, and decimals with different precision and scale were not allowed but from 2.0, this restriction was relexted.
here, https://issues.apache.org/jira/browse/HIVE-12070 and https://issues.apache.org/jira/browse/HIVE-12082
So, now Hive in current master produces as below:
{code}
hive> SELECT LEAST(2, 1.5BD);
OK
1.5
{code}
However, when the precision is too high, it became as below:
{code}
hive> SELECT LEAST(100000000000000000, 1.000000000000000000005BD);
OK
1
{code}
This seems goes double,
{code}
hive> SELECT LEAST(12, 1.00000000000000000000000000005BD);
OK
1.0000000000000000000000000001
{code}
In more details, [GenericUDFBaseNwayCompare.initialize|https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNwayCompare.java#L64-L69] is called first to find a common type.
And then, [FunctionRegistry.getCommonClassForComparison|https://github.com/apache/hive/blob/a55f4a3a4b017d7f3b9279ef7d843e4b5f7fcfa0/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L764-L767] is called. Because [PrimitiveObjectInspectorUtils.NUMERIC_GRUOP|https://github.com/apache/hive/blob/1b5ee3d88799a5e80948b7b0f5ca96bba8580efe/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java#L1235-L1243] seems including {{DECIAML}}, it becomes {{TypeInfoFactory.doubleTypeInfo}}. So, it seems always a double in this case.
If it fails to find a common type, this becomes a double anyway, (see [here|https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNwayCompare.java#L71-L73]).
was (Author: hyukjin.kwon):
It seems basically comparison between numbers and decimal, and decimals with different precision and scale were not allowed but from 2.0, this restriction was relexted.
here, https://issues.apache.org/jira/browse/HIVE-12070 and https://issues.apache.org/jira/browse/HIVE-12082
So, now Hive in current master produces as below:
{code}
hive> SELECT LEAST(2, 1.5BD);
OK
1.5
{code}
However, when the precision is too high, it became as below:
{code}
hive> SELECT LEAST(100000000000000000, 1.000000000000000000005BD);
OK
1
{code}
This seems goes double,
{code}
hive> SELECT LEAST(12, 1.00000000000000000000000000005BD);
OK
1.0000000000000000000000000001
{code}
https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNwayCompare.java#L71-L73
> LEAST doesn't accept numeric arguments with different data types
> ----------------------------------------------------------------
>
> Key: SPARK-16646
> URL: https://issues.apache.org/jira/browse/SPARK-16646
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Cheng Lian
> Assignee: Hyukjin Kwon
>
> {code:sql}
> SELECT LEAST(1, 1.5);
> {code}
> {noformat}
> Error: org.apache.spark.sql.AnalysisException: cannot resolve 'least(1, CAST(2.1 AS DECIMAL(2,1)))' due to data type mismatch: The expressions should all have the same type, got LEAST (ArrayBuffer(IntegerType, DecimalType(2,1))).; line 1 pos 7 (state=,code=0)
> {noformat}
> This query works for 1.6.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org