You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tony Zhang (JIRA)" <ji...@apache.org> on 2019/06/22 04:59:00 UTC
[jira] [Commented] (SPARK-28135) ceil/ceiling/floor/power returns
incorrect values
[ https://issues.apache.org/jira/browse/SPARK-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870057#comment-16870057 ]
Tony Zhang commented on SPARK-28135:
------------------------------------
*For _ceil_ and _floor_:* in sql/catalyst/expressions/mathExpressions.scala, the Ceil code is like below:
{code:java}
override def dataType: DataType = child.dataType match {
case dt @ DecimalType.Fixed(_, 0) => dt
case DecimalType.Fixed(precision, scale) =>
DecimalType.bounded(precision - scale + 1, 0)
case _ => LongType
}
protected override def nullSafeEval(input: Any): Any = child.dataType match {
case LongType => input.asInstanceOf[Long]
case DoubleType => f(input.asInstanceOf[Double]).toLong
case DecimalType.Fixed(_, _) => input.asInstanceOf[Decimal].ceil
}
{code}
I don't know ** why Long is prefered here, but after changing 'LongType' into 'DoubleType', and then removed the 'toLong' conversion, I got expected output:
{code:java}
spark.sql("select ceil(double(1.2345678901234e+200)), ceiling(double(1.2345678901234e+200)), floor(double(1.2345678901234e+200))").show
+------------------------------------------+------------------------------------------+-------------------------------------------+---------------------------------------------+
|CEIL(CAST(1.2345678901234E+200 AS DOUBLE))|CEIL(CAST(1.2345678901234E+200 AS DOUBLE))|FLOOR(CAST(1.2345678901234E+200 AS DOUBLE))|POWER(CAST(1 AS DOUBLE), CAST(NaN AS DOUBLE))|
+------------------------------------------+------------------------------------------+-------------------------------------------+---------------------------------------------+
| 1.2345678901234E200| 1.2345678901234E200| 1.2345678901234E200|
+------------------------------------------+------------------------------------------+-------------------------------------------+---------------------------------------------+{code}
*Fo**r pow*: it returns "NaN" because that's what Java returns with "1.0^NaN",
{code:java}
double num = Double.valueOf("NaN");
System.out.println(Math.pow(1.0, num)); // prints out `NaN`
{code}
> ceil/ceiling/floor/power returns incorrect values
> -------------------------------------------------
>
> Key: SPARK-28135
> URL: https://issues.apache.org/jira/browse/SPARK-28135
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Yuming Wang
> Priority: Major
>
> {noformat}
> spark-sql> select ceil(double(1.2345678901234e+200)), ceiling(double(1.2345678901234e+200)), floor(double(1.2345678901234e+200)), power('1', 'NaN');
> 9223372036854775807 9223372036854775807 9223372036854775807 NaN
> {noformat}
> {noformat}
> postgres=# select ceil(1.2345678901234e+200::float8), ceiling(1.2345678901234e+200::float8), floor(1.2345678901234e+200::float8), power('1', 'NaN');
> ceil | ceiling | floor | power
> ----------------------+----------------------+----------------------+-------
> 1.2345678901234e+200 | 1.2345678901234e+200 | 1.2345678901234e+200 | 1
> (1 row)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org