You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ian (JIRA)" <ji...@apache.org> on 2016/03/07 23:22:40 UTC
[jira] [Updated] (SPARK-13731) expression evaluation for NaN in
select statement
[ https://issues.apache.org/jira/browse/SPARK-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ian updated SPARK-13731:
------------------------
Description:
We are expecting arithmetic expression a/b should be:
1. returning NaN if a=0 and b=0
2. returning Infinity if a=1 and b=0
Is the expectation reasonable?
The following is a simple test case snippet that read from storage and evaluate arithmetic in select.
It si assuming org.apache.spark.sql.hive.execution.SQLQuerySuite:
{code}
test("Expression should be evaluated to Nan/Infinity in Select") {
withTable("testNan") {
withTempTable("src") {
Seq((1d, 0d), (0d, 0d)).toDF().registerTempTable("src")
sql("CREATE TABLE testNan(a double, b double) STORED AS PARQUET AS SELECT * FROM src")
}
checkAnswer(sql(
"""
|SELECT a/b FROM testNan
""".stripMargin),
Seq(
Row(Double.PositiveInfinity),
Row(Double.NaN)
)
)
}
}
== Physical Plan ==
Project [(a#28 / b#29) AS _c0#30]
+- Scan ParquetRelation: default.testnan[a#28,b#29] InputPaths: file:/private/var/folders/dy/19y6pfm92pj9s40mbs8xd9hm0000gp/T/warehouse--5b617080-e909-4812-90e8-63d2dd0aef5a/testnan
== Results ==
!== Correct Answer - 2 == == Spark Answer - 2 ==
![Infinity] [null]
![NaN] [null]
{code}
was:
We are expecting arithmetic expression a/b should be:
1. returning NaN if a=0 and b=0
2. returning Infinity if a=1 and b=0
Is the expectation reasonable?
The following is a simple test case snippet that read from storage and evaluate arithmetic in select.
It si assuming org.apache.spark.sql.hive.execution.SQLQuerySuite:
{code}
test("Expression should be evaluated to Nan/Infinity in Select") {
withTable("testNan") {
withTempTable("src") {
Seq((1d, 0d), (0d, 0d)).toDF().registerTempTable("src")
sql("CREATE TABLE testNan(a double, b double) STORED AS PARQUET AS SELECT * FROM src")
}
checkAnswer(sql(
"""
|SELECT a/b FROM testNan
""".stripMargin),
Seq(
Row(Double.PositiveInfinity),
Row(Double.NaN)
)
)
}
}
{code}
> expression evaluation for NaN in select statement
> -------------------------------------------------
>
> Key: SPARK-13731
> URL: https://issues.apache.org/jira/browse/SPARK-13731
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Ian
>
> We are expecting arithmetic expression a/b should be:
> 1. returning NaN if a=0 and b=0
> 2. returning Infinity if a=1 and b=0
> Is the expectation reasonable?
> The following is a simple test case snippet that read from storage and evaluate arithmetic in select.
> It si assuming org.apache.spark.sql.hive.execution.SQLQuerySuite:
> {code}
> test("Expression should be evaluated to Nan/Infinity in Select") {
> withTable("testNan") {
> withTempTable("src") {
> Seq((1d, 0d), (0d, 0d)).toDF().registerTempTable("src")
> sql("CREATE TABLE testNan(a double, b double) STORED AS PARQUET AS SELECT * FROM src")
> }
> checkAnswer(sql(
> """
> |SELECT a/b FROM testNan
> """.stripMargin),
> Seq(
> Row(Double.PositiveInfinity),
> Row(Double.NaN)
> )
> )
> }
> }
> == Physical Plan ==
> Project [(a#28 / b#29) AS _c0#30]
> +- Scan ParquetRelation: default.testnan[a#28,b#29] InputPaths: file:/private/var/folders/dy/19y6pfm92pj9s40mbs8xd9hm0000gp/T/warehouse--5b617080-e909-4812-90e8-63d2dd0aef5a/testnan
> == Results ==
> !== Correct Answer - 2 == == Spark Answer - 2 ==
> ![Infinity] [null]
> ![NaN] [null]
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org