You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Thomas Graves (Jira)" <ji...@apache.org> on 2020/08/17 20:04:00 UTC
[jira] [Created] (SPARK-32640) Spark 3.1 log(NaN) returns null
instead of NaN
Thomas Graves created SPARK-32640:
-------------------------------------
Summary: Spark 3.1 log(NaN) returns null instead of NaN
Key: SPARK-32640
URL: https://issues.apache.org/jira/browse/SPARK-32640
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.0
Reporter: Thomas Graves
I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now returns a null whereas in Spark 3.0 it returned a NaN. I'm not an expert in this but I thought NaN was correct.
Spark 3.1.0 Example:
>>> df.selectExpr(["value", "log1p(value)"]).show()
+-------------+------------------+
| value| LOG1P(value)|
+-------------+------------------+
|-3.4028235E38| null|
| 3.4028235E38| 88.72283906194683|
| 0.0| 0.0|
| -0.0| -0.0|
| 1.0|0.6931471805599453|
| -1.0| null|
| NaN| null|
+-------------+------------------+
Spark 3.0.0 example:
+-------------+-----------------+
| value| LOG(E(), value)|
+-------------+-----------------+
|-3.4028235E38| null|
| 3.4028235E38|88.72283906194683|
| 0.0| null|
| -0.0| null|
| 1.0| 0.0|
| -1.0| null|
| NaN| NaN|
+-------------+-----------------+
Note it also does the same for log1p, log2, log10
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org