You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "comphead (via GitHub)" <gi...@apache.org> on 2026/10/14 23:25:11 UTC
[I] [Bug] atan2 does not normalize -0.0 the way Spark does [datafusion]
comphead opened a new issue, #26333:
URL: https://github.com/apache/datafusion/issues/26333
### Describe the bug
`SparkAtan2` passes the raw arguments to Rust's `f64::atan2` ([atan2.rs#L82](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/spark/src/function/math/atan2.rs#L82)). Spark's `Atan2` adds `0.0` to both arguments first, which turns a negative zero into a positive zero ([mathExpressions.scala#L1229-L1237](https://github.com/apache/spark/blob/v4.0.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1229-L1237)). The Spark source explains why: "With codegen, the values returned by -0.0 and 0.0 are different. Handled with +0.0".
So `atan2` returns a different value whenever `y` or `x` is `-0.0`.
### To Reproduce
Spark 4.0.0:
```sql
SELECT atan2(CAST('-0.0' AS DOUBLE), -1.0D)
-- 3.141592653589793
SELECT atan2(0.0D, CAST('-0.0' AS DOUBLE))
-- 0.0
SELECT atan2(CAST('-0.0' AS DOUBLE), CAST('-0.0' AS DOUBLE))
-- 0.0
```
DataFusion: the existing test at [atan2.slt#L155](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/sqllogictest/test_files/spark/math/atan2.slt#L155) documents the C99 result:
```
query R
SELECT atan2(-0.0::double, -1.0::double)
----
-3.141592653589793
```
The other two inputs were not run on DataFusion. By the C99 definition that Rust follows, `atan2(0.0, -0.0)` is `pi` and `atan2(-0.0, -0.0)` is `-pi`.
### Expected behavior
Add `0.0` to both arguments before calling `atan2`, as Spark does, and change the expectation at `atan2.slt#L155` to `3.141592653589793`.
### Additional context
Found by running the queries in `datafusion/sqllogictest/test_files/spark` on a local Spark and comparing the results with the expected values in the files, which is the kind of check proposed in #17045. Spark results come from Apple's build of Spark 4.0.0 (`4.0.0.65-apple`) with a UTC session and `spark.sql.ansi.enabled=false` unless stated. The Spark source links point at the `v4.0.0` tag. DataFusion results are the sqllogictest expectations at 9dec77584, which CI enforces. Anything marked as not run was derived from reading the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Bug] atan2 does not normalize -0.0 the way Spark does [datafusion]
Posted by "galaxytemple (via GitHub) via github" <gi...@datafusion.apache.org>.
galaxytemple commented on issue #26333:
URL: https://github.com/apache/datafusion/issues/26333#issuecomment-6539416063
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org