You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "spicysharkco (via GitHub)" <gi...@apache.org> on 2026/10/15 22:45:28 UTC
[PR] fix(spark): normalize -0.0 in atan2 like Spark [datafusion]
spicysharkco opened a new pull request, #26394:
URL: https://github.com/apache/datafusion/pull/26394
## Which issue does this PR close?
- Closes #26333.
## Rationale for this change
Spark's `Atan2` adds `0.0` to both arguments before calling `atan2`, which turns `-0.0` into `0.0`. The source comment says why: "With codegen, the values returned by -0.0 and 0.0 are different. Handled with +0.0". Both the interpreted path and the codegen path do it:
- `nullSafeEval`: `math.atan2(input1.asInstanceOf[Double] + 0.0, input2.asInstanceOf[Double] + 0.0)`
- `doGenCode`: `java.lang.Math.atan2($c1 + 0.0, $c2 + 0.0)`
https://github.com/apache/spark/blob/v4.0.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1229-L1239
`SparkAtan2` passed the raw arguments to `f64::atan2`, so a `-0.0` argument could give a different result, for example `-pi` for `atan2(-0.0, -1.0)` where Spark returns `pi`.
## What changes are included in this PR?
- `SparkAtan2` adds `0.0` to both arguments before calling `f64::atan2`.
- `atan2.slt`: the expectation for `atan2(-0.0, -1.0)` was the C99 result and is now the Spark result, `3.141592653589793`.
## What is the testing strategy for this PR?
New cases in `datafusion/sqllogictest/test_files/spark/math/atan2.slt`:
- `query R` cases for `atan2(0.0, -0.0)` and `atan2(-0.0, -0.0)`, and an array-path query over those two plus `atan2(-0.0, -1.0)`.
- A `query T` check on the string form of `atan2(-0.0, 1.0)`, `atan2(-0.0, 0.0)`, `atan2(0.0, -0.0)` and `atan2(-0.0, -0.0)`, which must be `0.0` and not `-0.0`. A numeric `query R` cannot tell the two apart.
The expected values come from the Spark code above (`-0.0 + 0.0` is `0.0`), and match the three queries in the issue. These cases fail on `main` and pass with this change.
## Are there any user-facing changes?
`atan2` in `datafusion-spark` now returns the Spark result when either argument is `-0.0`. No API change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): normalize -0.0 in atan2 like Spark [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #26394: fix(spark): normalize -0.0 in atan2 like Spark
URL: https://github.com/apache/datafusion/pull/26394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): normalize -0.0 in atan2 like Spark [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #26394:
URL: https://github.com/apache/datafusion/pull/26394#issuecomment-6506382515
Thanks for the contribution! I am closing this under DataFusion's AI Dump policy.
https://datafusion.apache.org/contributor-guide/ai-policy.html#what-is-an-ai-dump-and-why-it-is-not-helpful
Review capacity is very limited and this is a PR from a new contributor flagged for opening multiple PRs in a short time, with no engagement since it was opened. The policy asks new contributors to work with maintainers on one PR at a time rather than opening several at once.
If this fix matters to you, please pick the single change you care most about, reopen or resubmit it with a short description in your own words, and we are happy to help with review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org