You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/09/05 14:51:57 UTC

[I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet]

andygrove opened a new issue, #5702:
URL: https://github.com/apache/datafusion-comet/issues/5702

   ### Describe the bug
   
   The signed-zero fixtures in the array SQL tests carry comments that contradict the tests around them, and would mislead anyone deciding whether to re-enable them.
   
   `spark/src/test/resources/sql-tests/expressions/array/array_distinct.sql:139`:
   
   ```
   -- negative zero (literal). Spark's NormalizeFloatingNumbers rewrites -0.0 to 0.0 at
   -- analysis time, so both Spark and Comet collapse it and agree here.
   query ignore(https://issues.apache.org/jira/browse/SPARK-54918)
   SELECT array_distinct(array(0.0, double('-0.0'), 1.0))
   ```
   
   The comment says the two agree, but the query immediately below it is skipped precisely because they do not. `NormalizeFloatingNumbers` only rewrites grouping keys, join keys, window partition specs and `Distinct`; the optimized plan for a plain `SELECT` keeps the `-0.0` literal intact.
   
   The same "NormalizeFloatingNumbers only rewrites literals, not parquet columns" framing appears in `array_except.sql:77`, `array_intersect.sql:162` and `array_union.sql:149`.
   
   ### Steps to reproduce
   
   Read the fixtures, or run `SELECT array_distinct(array(0.0, double('-0.0'), 1.0))` and observe that the literal case diverges too.
   
   ### Expected behavior
   
   The comments describe what actually happens: the divergence is not limited to column-sourced values, and the literal case is skipped for the same reason.
   
   ### Additional context
   
   Depends on the behavior decision in the linked signed-zero issue; the comments should be corrected when the skips are resolved. Found while reviewing #5262.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #5702:
URL: https://github.com/apache/datafusion-comet/issues/5702#issuecomment-5552616232

   Tracked under #5709 ([EPIC] DataFusion 55 upgrade follow-ups).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #5702: Signed-zero array test fixtures claim the literal case agrees with Spark when it does not
URL: https://github.com/apache/datafusion-comet/issues/5702


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Signed-zero array test fixtures claim the literal case agrees with Spark when it does not [datafusion-comet]

Posted by "cestercian (via GitHub)" <gi...@apache.org>.
cestercian commented on issue #5702:
URL: https://github.com/apache/datafusion-comet/issues/5702#issuecomment-5745233601

   Taking this — will fix signed-zero array test fixtures that incorrectly claim agreement with Spark.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org