You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Theodus (via GitHub)" <gi...@apache.org> on 2026/08/17 17:13:17 UTC
[I] `ScalarValue::eq_array` is inconsistent with `ScalarValue` equality for floating-point values [datafusion]
Theodus opened a new issue, #24431:
URL: https://github.com/apache/datafusion/issues/24431
### Describe the bug
`ScalarValue::eq_array` is documented as an optimized equivalent of extracting an array element with `ScalarValue::try_from_array` and comparing the resulting scalars. However, for floating-point values `eq_array` uses IEEE equality while `ScalarValue::PartialEq` compares bit representations, producing inconsistent results:
- Identical NaN bit patterns compare equal as `ScalarValue`s but unequal through `eq_array`.
- `+0.0` and `-0.0` compare unequal as `ScalarValue`s but equal through `eq_array`.
### To Reproduce
Compare these scalars with the corresponding Float64 array elements:
```text
ScalarValue(NaN).eq_array([NaN], 0) = false
ScalarValue(+0.0).eq_array([-0.0], 0) = true
```
In contrast, extracting those elements with `ScalarValue::try_from_array` and using `ScalarValue::eq` considers the identical NaN equal and the two signed zeros unequal.
### Expected behavior
`ScalarValue::eq_array` should have the same floating-point equality semantics as `ScalarValue::PartialEq`:
- Identical NaN bit patterns should compare equal.
- Different NaN bit patterns should remain distinct.
- `+0.0` and `-0.0` should remain distinct.
### Additional context
`ScalarValue` implements `Eq` and `Hash`, and its float hashes and ordering are based on bit representations/total ordering. Changing `ScalarValue::PartialEq` to IEEE equality would violate `Eq` reflexivity for NaNs and require coordinated changes to hashing and ordering. The narrow fix is therefore to make the Float16, Float32, and Float64 branches of `eq_array` compare `to_bits()` values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ScalarValue::eq_array` is inconsistent with `ScalarValue` equality for floating-point values [datafusion]
Posted by "shinzoxD (via GitHub)" <gi...@apache.org>.
shinzoxD commented on issue #24431:
URL: https://github.com/apache/datafusion/issues/24431#issuecomment-5346916572
https://github.com/apache/datafusion/pull/24499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ScalarValue::eq_array` is inconsistent with `ScalarValue` equality for floating-point values [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #24431: `ScalarValue::eq_array` is inconsistent with `ScalarValue` equality for floating-point values
URL: https://github.com/apache/datafusion/issues/24431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ScalarValue::eq_array` is inconsistent with `ScalarValue` equality for floating-point values [datafusion]
Posted by "shinzoxD (via GitHub)" <gi...@apache.org>.
shinzoxD commented on issue #24431:
URL: https://github.com/apache/datafusion/issues/24431#issuecomment-5346721961
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org