You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "IgnatiusPang (via GitHub)" <gi...@apache.org> on 2026/09/25 07:57:39 UTC
[PR] fix: prevent NaN on extreme precision in round and trunc functions [datafusion]
IgnatiusPang opened a new pull request, #25741:
URL: https://github.com/apache/datafusion/pull/25741
## Rationale for this change
In `datafusion-functions::math::round` and `datafusion-functions::math::trunc`:
1. **Floating-point overflow and underflow to `NaN`**:
Rounding or truncating valid finite floats with large or negative `decimal_places` evaluates `10_f64.powi(dp)`.
- For `f32` with `dp >= 39` (or `f64` with `dp >= 309`), the power factor evaluates to `+infinity`. The functions then compute `(value * inf).round() / inf = inf / inf = NaN`.
- For `f32` with `dp <= -46` (or `f64` with `dp <= -325`), the power factor underflows to `0.0`, computing `0.0 / 0.0 = NaN`.
- **User-visible symptom**: `SELECT round(1.5::float4, 40)` returns `NaN` instead of `1.5`, and `SELECT round(1.5::float4, -50)` returns `NaN` instead of `0.0`.
- **Root cause in `round_factor`**: `T::from(10_f64.powi(decimal_places))` returns `Some(inf)` on `f32`/`f64`, so the error check never triggers and infinity/zero factors pass unchecked into the division arithmetic.
2. **Silent 64-bit to 32-bit truncation in `trunc`**:
`compute_truncate32`, `compute_truncate64`, and `truncate_float_array` cast `y: i64` to `i32` via `y as i32`, which silently wraps large precision inputs.
## What changes are included in this PR?
1. **`round_with_factor` and `round_float` safety guards**:
- Return `value` when `!value.is_finite()`.
- When `factor.is_infinite()` (requested decimal places exceed float representable resolution), return `value` unchanged.
- When `factor.is_zero()` (negative decimal places exceed float magnitude), return `0.0.copysign(value)`.
2. **`truncate_with_factor` safety guards**:
- Return `x` when `!x.is_finite()` or `factor.is_infinite()`.
- Return `0.0.copysign(x)` when `factor.is_zero()`.
- Clamp 64-bit precision to `[i32::MIN, i32::MAX]` before passing to `powi`.
3. **Formal Verification (Lean 4)**:
- Formally verified boundary contracts synthesized and applied via Lean 4 Three-Way Semantic Merge:
- `round_with_factor_no_nan`
- `truncate_with_factor_no_nan`
4. **Comprehensive Regression Tests**:
- Unit tests in `round.rs` (`test_round_float_extreme_decimal_places`) and `trunc.rs` (`test_truncate_extreme_precision`).
- Integration tests in `tests/math_extreme_precision.rs`.
## What is the testing strategy for this PR?
- Full crate test suite pass (407 tests total):
```bash
cargo test -p datafusion-functions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: prevent NaN on extreme precision in round and trunc functions [datafusion]
Posted by "IgnatiusPang (via GitHub)" <gi...@apache.org>.
IgnatiusPang commented on PR #25741:
URL: https://github.com/apache/datafusion/pull/25741#issuecomment-5835247679
```markdown
### 1. SQL Reproducer (Extreme Precision Blowup)
```sql
-- High positive precision: factor overflows to Inf, producing Inf/Inf = NaN
SELECT round(1.5::float, 40) AS r;
-- Current: NaN
-- Expected: 1.5
SELECT trunc(42.5::float, 40) AS t;
-- Current: NaN
-- Expected: 42.5
-- High negative precision: factor underflows to 0.0, producing 0/0 = NaN
SELECT round(1.5::float, -50) AS r;
-- Current: NaN
-- Expected: 0.0
```
**Current Result (unpatched DataFusion):**
```text
+-----+
| r |
+-----+
| NaN |
+-----+
```
**Expected Result:**
- `round(1.5::float, 40)` should return `1.5` (a 32-bit float only has ~7 digits of precision; rounding beyond its precision is a no-op).
- `trunc(42.5::float, 40)` should return `42.5`.
- `round(1.5::float, -50)` should return `0.0`.
---
### 2. Root Cause
In `datafusion/functions/src/math/round.rs`:
```rust
fn round_factor<T: num_traits::Float>(decimal_places: i32) -> Result<T, ArrowError> {
T::from(10_f64.powi(decimal_places)).ok_or_else(...)
}
fn round_float<T>(value: T, decimal_places: i32) -> Result<T, ArrowError> {
let factor = round_factor::<T>(decimal_places)?;
Ok((value * factor).round() / factor)
}
```
- `T::from(10_f64.powi(40))` on `f32` evaluates to `Some(f32::INFINITY)`.
- `(1.5 * inf).round() / inf = inf / inf = NaN`.
- `T::from(10_f64.powi(-50))` on `f32` evaluates to `Some(0.0)`.
- `(1.5 * 0.0).round() / 0.0 = 0.0 / 0.0 = NaN`.
### 3. Proposed Fix
Guard `round_factor` and `truncate_with_factor` against infinite or zero factors: if the positive factor is infinite, return the original value; if the factor underflows to zero, return zero.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: prevent NaN on extreme precision in round and trunc functions [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #25741:
URL: https://github.com/apache/datafusion/pull/25741#issuecomment-5834288077
Thank you for this contribution, but you just opened 8 PRs in the last 8 hours. Can you please pick one and we'll work with you on getting it in before you open more
ALso, it would be very helpful if you could provide a SQL or dataframe reproducer for these issues -- as written they refer to internal details of the code and it is hard to know how/if users would hit them
- https://github.com/apache/datafusion/pull/25741
- https://github.com/apache/datafusion/pull/25743
- https://github.com/apache/datafusion/pull/25744
- https://github.com/apache/datafusion/pull/25745
- https://github.com/apache/datafusion/pull/25746
- https://github.com/apache/datafusion/pull/25747
- https://github.com/apache/datafusion/pull/25751
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: prevent NaN on extreme precision in round and trunc functions [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #25741:
URL: https://github.com/apache/datafusion/pull/25741#issuecomment-5835426453
```sql
andrewlamb@Andrews-MacBook-Pro-3:~/Software/influxdb_pro/ent$ datafusion-cli
DataFusion CLI v55.1.0
> SELECT round(1.5::float, 40) AS r;
+-----+
| r |
+-----+
| NaN |
+-----+
1 row(s) fetched.
Elapsed 0.044 seconds.
```
Indeed it does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org