[PR] fix(core): ignore NaN when tracking min/max column statistics [paimon-mosaic]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch opened a new pull request, #83:
URL: https://github.com/apache/paimon-mosaic/pull/83

   ## Purpose
   
   `compare_values` returns `None` for any pair involving NaN, so a NaN bound can never be replaced. A FLOAT/DOUBLE row group whose first non-null value was NaN had both min and max pinned to NaN, making the range useless for row-group skipping.
   
   ## Changes
   
   - Let a non-NaN value replace a NaN bound. An all-NaN column keeps NaN bounds, so a row group with any non-null value still has both set, as `serialize_stats` requires.
   
   ## Tests
   
   - `test_stats_nan_does_not_pin_min_max`, `test_stats_all_nan_keeps_both_bounds_set`.
   - `cargo fmt --all -- --check`, `cargo clippy --all-targets --workspace -- -D warnings`, `cargo test --release --workspace`: 475 passed, 0 failed (interop cases skipped locally).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(core): ignore NaN when tracking min/max column statistics [paimon-mosaic]

["jianguotian (via GitHub)" <gi...@apache.org>]

jianguotian commented on PR #83:
URL: https://github.com/apache/paimon-mosaic/pull/83#issuecomment-5750103020

   For the 0.3.0 release, I propose treating this as a correctness blocker: NaN can pin min/max bounds and make row-group statistics unusable for pruning. The current head is mergeable and all five CI jobs are green.
   
   @JingsongLi could you please review this for inclusion in 0.3.0?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org