[GitHub] [arrow-rs] nevi-me commented on a change in pull request #256: fix NaN handling in parquet statistics

[GitBox <gi...@apache.org>]

nevi-me commented on a change in pull request #256:
URL: https://github.com/apache/arrow-rs/pull/256#discussion_r626248018



##########
File path: parquet/src/column/writer.rs
##########
@@ -922,11 +922,14 @@ impl<T: DataType> ColumnWriterImpl<T> {
     }
 
     fn update_page_min_max(&mut self, val: &T::T) {
-        if self.min_page_value.as_ref().map_or(true, |min| min > val) {
-            self.min_page_value = Some(val.clone());
-        }
-        if self.max_page_value.as_ref().map_or(true, |max| max < val) {
-            self.max_page_value = Some(val.clone());
+        // simple "isNaN" check that works for all types
+        if val == val {

Review comment:
       I don't think that's necessary. It'll just be adding overhead for us.
   
   The compiler can figure out what we're trying to do, and will optimise out the branch for anything that's not a floating point number.
   
   Look at this example: https://godbolt.org/z/z7M3Mfqzz
   
   The `is_nan::<i32>()` always returns `false`, and the `ucomisd` does the floating point comparison




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org