[GitHub] [arrow] andygrove commented on a change in pull request #8860: ARROW-10783: [Rust] [DataFusion] Implement row count statistics for Parquet TableProviderParquet statistics [WIP]

[GitBox <gi...@apache.org>]

andygrove commented on a change in pull request #8860:
URL: https://github.com/apache/arrow/pull/8860#discussion_r539374447



##########
File path: rust/datafusion/src/physical_plan/parquet.rs
##########
@@ -99,6 +100,38 @@ impl ParquetExec {
             batch_size,
         }
     }
+
+    /// Get the statistics from parquet file format
+    pub fn statistics(&self) -> Option<Statistics> {
+        let mut num_rows = 0;
+        let mut total_byte_size = 0;
+        for i in 0..self.filenames.len() {
+            let file = File::open(&self.filenames[i]).ok()?;
+            let file_reader = Arc::new(SerializedFileReader::new(file).ok()?);
+            num_rows += file_reader.metadata().file_metadata().num_rows() as i64;
+            for g in file_reader.metadata().row_groups().iter() {
+                total_byte_size += g.total_byte_size() as i64;
+            }
+        }
+
+        if num_rows >= i64::MAX {

Review comment:
       nit: you could use `std::cmp::min` and `std::cmp::max` instead of these `if` statements to find min/max values




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org