[GitHub] [arrow-datafusion] tustvold commented on a diff in pull request #3989: Add parquet predicate pushdown metrics

[GitBox <gi...@apache.org>]

tustvold commented on code in PR #3989:
URL: https://github.com/apache/arrow-datafusion/pull/3989#discussion_r1007331972


##########
datafusion/core/src/physical_plan/file_format/row_filter.rs:
##########
@@ -98,14 +108,22 @@ impl ArrowPredicate for DatafusionArrowPredicate {
     }
 
     fn evaluate(&mut self, batch: RecordBatch) -> ArrowResult<BooleanArray> {
+        // scoped timer updates on drop
+        let mut timer = self.time.timer();
         match self
             .physical_expr
             .evaluate(&batch)
             .map(|v| v.into_array(batch.num_rows()))
         {
             Ok(array) => {
                 if let Some(mask) = array.as_any().downcast_ref::<BooleanArray>() {
-                    Ok(BooleanArray::from(mask.data().clone()))
+                    let bool_arr = BooleanArray::from(mask.data().clone());
+                    // TODO is there a more efficient way to count the rows that are filtered?

Review Comment:
   If you don't care about nulls
   
   ```
   bool_arr.values().count_set_bits_offset(self.offset(), self.len())
   ```
   
   If you do care about nulls it is slightly more complicated, I'll get something into arrow-rs



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org