[GitHub] [arrow] yordan-pavlov commented on a change in pull request #8364: ARROW-5350: [Rust] Allow filtering on simple lists

[GitBox <gi...@apache.org>]

yordan-pavlov commented on a change in pull request #8364:
URL: https://github.com/apache/arrow/pull/8364#discussion_r503415507



##########
File path: rust/arrow/src/compute/kernels/filter.rs
##########
@@ -230,6 +231,86 @@ macro_rules! filter_dictionary_array {
     }};
 }
 
+macro_rules! filter_primitive_item_list_array {
+    ($context:expr, $array:expr, $item_type:ident) => {{
+        let input_array = $array.as_any().downcast_ref::<ListArray>().unwrap();
+        let values_builder = PrimitiveBuilder::<$item_type>::new($context.filtered_count);
+        let mut builder = ListBuilder::new(values_builder);
+        for i in 0..$context.filter_u64.len() {
+            // foreach u64 batch
+            let filter_batch = $context.filter_u64[i];
+            if filter_batch == 0 {
+                // if batch == 0: skip

Review comment:
       the meaning of this condition is not that batch 0 is skipped, but rather that if the current batch (of 64 bits / elements from the filter array) contains only 0s (i.e. all items are filtered out) then the entire batch should be skipped as there is no items to copy to the output array (all items are filtered out); this continues until a batch is encountered where at least one item is not filtered out (and therefore the numeric representation of the batch of filter values is != 0); this skipping of entire batches of 64 items is a major factor to the performance improvements recently implemented in the filter kernel




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org