[GitHub] [arrow-rs] alamb commented on a change in pull request #1499: Alternative implementation of nullif kernel by slicing nested buffers

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #1499:
URL: https://github.com/apache/arrow-rs/pull/1499#discussion_r838871984



##########
File path: arrow/src/compute/kernels/boolean.rs
##########
@@ -575,10 +577,242 @@ where
     Ok(PrimitiveArray::<T>::from(data))
 }
 
+/// Creates a (mostly) zero-copy slice of the given buffers so that they can be combined
+/// in the same array with other buffers that start at offset 0.
+/// The only buffers that need an actual copy are booleans (if they are not byte-aligned)
+/// and list/binary/string offsets because the arrow implementation requires them to start at 0.
+/// This is useful when a kernel calculates a new validity bitmap but wants to reuse other buffers.
+fn slice_buffers(

Review comment:
       I am sorry for my ignorance here -- when implementing `nullif` why don't we simply manipulate the bitmask and create a new `ArrayData` that is otherwise the same?? As in why do we want to create a new buffer(s) that start at offset zero? 
   
   
   ```rust
   let new_validity_mask = compute::and(self.validitity, condition);
   
   let new_data = self
     .data()
     .clone()
     .replace_validitiy(new_validity)
   ```
   With apologies for making up `replace_validitiy` that doesn't really exist




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org