[GitHub] [arrow-rs] jhorstmann commented on pull request #1499: Alternative implementation of nullif kernel by slicing nested buffers

[GitBox <gi...@apache.org>]

jhorstmann commented on PR #1499:
URL: https://github.com/apache/arrow-rs/pull/1499#issuecomment-1096878229

   @alamb I'm also not sure what is the best way forward. It is a lot of code just to support one kernel and it changes quite a bit how we work with Buffers. And it still does not avoid copying completely for boolean types. If these tradeoffs are ok there are also some TODOs still left:
   
    - Map types are not yet supported
    - The previous PR #521 has a few more test cases that could be integrated
    - The function needs to be renamed, the new name was only choosen to make the diff easier to read
   
   Longer term I'd like to experiment with removing the `offset` from `ArrayData` and see if slicing could be implemented in general with a method like this.For the validity, an offset could be introduced in `Bitmap` so it can be sliced without copying. The open problem is then the `BooleanArray`, which would ideally need to contain a `Bitmap` instead of a `Buffer` for it's data which could be sliced zero-copy. But there is no way to store this data bitmap in `ArrayData`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org