[GitHub] [arrow] westonpace commented on issue #11491: apache arrow c++: Ways to invalidate data

[GitBox <gi...@apache.org>]

westonpace commented on issue #11491:
URL: https://github.com/apache/arrow/issues/11491#issuecomment-949082960


   So a few options jump to mind.  I think my answer would sort of depend on what you plan to do with the array after you've deleted the rows.  Also, what language are you working with?
   
   First, you could slice the input array into two arrays and then combine them into a chunked array.  This will be zero-copy but you may not be able to work with chunked arrays and it will keep the deleted elements alive in memory (the buffer will not be released until both slices are released).
   
   If you need a single array then you could do the above and then combine the chunks of the chunked array (the chunked array should have some kind of combine chunks method).  This will involve a memcpy of the entire array.
   
   You could keep a separate standalone "selection mask" bitmap separate from the nullable bitmap.  This would be one bitmap per table/record batch instead of one per column.
   
   If you want to edit/remove the value in place then there are not a lot of options at the moment.  There are some compute functions like `case_when` or `if_else` which could achieve this although I'm not sure how the performance would compare to something like combining chunks (described earlier).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org