[GitHub] [arrow] Dandandan commented on pull request #8796: [Rust] [Experiment] Vec vs current allocations

[GitBox <gi...@apache.org>]

Dandandan commented on pull request #8796:
URL: https://github.com/apache/arrow/pull/8796#issuecomment-736283187


   @jorgecarleitao 
   Maybe I'm saying something weird/impossible, but would it also be possible/ beneficial to store the buffer in a `Vec<T>`? 
   This way it could simplify mutation of the buffer for the different types, while also relying less on unsafe code / code that could segfault or lead to other errors when using it wrong. In profiling/benchmarks I saw there are mayor inefficiencies related to writing values as individual bytes / instead of being able to store them directly in the builder API (e.g. in the append function).
   
   For the rest, I think it really makes sense to push this idea forward as the current implementation is much more complicated without a good reason. I think using `Vec` it will be actually easier to optimize.
   
   Really look forward to those benchmarks too @alamb 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org