[GitHub] [arrow] alamb commented on a change in pull request #9330: [Rust] [Experiment] [WIP]: Use SmallVec in ArrayData to optimize the common usecase of single-buffer arrays

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #9330:
URL: https://github.com/apache/arrow/pull/9330#discussion_r566785994



##########
File path: rust/arrow/src/array/data.rs
##########
@@ -225,11 +226,11 @@ pub struct ArrayData {
     /// The buffers for this array data. Note that depending on the array types, this
     /// could hold different kinds of buffers (e.g., value buffer, value offset buffer)
     /// at different positions.
-    buffers: Vec<Buffer>,
+    buffers: SmallVec<[Buffer; 1]>,

Review comment:
       I wonder if using something like a specific enum might also work (and be more memory efficient) as well:
   
   ```
   pub enum  ArrayBuffer {
     One(ArrayDataRef),
     Two(ArrayDataRef, ArrayDataRef),
     Many(Vec<ArrayDataRef>)
   }
   ```
   
   I honestly don't know what the maximum number of buffers any specific ArrayData can have, 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org