[GitHub] [arrow-rs] jhorstmann opened a new issue, #1884: Specifying capacities for ListArrays is not supported in MutableArrayData

[GitBox <gi...@apache.org>]

jhorstmann opened a new issue, #1884:
URL: https://github.com/apache/arrow-rs/issues/1884

   **Describe the bug**
   
   `MutableArrayData::with_capacities` allows specifying the capacity for concatenating arrays. Unfortunately the `Capacities::List` variant does not seem to be supported by the method, leading to a lot of reallocations when concatenating list arrays.
   
   **To Reproduce**
   
   The following test panics with
   
   ```
   Capacities: List(6, Some(Array(15))) not yet supported
   thread 'array::transform::tests::test_list_append_with_capacities' panicked at 'Capacities: List(6, Some(Array(15))) not yet supported', arrow/src/array/transform/mod.rs:398:18
   ```
   
   ```rust
       #[test]
       fn test_list_append_with_capacities() -> Result<()> {
           let mut builder = ListBuilder::<Int64Builder>::new(Int64Builder::new(24));
           builder.values().append_slice(&[1, 2, 3])?;
           builder.append(true)?;
           builder.values().append_slice(&[4, 5])?;
           builder.append(true)?;
           builder.values().append_slice(&[6, 7, 8])?;
           builder.values().append_slice(&[9, 10, 11])?;
           builder.append(true)?;
           let a = builder.finish();
   
           let a_builder = Int64Builder::new(24);
           let mut a_builder = ListBuilder::<Int64Builder>::new(a_builder);
           a_builder.values().append_slice(&[12, 13])?;
           a_builder.append(true)?;
           a_builder.append(true)?;
           a_builder.values().append_slice(&[14, 15])?;
           a_builder.append(true)?;
           let b = a_builder.finish();
   
           let _mutable = MutableArrayData::with_capacities(
               vec![a.data(), b.data()], false,
               Capacities::List(6, Some(Box::new(Capacities::Array(15)))));
   
           Ok(())
       }
   ```
   
   **Expected behavior**
   
   `Capacities::List` should be supported and initialize the correct buffer size for child arrays.
   
   **Additional context**
   Capacities support was added in #382
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb commented on issue #1884: Support specifying capacities for `ListArrays` in `MutableArrayData`

[GitBox <gi...@apache.org>]

alamb commented on issue #1884:
URL: https://github.com/apache/arrow-rs/issues/1884#issuecomment-1164644466

   I am going to presumptively say this is really a feature enhancement rather than a bug, though I can see the case for either view.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb closed issue #1884: Specifying capacities for ListArrays is not supported in MutableArrayData

[GitBox <gi...@apache.org>]

alamb closed issue #1884: Specifying capacities for ListArrays is not supported in MutableArrayData
URL: https://github.com/apache/arrow-rs/issues/1884


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org