You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mightsleep (via GitHub)" <gi...@apache.org> on 2026/10/07 21:56:22 UTC

[PR] bench(arrow): concat of many small FixedSizeList arrays [arrow-rs]

mightsleep opened a new pull request, #11434:
URL: https://github.com/apache/arrow-rs/pull/11434

   # Rationale for this change
   
   No benchmark covers concatenating many small FixedSizeList arrays; "concat fixed size lists" concatenates two large arrays, where the copy dominates. Many one-row arrays is what DataFusion's ScalarValue::iter_to_array does for FixedSizeList scalars (GroupsAccumulatorAdapter::evaluate, first_value/last_value) and what Rerun's chunk batcher does per logged row. Split out so the follow-up fast path can be compared against main.
   
   # What changes are included in this PR?
   
   - concat 1024 arrays fixed size list f32x3 1
   - concat 1024 arrays fixed size list f32x3 nulls 1, built as slices of one array since the generator's fixed seed would give every one-row array the same validity
   
   # Are these changes tested?
   
   Benchmarks only.
   
   # Are there any significant user-facing changes?
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] bench(arrow): concat of many small FixedSizeList arrays [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #11434:
URL: https://github.com/apache/arrow-rs/pull/11434#issuecomment-6050819724

   thanks @mightsleep 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] bench(arrow): concat of many small FixedSizeList arrays [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey merged PR #11434:
URL: https://github.com/apache/arrow-rs/pull/11434


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org