You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/12/28 22:36:41 UTC
[PR] WIP: Reduce cloning while reading from Parquet reader [arrow-rs]
alamb opened a new pull request, #9058:
URL: https://github.com/apache/arrow-rs/pull/9058
TODO:
- [ ] Use the new API in other make_array APIs
# Which issue does this PR close?
- related to https://github.com/apache/datafusion/pull/19477
# Rationale for this change
While looking at profiles for some of the ClickBench queries, I found that some non trivial amount of time was creating record batches, so
<img width="1461" height="994" alt="Screenshot 2025-12-28 at 5 27 02 PM" src="https://github.com/user-attachments/assets/ee95d2a5-d722-4310-aa60-5885d5ae9ad4" />
<img width="1420" height="189" alt="Screenshot 2025-12-28 at 5 29 19 PM" src="https://github.com/user-attachments/assets/278ed5e7-448a-4491-bb6e-26c7510d98f8" />
# What changes are included in this PR?
- Add an API to get the fields of an ArrayData
- Use it while creating StructArray
# Are these changes tested?
By existing CI
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
If there are any breaking changes to public APIs, please call them out.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "jhorstmann (via GitHub)" <gi...@apache.org>.
jhorstmann commented on code in PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#discussion_r2683034659
##########
arrow-array/src/array/byte_view_array.rs:
##########
@@ -943,15 +943,15 @@ impl<'a, T: ByteViewType + ?Sized> IntoIterator for &'a GenericByteViewArray<T>
}
impl<T: ByteViewType + ?Sized> From<ArrayData> for GenericByteViewArray<T> {
- fn from(value: ArrayData) -> Self {
- let views = value.buffers()[0].clone();
- let views = ScalarBuffer::new(views, value.offset(), value.len());
- let buffers = value.buffers()[1..].to_vec();
+ fn from(data: ArrayData) -> Self {
+ let (_data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
Review Comment:
Unrelated to the performance improvement: I think this also needs to assert that `data_type` equals `T::DATA_TYPE`, otherwise it allows unchecked casting from binary to string without utf8 validation.
From a performance perspective, not sure if it makes any measurable difference, but after that assert, you could use `data_type` instead of `T::DATA_TYPE` below. That might avoid a call to `DataType::drop` .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697583599
If the benchmarks look good I will split this PR up into some smaller pieces for easier review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3721257035
I made a PR to document that we shouldn't be using make_array if possible
- https://github.com/apache/arrow-rs/pull/9112
I also made a first PR to avoid some clones:
- https://github.com/apache/arrow-rs/pull/9114
I will in parallel look into updating the parquet reader to construct the arrays directly rather than using ArrayData
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#discussion_r2686953805
##########
arrow-array/src/array/byte_view_array.rs:
##########
@@ -943,15 +943,15 @@ impl<'a, T: ByteViewType + ?Sized> IntoIterator for &'a GenericByteViewArray<T>
}
impl<T: ByteViewType + ?Sized> From<ArrayData> for GenericByteViewArray<T> {
- fn from(value: ArrayData) -> Self {
- let views = value.buffers()[0].clone();
- let views = ScalarBuffer::new(views, value.offset(), value.len());
- let buffers = value.buffers()[1..].to_vec();
+ fn from(data: ArrayData) -> Self {
+ let (_data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
Review Comment:
I was able to make a reproducer and filed a ticket.
- https://github.com/apache/arrow-rs/issues/9157
And here is a PR to fix:
- https://github.com/apache/arrow-rs/pull/9158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #9058: Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader)
URL: https://github.com/apache/arrow-rs/pull/9058
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3729354113
> My 2 cents is it would be better to move the codepaths relying on ArrayData over to using the typed arrays directly, this should not only cut down on allocations but unnecessary validation and dispatch overheads.
Per your suggestion, I have made a few PRs to start this process
- https://github.com/apache/arrow-rs/pull/9120
- https://github.com/apache/arrow-rs/pull/9121
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3712272488
My 2 cents is it would be better to move the codepaths relying on ArrayData over to using the typed arrays directly, this should not only cut down on allocations but unnecessary validation and dispatch overheads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697566582
run benchmark arrow_reader arrow_reader_clickbench arrow_reader_row_filter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697794627
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group alamb_reduce_struct_array_clones main
----- -------------------------------- ----
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs 1.09 1289.9±14.64µs ? ?/sec 1.00 1180.0±4.55µs ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs 1.05 1329.7±14.66µs ? ?/sec 1.00 1260.6±25.84µs ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs 1.09 1303.5±25.79µs ? ?/sec 1.00 1190.9±18.91µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, mandatory, no NULLs 1.00 512.2±5.67µs ? ?/sec 1.00 514.3±4.79µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, half NULLs 1.02 692.5±5.04µs ? ?/sec 1.00 676.2±5.27µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, no NULLs 1.00 518.0±3.92µs ? ?/sec 1.00 520.5±10.18µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, mandatory, no NULLs 1.09 591.3±6.11µs ? ?/sec 1.00 541.0±7.51µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, half NULLs 1.06 768.0±4.92µs ? ?/sec 1.00 722.9±5.47µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, no NULLs 1.08 598.3±6.95µs ? ?/sec 1.00 551.6±12.13µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, mandatory, no NULLs 1.10 284.1±4.58µs ? ?/sec 1.00 258.2±7.35µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, half NULLs 1.04 260.2±1.47µs ? ?/sec 1.00 250.7±4.51µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, no NULLs 1.09 278.3±3.98µs ? ?/sec 1.00 255.4±3.30µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs 1.30 379.6±1.83µs ? ?/sec 1.00 291.9±5.18µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs, short string 1.22 351.8±4.77µs ? ?/sec 1.00 287.3±8.34µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, half NULLs 1.12 325.9±14.21µs ? ?/sec 1.00 290.2±2.31µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, no NULLs 1.29 386.0±4.54µs ? ?/sec 1.00 299.5±2.28µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.00 1034.2±5.50µs ? ?/sec 1.09 1130.5±13.71µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.00 872.6±5.39µs ? ?/sec 1.14 996.5±16.54µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.00 1040.7±6.03µs ? ?/sec 1.10 1139.6±11.84µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs 1.00 412.4±5.34µs ? ?/sec 1.11 459.2±5.53µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs 1.00 572.0±10.37µs ? ?/sec 1.16 661.7±55.57µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs 1.00 420.8±18.03µs ? ?/sec 1.11 467.3±4.35µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, mandatory, no NULLs 1.21 194.5±1.00µs ? ?/sec 1.00 160.7±4.67µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, half NULLs 1.17 335.5±4.93µs ? ?/sec 1.00 287.9±3.88µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, no NULLs 1.20 199.5±1.07µs ? ?/sec 1.00 166.1±5.18µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, mandatory, no NULLs 1.55 117.8±1.09µs ? ?/sec 1.00 76.1±0.99µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, half NULLs 1.22 296.7±2.21µs ? ?/sec 1.00 243.6±0.79µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, no NULLs 1.51 122.6±2.21µs ? ?/sec 1.00 81.1±0.71µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, mandatory, no NULLs 1.00 685.7±10.63µs ? ?/sec 1.08 741.1±12.58µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, half NULLs 1.00 534.0±9.77µs ? ?/sec 1.11 591.7±8.13µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, no NULLs 1.00 691.5±4.59µs ? ?/sec 1.08 748.2±5.96µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, mandatory, no NULLs 1.00 66.2±4.99µs ? ?/sec 1.09 71.9±2.76µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, half NULLs 1.00 224.2±3.09µs ? ?/sec 1.14 255.2±4.19µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, no NULLs 1.00 73.9±4.57µs ? ?/sec 1.05 77.6±2.66µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, mandatory, no NULLs 1.00 85.5±0.72µs ? ?/sec 1.10 93.9±0.48µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, half NULLs 1.04 227.3±3.76µs ? ?/sec 1.00 219.2±2.63µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, no NULLs 1.00 91.6±0.89µs ? ?/sec 1.08 99.2±1.91µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, mandatory, no NULLs 1.00 9.1±0.26µs ? ?/sec 1.04 9.4±0.32µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, half NULLs 1.09 191.3±19.55µs ? ?/sec 1.00 175.9±0.69µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, no NULLs 1.00 14.0±0.25µs ? ?/sec 1.03 14.4±0.38µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, mandatory, no NULLs 1.00 170.4±3.53µs ? ?/sec 1.08 184.4±1.76µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, half NULLs 1.00 333.6±9.00µs ? ?/sec 1.02 339.6±1.52µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, no NULLs 1.00 176.6±3.17µs ? ?/sec 1.08 189.9±0.89µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, mandatory, no NULLs 1.03 14.5±0.79µs ? ?/sec 1.00 14.0±0.47µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, half NULLs 1.01 256.4±1.35µs ? ?/sec 1.00 255.0±1.14µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, no NULLs 1.00 20.3±0.82µs ? ?/sec 1.03 20.8±0.98µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, mandatory, no NULLs 1.00 339.3±0.63µs ? ?/sec 1.09 369.3±5.41µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, half NULLs 1.00 347.3±26.87µs ? ?/sec 1.12 390.7±2.80µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, no NULLs 1.00 347.8±6.81µs ? ?/sec 1.08 374.9±1.97µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, mandatory, no NULLs 1.00 25.8±0.44µs ? ?/sec 1.10 28.3±0.64µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, half NULLs 1.00 186.2±1.38µs ? ?/sec 1.19 222.1±1.60µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, no NULLs 1.00 32.3±0.45µs ? ?/sec 1.06 34.3±0.49µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, mandatory, no NULLs 1.00 110.2±3.94µs ? ?/sec 1.00 110.5±0.86µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, half NULLs 1.00 129.4±2.63µs ? ?/sec 1.01 130.4±0.75µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, no NULLs 1.00 112.4±1.39µs ? ?/sec 1.01 113.6±0.45µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, mandatory, no NULLs 1.00 160.1±1.36µs ? ?/sec 1.01 161.8±0.74µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, half NULLs 1.00 221.9±1.62µs ? ?/sec 1.01 223.2±1.48µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, no NULLs 1.00 165.2±1.13µs ? ?/sec 1.01 166.5±2.16µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.00 76.7±0.36µs ? ?/sec 1.01 77.7±0.29µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.00 177.7±1.48µs ? ?/sec 1.01 178.8±1.39µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.00 81.6±0.30µs ? ?/sec 1.01 82.7±1.20µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, mandatory, no NULLs 1.00 136.2±0.79µs ? ?/sec 1.00 136.4±0.76µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, half NULLs 1.00 210.7±2.32µs ? ?/sec 1.01 212.0±3.06µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, no NULLs 1.00 141.5±4.02µs ? ?/sec 1.00 141.8±1.78µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, mandatory, no NULLs 1.00 73.7±0.60µs ? ?/sec 1.02 74.8±1.09µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, half NULLs 1.00 175.6±2.92µs ? ?/sec 1.01 178.0±1.38µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, no NULLs 1.00 76.8±1.22µs ? ?/sec 1.05 80.7±0.86µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, mandatory, no NULLs 1.00 107.0±0.43µs ? ?/sec 1.01 107.9±1.21µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, half NULLs 1.05 135.2±1.61µs ? ?/sec 1.00 129.2±2.02µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, no NULLs 1.02 112.0±0.46µs ? ?/sec 1.00 110.1±0.47µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, mandatory, no NULLs 1.00 163.2±2.17µs ? ?/sec 1.00 162.8±1.26µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, half NULLs 1.01 233.8±1.46µs ? ?/sec 1.00 230.4±1.95µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, no NULLs 1.01 168.8±4.98µs ? ?/sec 1.00 167.9±1.18µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.00 201.0±1.45µs ? ?/sec 1.01 203.7±1.80µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.01 251.2±4.71µs ? ?/sec 1.00 248.2±8.19µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.00 207.1±1.12µs ? ?/sec 1.01 209.8±0.72µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, mandatory, no NULLs 1.00 150.9±1.35µs ? ?/sec 1.00 150.7±0.59µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, half NULLs 1.00 222.0±5.05µs ? ?/sec 1.02 226.1±1.39µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, no NULLs 1.00 156.7±3.05µs ? ?/sec 1.01 158.0±1.05µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, mandatory, no NULLs 1.00 106.6±1.10µs ? ?/sec 1.00 106.6±1.06µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, half NULLs 1.03 199.0±1.74µs ? ?/sec 1.00 193.3±2.07µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, no NULLs 1.03 117.4±2.34µs ? ?/sec 1.00 114.5±1.26µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, mandatory, no NULLs 1.01 77.8±1.05µs ? ?/sec 1.00 77.3±1.10µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, half NULLs 1.00 102.1±0.91µs ? ?/sec 1.01 103.3±1.28µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, no NULLs 1.01 80.4±0.65µs ? ?/sec 1.00 79.8±0.29µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, mandatory, no NULLs 1.00 108.2±1.52µs ? ?/sec 1.00 107.8±2.49µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, half NULLs 1.00 172.3±3.76µs ? ?/sec 1.02 175.1±6.49µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, no NULLs 1.00 112.1±1.24µs ? ?/sec 1.00 111.8±0.46µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, mandatory, no NULLs 1.00 43.5±0.92µs ? ?/sec 1.01 44.0±0.16µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, half NULLs 1.00 138.8±1.71µs ? ?/sec 1.01 140.8±3.46µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, no NULLs 1.00 47.3±0.48µs ? ?/sec 1.02 48.1±0.71µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, mandatory, no NULLs 1.00 101.5±0.85µs ? ?/sec 1.01 102.4±0.52µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, half NULLs 1.00 172.9±2.77µs ? ?/sec 1.01 174.1±0.99µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, no NULLs 1.00 106.6±2.67µs ? ?/sec 1.01 107.4±1.19µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, mandatory, no NULLs 1.00 37.8±0.35µs ? ?/sec 1.00 38.0±0.26µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, half NULLs 1.00 136.9±1.22µs ? ?/sec 1.01 138.3±0.84µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, no NULLs 1.00 42.2±0.52µs ? ?/sec 1.01 42.4±0.42µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, mandatory, no NULLs 1.00 84.1±2.07µs ? ?/sec 1.04 87.4±1.11µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, half NULLs 1.00 101.1±1.02µs ? ?/sec 1.02 103.1±1.03µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, no NULLs 1.00 86.7±1.51µs ? ?/sec 1.02 88.1±1.80µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs 1.00 109.1±0.95µs ? ?/sec 1.02 110.8±0.91µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, half NULLs 1.00 166.1±4.78µs ? ?/sec 1.01 168.4±2.38µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, no NULLs 1.00 113.9±1.21µs ? ?/sec 1.02 115.8±3.88µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, mandatory, no NULLs 1.00 26.1±0.42µs ? ?/sec 1.01 26.4±0.65µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, half NULLs 1.00 120.7±1.16µs ? ?/sec 1.02 123.5±2.40µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, no NULLs 1.00 30.0±0.38µs ? ?/sec 1.02 30.7±0.45µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs 1.00 85.3±0.38µs ? ?/sec 1.00 85.3±0.57µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs 1.00 154.9±0.75µs ? ?/sec 1.01 156.7±0.62µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs 1.00 90.3±1.24µs ? ?/sec 1.00 90.1±1.18µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs 1.01 18.4±0.81µs ? ?/sec 1.00 18.1±0.61µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs 1.00 118.3±1.09µs ? ?/sec 1.02 120.9±0.75µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs 1.01 24.3±0.85µs ? ?/sec 1.00 24.1±0.49µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, mandatory, no NULLs 1.02 82.7±0.83µs ? ?/sec 1.00 81.5±1.58µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, half NULLs 1.01 102.8±1.40µs ? ?/sec 1.00 102.2±0.77µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, no NULLs 1.00 84.1±2.19µs ? ?/sec 1.00 83.9±0.39µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs 1.01 109.7±0.88µs ? ?/sec 1.00 108.4±0.88µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, half NULLs 1.00 168.4±0.89µs ? ?/sec 1.03 173.0±1.82µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, no NULLs 1.00 111.8±0.80µs ? ?/sec 1.03 115.6±0.93µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, mandatory, no NULLs 1.01 150.8±0.75µs ? ?/sec 1.00 150.0±1.09µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, half NULLs 1.00 192.9±4.29µs ? ?/sec 1.00 192.2±2.52µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, no NULLs 1.01 155.5±0.91µs ? ?/sec 1.00 154.7±0.88µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs 1.00 99.6±1.20µs ? ?/sec 1.01 100.3±1.27µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs 1.05 171.8±3.68µs ? ?/sec 1.00 163.0±0.92µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs 1.00 104.1±1.12µs ? ?/sec 1.00 104.6±0.59µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs 1.00 40.5±1.14µs ? ?/sec 1.11 45.0±2.78µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs 1.01 134.6±1.14µs ? ?/sec 1.00 132.9±4.61µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs 1.00 45.3±1.72µs ? ?/sec 1.15 52.3±2.55µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, mandatory, no NULLs 1.00 81.1±0.48µs ? ?/sec 1.00 81.5±0.35µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, half NULLs 1.00 102.4±4.58µs ? ?/sec 1.01 102.9±0.36µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, no NULLs 1.00 83.5±0.27µs ? ?/sec 1.01 84.6±0.84µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, mandatory, no NULLs 1.00 109.1±0.94µs ? ?/sec 1.01 109.9±3.49µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, half NULLs 1.00 169.2±3.02µs ? ?/sec 1.02 171.9±2.40µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, no NULLs 1.00 112.9±1.41µs ? ?/sec 1.01 114.2±3.12µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, mandatory, no NULLs 1.00 36.0±0.19µs ? ?/sec 1.01 36.3±0.58µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, half NULLs 1.00 131.1±1.76µs ? ?/sec 1.02 134.2±1.09µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, no NULLs 1.03 39.8±0.33µs ? ?/sec 1.00 38.7±0.64µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, mandatory, no NULLs 1.00 94.8±0.88µs ? ?/sec 1.00 94.7±0.49µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, half NULLs 1.00 165.5±3.69µs ? ?/sec 1.01 167.5±1.75µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, no NULLs 1.00 99.3±0.42µs ? ?/sec 1.00 99.6±1.00µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, mandatory, no NULLs 1.00 30.0±0.32µs ? ?/sec 1.00 30.0±0.29µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, half NULLs 1.00 129.0±2.17µs ? ?/sec 1.01 130.0±0.64µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, no NULLs 1.00 34.1±0.15µs ? ?/sec 1.01 34.6±0.41µs ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings half NULLs 1.00 7.3±0.03ms ? ?/sec 1.01 7.4±0.06ms ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings no NULLs 1.00 13.0±0.14ms ? ?/sec 1.00 13.0±0.16ms ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs 1.00 510.7±9.84µs ? ?/sec 1.01 517.7±19.38µs ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs 1.02 689.3±3.55µs ? ?/sec 1.00 674.8±9.63µs ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs 1.00 510.5±3.10µs ? ?/sec 1.02 522.1±10.93µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs 1.13 718.9±7.70µs ? ?/sec 1.00 635.5±6.29µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, half NULLs 1.06 826.0±7.64µs ? ?/sec 1.00 778.0±6.97µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, no NULLs 1.13 725.8±5.20µs ? ?/sec 1.00 644.4±6.97µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs 1.00 336.1±10.17µs ? ?/sec 1.00 336.0±4.65µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs 1.00 404.9±6.34µs ? ?/sec 1.03 416.8±2.47µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs 1.00 340.9±1.94µs ? ?/sec 1.01 344.2±3.02µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, mandatory, no NULLs 1.12 280.5±2.63µs ? ?/sec 1.00 251.4±7.20µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, half NULLs 1.03 260.9±3.91µs ? ?/sec 1.00 252.7±1.62µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, no NULLs 1.07 277.2±8.15µs ? ?/sec 1.00 258.1±3.42µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, mandatory, no NULLs 1.08 500.9±4.31µs ? ?/sec 1.00 465.3±2.41µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, half NULLs 1.03 385.5±5.15µs ? ?/sec 1.00 372.8±2.47µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, no NULLs 1.07 507.5±2.91µs ? ?/sec 1.00 474.6±3.36µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, mandatory, no NULLs 1.00 92.8±2.06µs ? ?/sec 1.00 93.0±0.83µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, half NULLs 1.00 110.6±0.34µs ? ?/sec 1.01 111.7±0.48µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, no NULLs 1.00 95.0±0.42µs ? ?/sec 1.00 95.2±1.76µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, mandatory, no NULLs 1.00 126.6±0.95µs ? ?/sec 1.00 127.2±0.73µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, half NULLs 1.00 183.4±2.69µs ? ?/sec 1.02 186.4±2.30µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, no NULLs 1.00 130.8±0.46µs ? ?/sec 1.01 131.7±2.87µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, mandatory, no NULLs 1.05 43.7±0.31µs ? ?/sec 1.00 41.7±0.19µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, half NULLs 1.00 138.1±2.42µs ? ?/sec 1.02 140.7±1.02µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, no NULLs 1.03 47.6±0.31µs ? ?/sec 1.00 46.2±0.25µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, mandatory, no NULLs 1.00 101.9±1.54µs ? ?/sec 1.00 102.2±1.95µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, half NULLs 1.00 173.2±2.73µs ? ?/sec 1.00 174.0±1.75µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, no NULLs 1.00 106.1±1.04µs ? ?/sec 1.01 107.3±0.95µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, mandatory, no NULLs 1.01 38.2±0.28µs ? ?/sec 1.00 37.9±0.30µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, half NULLs 1.00 136.4±3.53µs ? ?/sec 1.02 139.4±1.41µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, no NULLs 1.00 42.4±0.34µs ? ?/sec 1.00 42.6±0.12µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, mandatory, no NULLs 1.00 83.8±0.73µs ? ?/sec 1.02 85.7±4.55µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, half NULLs 1.00 101.0±0.57µs ? ?/sec 1.02 103.3±2.49µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, no NULLs 1.00 86.3±1.72µs ? ?/sec 1.01 87.4±0.58µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, mandatory, no NULLs 1.00 110.3±3.10µs ? ?/sec 1.01 111.7±0.85µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, half NULLs 1.00 164.8±1.48µs ? ?/sec 1.02 168.4±2.54µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, no NULLs 1.00 113.1±1.48µs ? ?/sec 1.03 116.1±1.63µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, mandatory, no NULLs 1.00 26.4±0.36µs ? ?/sec 1.02 27.0±0.31µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, half NULLs 1.00 121.7±0.48µs ? ?/sec 1.01 123.1±0.78µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, no NULLs 1.00 30.6±0.47µs ? ?/sec 1.02 31.1±1.15µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, mandatory, no NULLs 1.00 85.5±2.07µs ? ?/sec 1.00 85.9±2.06µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, half NULLs 1.00 154.4±1.05µs ? ?/sec 1.01 156.3±1.32µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, no NULLs 1.00 89.7±0.48µs ? ?/sec 1.00 90.0±0.99µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, mandatory, no NULLs 1.00 20.6±0.81µs ? ?/sec 1.02 20.9±0.78µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, half NULLs 1.00 118.9±0.62µs ? ?/sec 1.01 120.2±1.28µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, no NULLs 1.00 25.4±1.29µs ? ?/sec 1.02 25.9±0.83µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, mandatory, no NULLs 1.02 82.4±0.65µs ? ?/sec 1.00 81.1±0.63µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, half NULLs 1.00 102.2±1.43µs ? ?/sec 1.01 103.5±5.76µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, no NULLs 1.00 84.1±3.56µs ? ?/sec 1.00 84.0±1.78µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, mandatory, no NULLs 1.00 109.2±1.99µs ? ?/sec 1.00 108.9±0.95µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, half NULLs 1.00 168.1±3.73µs ? ?/sec 1.04 175.3±1.59µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, no NULLs 1.00 113.9±4.45µs ? ?/sec 1.02 115.8±1.31µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, mandatory, no NULLs 1.00 149.2±0.54µs ? ?/sec 1.01 150.9±0.68µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, half NULLs 1.02 195.1±13.98µs ? ?/sec 1.00 191.3±1.27µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, no NULLs 1.00 153.9±0.56µs ? ?/sec 1.01 155.5±0.62µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, mandatory, no NULLs 1.00 99.4±3.77µs ? ?/sec 1.00 99.9±1.04µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, half NULLs 1.00 164.3±5.04µs ? ?/sec 1.04 170.2±0.86µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, no NULLs 1.00 103.8±0.63µs ? ?/sec 1.01 104.4±0.68µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, mandatory, no NULLs 1.00 41.3±1.24µs ? ?/sec 1.07 44.3±2.44µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, half NULLs 1.00 134.3±4.46µs ? ?/sec 1.00 134.4±0.78µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, no NULLs 1.00 45.4±1.30µs ? ?/sec 1.25 56.7±3.01µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, mandatory, no NULLs 1.01 89.8±3.02µs ? ?/sec 1.00 88.9±3.35µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, half NULLs 1.00 106.2±1.55µs ? ?/sec 1.00 106.6±0.71µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, no NULLs 1.01 92.3±2.24µs ? ?/sec 1.00 91.0±0.75µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, mandatory, no NULLs 1.01 119.3±5.42µs ? ?/sec 1.00 118.4±0.36µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, half NULLs 1.00 175.2±2.23µs ? ?/sec 1.01 176.4±1.35µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, no NULLs 1.01 123.6±2.00µs ? ?/sec 1.00 122.8±0.65µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, mandatory, no NULLs 1.00 35.8±0.40µs ? ?/sec 1.00 35.7±0.17µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, half NULLs 1.00 130.6±0.80µs ? ?/sec 1.02 132.9±1.38µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, no NULLs 1.00 39.6±0.35µs ? ?/sec 1.01 40.0±0.24µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, mandatory, no NULLs 1.00 94.2±0.67µs ? ?/sec 1.01 94.6±0.47µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, half NULLs 1.00 165.1±1.98µs ? ?/sec 1.01 167.5±2.75µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, no NULLs 1.00 99.3±0.46µs ? ?/sec 1.00 99.2±1.11µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, mandatory, no NULLs 1.02 30.5±1.75µs ? ?/sec 1.00 30.0±0.18µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, half NULLs 1.00 128.3±1.74µs ? ?/sec 1.01 129.9±0.72µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, no NULLs 1.00 34.0±0.18µs ? ?/sec 1.02 34.7±0.26µs ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#discussion_r2686443060
##########
arrow-array/src/array/byte_view_array.rs:
##########
@@ -943,15 +943,15 @@ impl<'a, T: ByteViewType + ?Sized> IntoIterator for &'a GenericByteViewArray<T>
}
impl<T: ByteViewType + ?Sized> From<ArrayData> for GenericByteViewArray<T> {
- fn from(value: ArrayData) -> Self {
- let views = value.buffers()[0].clone();
- let views = ScalarBuffer::new(views, value.offset(), value.len());
- let buffers = value.buffers()[1..].to_vec();
+ fn from(data: ArrayData) -> Self {
+ let (_data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
Review Comment:
Will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3755018917
I split the rest of this PR up into smaller PRs for review
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
- https://github.com/apache/arrow-rs/pull/9185
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3779314510
This PR has now been merged as individual parts, so closing this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3715131251
> My 2 cents is it would be better to move the codepaths relying on ArrayData over to using the typed arrays directly, this should not only cut down on allocations but unnecessary validation and dispatch overheads.
This is a good idea -- I will see what I can do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697853251
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group alamb_reduce_struct_array_clones main
----- -------------------------------- ----
arrow_reader_row_filter/float64 <= 99.0/all_columns/async 1.00 1709.3±14.61µs ? ?/sec 1.00 1711.1±17.26µs ? ?/sec
arrow_reader_row_filter/float64 <= 99.0/all_columns/sync 1.00 1835.0±23.13µs ? ?/sec 1.00 1844.1±13.83µs ? ?/sec
arrow_reader_row_filter/float64 <= 99.0/exclude_filter_column/async 1.00 1569.6±16.21µs ? ?/sec 1.01 1582.1±26.31µs ? ?/sec
arrow_reader_row_filter/float64 <= 99.0/exclude_filter_column/sync 1.00 1550.3±24.80µs ? ?/sec 1.01 1562.7±20.42µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0 AND ts >= 9000/all_columns/async 1.00 1545.4±16.51µs ? ?/sec 1.00 1551.2±13.68µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0 AND ts >= 9000/all_columns/sync 1.01 1710.3±17.91µs ? ?/sec 1.00 1701.3±19.79µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0 AND ts >= 9000/exclude_filter_column/async 1.02 1386.4±10.30µs ? ?/sec 1.00 1354.7±15.64µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0 AND ts >= 9000/exclude_filter_column/sync 1.02 1388.9±14.54µs ? ?/sec 1.00 1364.7±15.36µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0/all_columns/async 1.00 1717.2±13.33µs ? ?/sec 1.00 1723.0±42.90µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0/all_columns/sync 1.01 1845.7±13.37µs ? ?/sec 1.00 1836.4±24.69µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0/exclude_filter_column/async 1.00 1574.2±49.42µs ? ?/sec 1.00 1581.9±16.13µs ? ?/sec
arrow_reader_row_filter/float64 > 99.0/exclude_filter_column/sync 1.00 1553.4±33.36µs ? ?/sec 1.00 1559.1±9.53µs ? ?/sec
arrow_reader_row_filter/int64 == 9999/all_columns/async 1.00 940.3±8.72µs ? ?/sec 1.00 936.4±10.89µs ? ?/sec
arrow_reader_row_filter/int64 == 9999/all_columns/sync 1.02 886.0±8.24µs ? ?/sec 1.00 869.9±8.59µs ? ?/sec
arrow_reader_row_filter/int64 == 9999/exclude_filter_column/async 1.02 868.2±8.41µs ? ?/sec 1.00 855.0±13.60µs ? ?/sec
arrow_reader_row_filter/int64 == 9999/exclude_filter_column/sync 1.02 880.0±17.67µs ? ?/sec 1.00 864.3±18.83µs ? ?/sec
arrow_reader_row_filter/int64 > 90/all_columns/async 1.04 2.9±0.04ms ? ?/sec 1.00 2.8±0.09ms ? ?/sec
arrow_reader_row_filter/int64 > 90/all_columns/sync 1.00 2.6±0.05ms ? ?/sec 1.41 3.6±0.06ms ? ?/sec
arrow_reader_row_filter/int64 > 90/exclude_filter_column/async 1.00 2.6±0.03ms ? ?/sec 1.02 2.7±0.04ms ? ?/sec
arrow_reader_row_filter/int64 > 90/exclude_filter_column/sync 1.00 2.3±0.02ms ? ?/sec 1.01 2.3±0.03ms ? ?/sec
arrow_reader_row_filter/ts < 9000/all_columns/async 1.01 2.0±0.03ms ? ?/sec 1.00 2.0±0.01ms ? ?/sec
arrow_reader_row_filter/ts < 9000/all_columns/sync 1.02 2.1±0.02ms ? ?/sec 1.00 2.1±0.02ms ? ?/sec
arrow_reader_row_filter/ts < 9000/exclude_filter_column/async 1.04 1871.8±27.20µs ? ?/sec 1.00 1808.2±17.23µs ? ?/sec
arrow_reader_row_filter/ts < 9000/exclude_filter_column/sync 1.02 1868.2±13.03µs ? ?/sec 1.00 1826.1±38.14µs ? ?/sec
arrow_reader_row_filter/ts >= 9000/all_columns/async 1.00 1279.0±17.58µs ? ?/sec 1.00 1278.4±19.48µs ? ?/sec
arrow_reader_row_filter/ts >= 9000/all_columns/sync 1.00 1288.9±13.89µs ? ?/sec 1.00 1294.5±15.08µs ? ?/sec
arrow_reader_row_filter/ts >= 9000/exclude_filter_column/async 1.00 1158.6±13.75µs ? ?/sec 1.00 1154.8±8.17µs ? ?/sec
arrow_reader_row_filter/ts >= 9000/exclude_filter_column/sync 1.00 1160.2±16.30µs ? ?/sec 1.01 1166.7±26.41µs ? ?/sec
arrow_reader_row_filter/utf8View <> ''/all_columns/async 1.02 3.3±0.05ms ? ?/sec 1.00 3.3±0.03ms ? ?/sec
arrow_reader_row_filter/utf8View <> ''/all_columns/sync 1.02 3.7±0.05ms ? ?/sec 1.00 3.7±0.10ms ? ?/sec
arrow_reader_row_filter/utf8View <> ''/exclude_filter_column/async 1.04 2.9±0.02ms ? ?/sec 1.00 2.8±0.05ms ? ?/sec
arrow_reader_row_filter/utf8View <> ''/exclude_filter_column/sync 1.04 2.7±0.05ms ? ?/sec 1.00 2.5±0.02ms ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697794718
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/reduce_struct_array_clones (f4fcd774ff48756d7536699c44eb0e16d1aa1140) to 814ee4227c01fce478bdd3594dd156250286b46e [diff](https://github.com/apache/arrow-rs/compare/814ee4227c01fce478bdd3594dd156250286b46e..f4fcd774ff48756d7536699c44eb0e16d1aa1140)
BENCH_NAME=arrow_reader_clickbench
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_clickbench
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_reduce_struct_array_clones
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#discussion_r2651798669
##########
arrow-array/src/array/byte_view_array.rs:
##########
@@ -941,15 +941,15 @@ impl<'a, T: ByteViewType + ?Sized> IntoIterator for &'a GenericByteViewArray<T>
}
impl<T: ByteViewType + ?Sized> From<ArrayData> for GenericByteViewArray<T> {
- fn from(value: ArrayData) -> Self {
- let views = value.buffers()[0].clone();
- let views = ScalarBuffer::new(views, value.offset(), value.len());
- let buffers = value.buffers()[1..].to_vec();
Review Comment:
this call to `to_vec()` creates a new vec (aka is a new allocation) while using the new API reuses the existing allocation
I am pretty sure I was seeing this allocation show up in my profiling of the parquet reader for clickbench
##########
arrow-array/src/array/boolean_array.rs:
##########
@@ -387,24 +387,21 @@ impl From<Vec<Option<bool>>> for BooleanArray {
impl From<ArrayData> for BooleanArray {
fn from(data: ArrayData) -> Self {
+ let (data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
assert_eq!(
- data.data_type(),
- &DataType::Boolean,
- "BooleanArray expected ArrayData with type {} got {}",
+ data_type,
DataType::Boolean,
- data.data_type()
+ "BooleanArray expected ArrayData with type Boolean got {data_type:?}",
);
assert_eq!(
- data.buffers().len(),
+ buffers.len(),
1,
"BooleanArray data should contain a single buffer only (values buffer)"
);
- let values = BooleanBuffer::new(data.buffers()[0].clone(), data.offset(), data.len());
+ let buffer = buffers.pop().expect("checked above");
Review Comment:
this basically shows the new pattern -- rather than cloning parts of `ArrayData`, instead simply get the relevant structures that are needed directly as it already gets an owned `ArrayData`
For BooleanArray this probably saves some `Arc::clone`s which isn't a big deal. For more complex types like StructArray and GenericByteViewArray it also saves some Vec allocations which I do think is a bigger deal
##########
arrow-array/src/array/dictionary_array.rs:
##########
@@ -603,17 +606,17 @@ impl<T: ArrowDictionaryKeyType> From<ArrayData> for DictionaryArray<T> {
key_data_type
);
- let values = make_array(data.child_data()[0].clone());
Review Comment:
here is an example where the entire child_data `ArrayData` was cloned (this potentially includes two `Vec`s )
##########
arrow-array/src/array/list_array.rs:
##########
@@ -479,23 +479,26 @@ impl<OffsetSize: OffsetSizeTrait> From<FixedSizeListArray> for GenericListArray<
impl<OffsetSize: OffsetSizeTrait> GenericListArray<OffsetSize> {
fn try_new_from_array_data(data: ArrayData) -> Result<Self, ArrowError> {
- if data.buffers().len() != 1 {
+ let (data_type, len, nulls, offset, mut buffers, mut child_data) = data.into_parts();
+
+ if buffers.len() != 1 {
return Err(ArrowError::InvalidArgumentError(format!(
"ListArray data should contain a single buffer only (value offsets), had {}",
- data.buffers().len()
+ buffers.len()
)));
}
+ let buffer = buffers.pop().expect("checked above");
- if data.child_data().len() != 1 {
+ if child_data.len() != 1 {
return Err(ArrowError::InvalidArgumentError(format!(
"ListArray should contain a single child array (values array), had {}",
- data.child_data().len()
+ child_data.len()
)));
}
- let values = data.child_data()[0].clone();
Review Comment:
here is another example of not having to clone (and allocate `Vec`s) an entire `ArrayData`
##########
arrow-data/src/data.rs:
##########
@@ -351,6 +354,33 @@ impl ArrayData {
Ok(new_self)
}
+ /// Return the constituent parts of this ArrayData
Review Comment:
Here is the key new API in ArrayData to return all its fields
##########
arrow-array/src/array/struct_array.rs:
##########
@@ -347,25 +347,26 @@ impl StructArray {
impl From<ArrayData> for StructArray {
fn from(data: ArrayData) -> Self {
- let parent_offset = data.offset();
- let parent_len = data.len();
+ let (data_type, len, nulls, offset, _buffers, child_data) = data.into_parts();
- let fields = data
- .child_data()
- .iter()
+ let parent_offset = offset;
+ let parent_len = len;
+
+ let fields = child_data
+ .into_iter()
.map(|cd| {
if parent_offset != 0 || parent_len != cd.len() {
make_array(cd.slice(parent_offset, parent_len))
} else {
- make_array(cd.clone())
Review Comment:
this clone in particular is called for every batch created by the Parquet reader -- and each field (aka parquet column) likely has at least one Vec (the buffers)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697825821
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group alamb_reduce_struct_array_clones main
----- -------------------------------- ----
arrow_reader_clickbench/async/Q1 1.00 2.3±0.03ms ? ?/sec 1.00 2.3±0.04ms ? ?/sec
arrow_reader_clickbench/async/Q10 1.00 13.2±0.25ms ? ?/sec 1.00 13.2±0.29ms ? ?/sec
arrow_reader_clickbench/async/Q11 1.01 15.0±0.42ms ? ?/sec 1.00 14.9±0.38ms ? ?/sec
arrow_reader_clickbench/async/Q12 1.01 26.2±0.47ms ? ?/sec 1.00 25.9±0.29ms ? ?/sec
arrow_reader_clickbench/async/Q13 1.00 30.9±0.31ms ? ?/sec 1.01 31.2±0.28ms ? ?/sec
arrow_reader_clickbench/async/Q14 1.01 28.7±0.32ms ? ?/sec 1.00 28.5±0.31ms ? ?/sec
arrow_reader_clickbench/async/Q19 1.00 5.3±0.06ms ? ?/sec 1.01 5.3±0.07ms ? ?/sec
arrow_reader_clickbench/async/Q20 1.00 112.7±1.18ms ? ?/sec 1.00 112.6±0.87ms ? ?/sec
arrow_reader_clickbench/async/Q21 1.00 130.2±0.57ms ? ?/sec 1.00 130.5±1.69ms ? ?/sec
arrow_reader_clickbench/async/Q22 1.00 279.2±5.31ms ? ?/sec 1.01 282.5±9.45ms ? ?/sec
arrow_reader_clickbench/async/Q23 1.01 406.0±10.70ms ? ?/sec 1.00 403.0±2.77ms ? ?/sec
arrow_reader_clickbench/async/Q24 1.00 33.9±0.70ms ? ?/sec 1.03 34.8±0.89ms ? ?/sec
arrow_reader_clickbench/async/Q27 1.00 97.7±0.48ms ? ?/sec 1.01 98.9±0.70ms ? ?/sec
arrow_reader_clickbench/async/Q28 1.00 96.5±0.88ms ? ?/sec 1.01 97.7±0.60ms ? ?/sec
arrow_reader_clickbench/async/Q30 1.00 30.9±0.29ms ? ?/sec 1.01 31.1±0.38ms ? ?/sec
arrow_reader_clickbench/async/Q36 1.00 106.7±0.58ms ? ?/sec 1.01 107.7±1.21ms ? ?/sec
arrow_reader_clickbench/async/Q37 1.00 84.7±1.80ms ? ?/sec 1.00 84.7±0.36ms ? ?/sec
arrow_reader_clickbench/async/Q38 1.00 30.9±0.38ms ? ?/sec 1.02 31.6±0.81ms ? ?/sec
arrow_reader_clickbench/async/Q39 1.00 43.9±0.52ms ? ?/sec 1.01 44.2±0.49ms ? ?/sec
arrow_reader_clickbench/async/Q40 1.00 25.9±0.43ms ? ?/sec 1.01 26.1±0.43ms ? ?/sec
arrow_reader_clickbench/async/Q41 1.00 20.6±0.48ms ? ?/sec 1.02 20.9±0.53ms ? ?/sec
arrow_reader_clickbench/async/Q42 1.00 9.9±0.13ms ? ?/sec 1.01 10.0±0.14ms ? ?/sec
arrow_reader_clickbench/sync/Q1 1.00 2.1±0.02ms ? ?/sec 1.01 2.1±0.02ms ? ?/sec
arrow_reader_clickbench/sync/Q10 1.00 10.1±0.24ms ? ?/sec 1.00 10.2±0.12ms ? ?/sec
arrow_reader_clickbench/sync/Q11 1.00 11.6±0.10ms ? ?/sec 1.02 11.8±0.09ms ? ?/sec
arrow_reader_clickbench/sync/Q12 1.00 32.7±0.51ms ? ?/sec 1.08 35.4±1.36ms ? ?/sec
arrow_reader_clickbench/sync/Q13 1.01 46.9±0.66ms ? ?/sec 1.00 46.3±1.63ms ? ?/sec
arrow_reader_clickbench/sync/Q14 1.00 41.8±1.14ms ? ?/sec 1.00 41.7±1.13ms ? ?/sec
arrow_reader_clickbench/sync/Q19 1.00 4.3±0.03ms ? ?/sec 1.01 4.3±0.06ms ? ?/sec
arrow_reader_clickbench/sync/Q20 1.00 173.6±1.72ms ? ?/sec 1.01 174.8±1.42ms ? ?/sec
arrow_reader_clickbench/sync/Q21 1.00 231.9±1.76ms ? ?/sec 1.00 232.1±1.61ms ? ?/sec
arrow_reader_clickbench/sync/Q22 1.00 469.5±3.72ms ? ?/sec 1.01 473.0±4.50ms ? ?/sec
arrow_reader_clickbench/sync/Q23 1.00 420.4±10.26ms ? ?/sec 1.01 423.4±13.29ms ? ?/sec
arrow_reader_clickbench/sync/Q24 1.00 44.8±1.02ms ? ?/sec 1.00 44.9±0.72ms ? ?/sec
arrow_reader_clickbench/sync/Q27 1.00 148.7±0.96ms ? ?/sec 1.01 150.7±0.98ms ? ?/sec
arrow_reader_clickbench/sync/Q28 1.00 143.5±1.34ms ? ?/sec 1.02 146.2±1.35ms ? ?/sec
arrow_reader_clickbench/sync/Q30 1.00 30.0±0.69ms ? ?/sec 1.05 31.3±0.59ms ? ?/sec
arrow_reader_clickbench/sync/Q36 1.00 138.0±1.58ms ? ?/sec 1.01 139.6±1.39ms ? ?/sec
arrow_reader_clickbench/sync/Q37 1.00 77.4±1.30ms ? ?/sec 1.01 77.9±1.02ms ? ?/sec
arrow_reader_clickbench/sync/Q38 1.01 24.4±0.38ms ? ?/sec 1.00 24.2±0.42ms ? ?/sec
arrow_reader_clickbench/sync/Q39 1.00 31.3±0.49ms ? ?/sec 1.00 31.3±0.22ms ? ?/sec
arrow_reader_clickbench/sync/Q40 1.00 23.8±0.23ms ? ?/sec 1.00 23.8±0.23ms ? ?/sec
arrow_reader_clickbench/sync/Q41 1.00 19.2±0.62ms ? ?/sec 1.00 19.2±0.21ms ? ?/sec
arrow_reader_clickbench/sync/Q42 1.01 9.3±0.08ms ? ?/sec 1.00 9.2±0.16ms ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697825899
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/reduce_struct_array_clones (f4fcd774ff48756d7536699c44eb0e16d1aa1140) to 814ee4227c01fce478bdd3594dd156250286b46e [diff](https://github.com/apache/arrow-rs/compare/814ee4227c01fce478bdd3594dd156250286b46e..f4fcd774ff48756d7536699c44eb0e16d1aa1140)
BENCH_NAME=arrow_reader_row_filter
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_row_filter
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_reduce_struct_array_clones
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3697643170
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/reduce_struct_array_clones (f4fcd774ff48756d7536699c44eb0e16d1aa1140) to 814ee4227c01fce478bdd3594dd156250286b46e [diff](https://github.com/apache/arrow-rs/compare/814ee4227c01fce478bdd3594dd156250286b46e..f4fcd774ff48756d7536699c44eb0e16d1aa1140)
BENCH_NAME=arrow_reader
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_reduce_struct_array_clones
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3699144754
> If the benchmarks look good I will split this PR up into some smaller pieces for easier review
The benchmarks don't show much of an improvement -- I could make a micro benchmark, but I am not sure how useful that would be
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid clones while creating Arrays from `ArrayData` (speed up reading from Parquet reader) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9058:
URL: https://github.com/apache/arrow-rs/pull/9058#issuecomment-3699155553
So TLDR is I will break this up into smaller PRs and start getting it reviewed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org