You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "lyang24 (via GitHub)" <gi...@apache.org> on 2026/01/27 19:57:09 UTC
[PR] reduce clone in delta byte array decoder [arrow-rs]
lyang24 opened a new pull request, #9282:
URL: https://github.com/apache/arrow-rs/pull/9282
# Which issue does this PR close?
small optimization
# Rationale for this change
key insight is the byte clone is cheap just a ref count compare to vec clone is a alloc + memcopy.
before
```
let mut result = Vec::new(); // alloc #1
result.extend_from_slice(prefix);
result.extend_from_slice(suffix);
let data = Bytes::from(result.clone()); // alloc #2 + memcpy
item.set_from_bytes(data);
self.previous_value = result; // keep Vec
```
after
```
let mut result = Vec::with_capacity(prefix_len + suffix.len()); // alloc #1
result.extend_from_slice(&self.previous_value[..prefix_len]);
result.extend_from_slice(suffix);
let data = Bytes::from(result); // no alloc, takes Vec buffer
item.set_from_bytes(data.clone()); // cheap refcount bump
self.previous_value = data; // move, no alloc
```
# What changes are included in this PR?
previous_value type changed to Bytes
preallocate result vec capacity.
# Are these changes tested?
the existing test should pass
# Are there any user-facing changes?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282#issuecomment-3809950401
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing reduce_decoder_clone (0719229475279e0cc017ffa56b0e88bd3d07354b) to 3c6ca57f9824e4eca20f52c81bda40d3d2dcd5f3 [diff](https://github.com/apache/arrow-rs/compare/3c6ca57f9824e4eca20f52c81bda40d3d2dcd5f3..0719229475279e0cc017ffa56b0e88bd3d07354b)
BENCH_NAME=arrow_reader
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_reader
BENCH_FILTER=
BENCH_BRANCH_NAME=reduce_decoder_clone
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan merged PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282#issuecomment-3828403403
Thanks @lyang24 !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282#issuecomment-3810416375
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group main reduce_decoder_clone
----- ---- --------------------
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs 1.00 1178.2±7.43µs ? ?/sec 1.00 1183.6±9.45µs ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs 1.00 1249.5±4.90µs ? ?/sec 1.00 1244.8±18.40µs ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs 1.00 1187.6±5.35µs ? ?/sec 1.00 1192.6±6.60µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, mandatory, no NULLs 1.02 484.9±6.69µs ? ?/sec 1.00 474.9±6.36µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, half NULLs 1.00 657.2±2.64µs ? ?/sec 1.00 659.1±5.09µs ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, no NULLs 1.00 490.8±14.01µs ? ?/sec 1.02 499.3±38.52µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, mandatory, no NULLs 1.00 554.6±8.24µs ? ?/sec 1.02 564.7±6.83µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, half NULLs 1.00 728.4±14.17µs ? ?/sec 1.00 728.5±7.22µs ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, no NULLs 1.00 560.3±9.20µs ? ?/sec 1.03 576.4±6.39µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, mandatory, no NULLs 1.00 153.9±2.28µs ? ?/sec 1.00 153.2±2.30µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, half NULLs 1.02 220.5±6.06µs ? ?/sec 1.00 216.2±1.62µs ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, no NULLs 1.03 159.7±2.51µs ? ?/sec 1.00 155.0±2.10µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs 1.00 224.2±3.40µs ? ?/sec 1.00 224.8±5.95µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs, short string 1.00 193.3±2.58µs ? ?/sec 1.01 194.3±1.01µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, half NULLs 1.00 259.3±1.45µs ? ?/sec 1.01 260.9±5.56µs ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, no NULLs 1.00 231.3±3.19µs ? ?/sec 1.00 231.9±3.12µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.00 1090.0±10.06µs ? ?/sec 1.00 1088.1±9.40µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.01 979.0±16.50µs ? ?/sec 1.00 972.1±4.04µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.00 1100.1±35.49µs ? ?/sec 1.00 1095.9±7.65µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs 1.02 468.0±5.22µs ? ?/sec 1.00 458.6±4.39µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs 1.02 666.9±12.10µs ? ?/sec 1.00 654.7±11.18µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs 1.01 477.4±8.80µs ? ?/sec 1.00 472.0±5.75µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, mandatory, no NULLs 1.00 195.5±3.77µs ? ?/sec 1.00 195.3±1.49µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, half NULLs 1.00 340.4±2.01µs ? ?/sec 1.00 340.0±2.15µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, no NULLs 1.00 200.0±0.72µs ? ?/sec 1.00 200.1±1.01µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, mandatory, no NULLs 1.00 118.5±1.15µs ? ?/sec 1.00 118.6±1.95µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, half NULLs 1.00 302.2±2.00µs ? ?/sec 1.00 301.2±1.81µs ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, no NULLs 1.00 124.4±0.96µs ? ?/sec 1.00 124.6±1.68µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, mandatory, no NULLs 1.00 687.3±6.45µs ? ?/sec 1.01 695.5±10.33µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, half NULLs 1.00 584.4±7.38µs ? ?/sec 1.00 581.9±5.66µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, no NULLs 1.00 693.3±3.69µs ? ?/sec 1.01 701.4±4.33µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, mandatory, no NULLs 1.00 65.8±7.16µs ? ?/sec 1.09 71.6±3.01µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, half NULLs 1.01 272.4±2.62µs ? ?/sec 1.00 270.2±1.82µs ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, no NULLs 1.00 73.3±5.62µs ? ?/sec 1.08 78.8±2.57µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, mandatory, no NULLs 1.00 86.6±0.43µs ? ?/sec 1.00 86.3±1.11µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, half NULLs 1.00 228.0±1.10µs ? ?/sec 1.00 228.6±2.31µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, no NULLs 1.00 91.7±0.40µs ? ?/sec 1.00 91.3±1.43µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, mandatory, no NULLs 1.05 9.7±0.35µs ? ?/sec 1.00 9.3±0.16µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, half NULLs 1.00 189.9±1.04µs ? ?/sec 1.00 189.3±1.47µs ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, no NULLs 1.04 14.7±0.21µs ? ?/sec 1.00 14.2±0.12µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, mandatory, no NULLs 1.01 171.2±2.16µs ? ?/sec 1.00 170.0±0.81µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, half NULLs 1.02 373.4±6.43µs ? ?/sec 1.00 367.6±1.80µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, no NULLs 1.00 175.8±1.70µs ? ?/sec 1.00 175.3±1.53µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, mandatory, no NULLs 1.06 14.5±0.42µs ? ?/sec 1.00 13.7±0.13µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, half NULLs 1.01 296.0±8.29µs ? ?/sec 1.00 293.7±2.33µs ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, no NULLs 1.05 20.7±0.85µs ? ?/sec 1.00 19.7±0.44µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, mandatory, no NULLs 1.00 341.0±2.17µs ? ?/sec 1.00 342.1±2.54µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, half NULLs 1.01 398.4±5.57µs ? ?/sec 1.00 395.6±14.70µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, no NULLs 1.00 347.3±3.62µs ? ?/sec 1.01 350.6±4.07µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, mandatory, no NULLs 1.00 27.4±0.66µs ? ?/sec 1.13 30.9±0.95µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, half NULLs 1.02 242.7±5.06µs ? ?/sec 1.00 237.6±2.28µs ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, no NULLs 1.00 33.3±0.77µs ? ?/sec 1.14 38.0±0.97µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, mandatory, no NULLs 1.00 109.1±2.20µs ? ?/sec 1.00 109.2±0.65µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, half NULLs 1.00 129.8±1.46µs ? ?/sec 1.00 130.0±2.26µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, no NULLs 1.00 111.7±0.60µs ? ?/sec 1.00 112.2±0.55µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, mandatory, no NULLs 1.01 161.1±10.25µs ? ?/sec 1.00 159.6±2.21µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, half NULLs 1.00 222.2±7.31µs ? ?/sec 1.00 222.2±3.01µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, no NULLs 1.01 165.6±1.70µs ? ?/sec 1.00 164.3±0.97µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.03 76.2±0.64µs ? ?/sec 1.00 74.0±0.43µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.00 178.6±6.78µs ? ?/sec 1.00 179.2±1.14µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.01 80.9±3.22µs ? ?/sec 1.00 80.2±0.49µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, mandatory, no NULLs 1.02 145.2±2.16µs ? ?/sec 1.00 142.7±0.55µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, half NULLs 1.00 217.6±1.36µs ? ?/sec 1.00 217.2±7.04µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, no NULLs 1.01 151.0±1.55µs ? ?/sec 1.00 149.6±7.06µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, mandatory, no NULLs 1.03 74.1±1.24µs ? ?/sec 1.00 72.1±0.56µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, half NULLs 1.01 177.4±0.83µs ? ?/sec 1.00 176.3±2.46µs ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, no NULLs 1.00 76.8±0.44µs ? ?/sec 1.02 78.4±1.36µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, mandatory, no NULLs 1.01 108.7±0.47µs ? ?/sec 1.00 107.2±0.75µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, half NULLs 1.00 118.1±1.26µs ? ?/sec 1.02 120.4±0.93µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, no NULLs 1.00 110.5±1.18µs ? ?/sec 1.01 112.1±1.45µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, mandatory, no NULLs 1.00 160.1±1.09µs ? ?/sec 1.00 160.6±0.77µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, half NULLs 1.00 207.9±2.07µs ? ?/sec 1.00 208.8±2.37µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, no NULLs 1.00 165.2±0.55µs ? ?/sec 1.01 167.1±2.94µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs 1.00 201.0±1.41µs ? ?/sec 1.00 201.1±1.67µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, half NULLs 1.00 222.5±1.65µs ? ?/sec 1.01 223.9±1.42µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, no NULLs 1.00 206.7±1.61µs ? ?/sec 1.00 206.9±0.76µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, mandatory, no NULLs 1.01 142.8±3.09µs ? ?/sec 1.00 141.8±0.70µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, half NULLs 1.00 195.1±1.02µs ? ?/sec 1.01 196.9±14.93µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, no NULLs 1.01 148.6±0.76µs ? ?/sec 1.00 147.3±1.27µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, mandatory, no NULLs 1.05 103.0±1.47µs ? ?/sec 1.00 98.2±1.66µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, half NULLs 1.00 168.2±2.84µs ? ?/sec 1.03 173.3±2.50µs ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, no NULLs 1.01 110.2±1.17µs ? ?/sec 1.00 108.7±1.95µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, mandatory, no NULLs 1.00 76.9±1.19µs ? ?/sec 1.00 77.2±0.40µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, half NULLs 1.00 103.5±2.23µs ? ?/sec 1.00 103.6±0.42µs ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, no NULLs 1.00 79.6±1.37µs ? ?/sec 1.01 80.7±1.60µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, mandatory, no NULLs 1.00 107.0±2.13µs ? ?/sec 1.00 107.4±1.13µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, half NULLs 1.00 174.9±2.95µs ? ?/sec 1.00 175.1±1.14µs ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, no NULLs 1.00 111.2±1.25µs ? ?/sec 1.01 112.0±0.67µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, mandatory, no NULLs 1.00 40.9±0.42µs ? ?/sec 1.05 42.8±0.39µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, half NULLs 1.00 141.2±1.14µs ? ?/sec 1.00 140.8±2.60µs ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, no NULLs 1.00 45.5±0.54µs ? ?/sec 1.04 47.3±0.43µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, mandatory, no NULLs 1.02 111.8±1.08µs ? ?/sec 1.00 109.6±1.36µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, half NULLs 1.00 180.7±2.26µs ? ?/sec 1.00 179.8±2.71µs ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, no NULLs 1.02 116.2±0.90µs ? ?/sec 1.00 114.5±1.13µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, mandatory, no NULLs 1.00 36.6±0.16µs ? ?/sec 1.00 36.8±0.30µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, half NULLs 1.00 138.8±0.80µs ? ?/sec 1.00 138.1±0.89µs ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, no NULLs 1.00 41.5±0.28µs ? ?/sec 1.00 41.7±0.35µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, mandatory, no NULLs 1.00 82.6±1.92µs ? ?/sec 1.01 83.1±0.52µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, half NULLs 1.00 101.4±0.50µs ? ?/sec 1.01 102.7±4.58µs ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, no NULLs 1.01 87.2±1.36µs ? ?/sec 1.00 86.0±1.10µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs 1.00 106.6±0.88µs ? ?/sec 1.01 107.7±3.34µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, half NULLs 1.00 165.7±2.07µs ? ?/sec 1.00 166.0±2.06µs ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, no NULLs 1.00 111.6±1.55µs ? ?/sec 1.00 111.8±2.75µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, mandatory, no NULLs 1.00 23.3±0.28µs ? ?/sec 1.02 23.8±0.27µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, half NULLs 1.00 122.0±0.74µs ? ?/sec 1.00 121.7±0.54µs ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, no NULLs 1.00 28.1±0.48µs ? ?/sec 1.00 28.1±0.39µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs 1.02 93.3±0.48µs ? ?/sec 1.00 91.2±1.86µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs 1.01 161.2±0.84µs ? ?/sec 1.00 159.5±1.20µs ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs 1.01 97.2±0.97µs ? ?/sec 1.00 95.9±0.68µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs 1.01 15.7±0.23µs ? ?/sec 1.00 15.5±0.18µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs 1.01 121.2±3.98µs ? ?/sec 1.00 120.0±2.35µs ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs 1.00 21.4±0.40µs ? ?/sec 1.00 21.4±0.24µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, mandatory, no NULLs 1.00 80.0±0.75µs ? ?/sec 1.01 80.6±0.80µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, half NULLs 1.00 89.5±0.76µs ? ?/sec 1.00 89.7±1.19µs ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, no NULLs 1.00 82.3±0.29µs ? ?/sec 1.01 82.8±0.56µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs 1.01 108.1±1.01µs ? ?/sec 1.00 107.2±0.73µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, half NULLs 1.00 142.6±1.67µs ? ?/sec 1.00 142.7±1.19µs ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, no NULLs 1.00 110.1±3.41µs ? ?/sec 1.00 110.6±3.50µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, mandatory, no NULLs 1.00 146.5±1.01µs ? ?/sec 1.00 146.7±0.46µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, half NULLs 1.01 168.1±1.81µs ? ?/sec 1.00 166.8±1.40µs ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, no NULLs 1.00 151.3±0.63µs ? ?/sec 1.01 152.1±1.03µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs 1.01 89.1±1.08µs ? ?/sec 1.00 87.9±0.62µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs 1.01 136.5±1.06µs ? ?/sec 1.00 135.4±0.84µs ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs 1.02 94.4±0.66µs ? ?/sec 1.00 92.7±0.77µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs 1.00 37.9±0.84µs ? ?/sec 1.08 41.0±1.21µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs 1.02 111.9±1.91µs ? ?/sec 1.00 110.0±0.73µs ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs 1.00 42.3±0.67µs ? ?/sec 1.13 47.8±1.26µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, mandatory, no NULLs 1.00 80.6±0.81µs ? ?/sec 1.01 81.4±0.56µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, half NULLs 1.01 103.7±1.34µs ? ?/sec 1.00 103.1±0.54µs ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, no NULLs 1.00 83.3±0.82µs ? ?/sec 1.01 83.9±0.58µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, mandatory, no NULLs 1.00 108.0±1.05µs ? ?/sec 1.01 109.2±1.93µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, half NULLs 1.00 171.5±2.97µs ? ?/sec 1.00 171.8±2.23µs ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, no NULLs 1.00 112.7±1.00µs ? ?/sec 1.01 113.3±0.48µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, mandatory, no NULLs 1.00 34.5±0.16µs ? ?/sec 1.01 34.8±0.18µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, half NULLs 1.01 133.7±0.82µs ? ?/sec 1.00 132.9±0.89µs ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, no NULLs 1.00 39.2±0.78µs ? ?/sec 1.00 39.1±0.21µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, mandatory, no NULLs 1.01 103.6±0.59µs ? ?/sec 1.00 102.8±3.53µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, half NULLs 1.01 173.2±2.43µs ? ?/sec 1.00 170.8±0.93µs ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, no NULLs 1.01 108.4±0.61µs ? ?/sec 1.00 106.9±1.29µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, mandatory, no NULLs 1.00 28.8±0.14µs ? ?/sec 1.01 29.0±0.18µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, half NULLs 1.02 132.4±1.30µs ? ?/sec 1.00 129.9±0.78µs ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, no NULLs 1.00 33.6±0.30µs ? ?/sec 1.00 33.6±0.21µs ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings half NULLs 1.00 6.1±0.07ms ? ?/sec 1.05 6.4±0.07ms ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings no NULLs 1.00 12.1±0.20ms ? ?/sec 1.10 13.3±0.20ms ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs 1.00 492.2±6.02µs ? ?/sec 1.00 494.1±14.55µs ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs 1.01 657.4±10.10µs ? ?/sec 1.00 647.9±7.94µs ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs 1.02 489.2±6.99µs ? ?/sec 1.00 480.2±7.14µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs 1.00 656.5±25.53µs ? ?/sec 1.02 667.1±6.12µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, half NULLs 1.00 773.7±8.49µs ? ?/sec 1.01 778.5±9.79µs ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, no NULLs 1.00 658.4±7.84µs ? ?/sec 1.03 679.4±10.09µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs 1.03 339.0±6.01µs ? ?/sec 1.00 329.0±2.97µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs 1.01 417.7±6.25µs ? ?/sec 1.00 411.8±2.62µs ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs 1.03 344.3±4.44µs ? ?/sec 1.00 333.8±3.21µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, mandatory, no NULLs 1.02 151.7±3.33µs ? ?/sec 1.00 149.3±1.85µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, half NULLs 1.00 206.4±0.96µs ? ?/sec 1.02 211.3±6.26µs ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, no NULLs 1.00 146.3±9.72µs ? ?/sec 1.02 149.5±2.04µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, mandatory, no NULLs 1.00 363.0±3.10µs ? ?/sec 1.00 364.5±4.68µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, half NULLs 1.00 327.5±1.95µs ? ?/sec 1.00 326.9±4.05µs ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, no NULLs 1.00 373.6±4.72µs ? ?/sec 1.01 378.2±9.09µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, mandatory, no NULLs 1.00 92.2±0.43µs ? ?/sec 1.00 92.7±0.54µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, half NULLs 1.00 112.0±3.52µs ? ?/sec 1.00 111.9±2.66µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, no NULLs 1.00 95.1±0.69µs ? ?/sec 1.01 95.6±1.45µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, mandatory, no NULLs 1.00 126.3±1.95µs ? ?/sec 1.00 126.9±1.25µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, half NULLs 1.00 186.4±1.84µs ? ?/sec 1.00 186.7±5.23µs ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, no NULLs 1.00 130.9±1.31µs ? ?/sec 1.00 131.4±0.57µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, mandatory, no NULLs 1.00 41.0±0.32µs ? ?/sec 1.04 42.6±0.20µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, half NULLs 1.00 141.1±1.43µs ? ?/sec 1.01 142.4±2.82µs ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, no NULLs 1.00 45.7±1.46µs ? ?/sec 1.03 46.9±0.36µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, mandatory, no NULLs 1.02 111.6±0.60µs ? ?/sec 1.00 109.9±1.17µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, half NULLs 1.00 182.0±11.10µs ? ?/sec 1.00 181.4±9.72µs ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, no NULLs 1.02 116.6±0.83µs ? ?/sec 1.00 114.7±1.95µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, mandatory, no NULLs 1.00 36.9±0.20µs ? ?/sec 1.01 37.3±0.95µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, half NULLs 1.00 138.9±1.05µs ? ?/sec 1.00 138.5±2.34µs ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, no NULLs 1.00 41.5±0.28µs ? ?/sec 1.00 41.5±0.29µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, mandatory, no NULLs 1.00 82.8±3.84µs ? ?/sec 1.00 82.9±0.87µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, half NULLs 1.00 101.7±1.37µs ? ?/sec 1.00 101.6±1.07µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, no NULLs 1.00 85.1±1.17µs ? ?/sec 1.02 86.8±2.55µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, mandatory, no NULLs 1.00 106.8±0.94µs ? ?/sec 1.01 107.7±0.86µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, half NULLs 1.01 167.4±6.62µs ? ?/sec 1.00 165.7±1.61µs ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, no NULLs 1.00 111.0±0.53µs ? ?/sec 1.01 112.0±1.59µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, mandatory, no NULLs 1.00 22.3±0.36µs ? ?/sec 1.08 24.1±0.26µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, half NULLs 1.00 121.9±1.44µs ? ?/sec 1.00 121.8±0.80µs ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, no NULLs 1.00 26.8±0.40µs ? ?/sec 1.05 28.2±0.61µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, mandatory, no NULLs 1.02 93.3±3.44µs ? ?/sec 1.00 91.4±0.63µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, half NULLs 1.01 161.3±1.65µs ? ?/sec 1.00 159.8±0.99µs ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, no NULLs 1.02 97.3±0.67µs ? ?/sec 1.00 95.8±0.72µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, mandatory, no NULLs 1.00 17.9±0.71µs ? ?/sec 1.08 19.3±0.51µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, half NULLs 1.01 120.5±2.18µs ? ?/sec 1.00 119.6±1.30µs ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, no NULLs 1.00 23.5±0.91µs ? ?/sec 1.03 24.1±0.50µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, mandatory, no NULLs 1.01 81.2±5.44µs ? ?/sec 1.00 80.6±1.49µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, half NULLs 1.01 90.9±2.93µs ? ?/sec 1.00 89.9±0.61µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, no NULLs 1.00 82.9±0.66µs ? ?/sec 1.00 83.1±1.21µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, mandatory, no NULLs 1.00 107.0±3.20µs ? ?/sec 1.01 107.6±1.30µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, half NULLs 1.00 144.7±0.67µs ? ?/sec 1.00 144.0±2.55µs ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, no NULLs 1.00 110.4±1.22µs ? ?/sec 1.00 110.8±1.79µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, mandatory, no NULLs 1.00 146.7±1.34µs ? ?/sec 1.01 148.5±1.06µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, half NULLs 1.00 167.2±1.47µs ? ?/sec 1.01 168.3±1.23µs ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, no NULLs 1.00 151.5±0.87µs ? ?/sec 1.01 153.7±1.13µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, mandatory, no NULLs 1.01 88.9±1.55µs ? ?/sec 1.00 88.1±0.54µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, half NULLs 1.00 137.0±0.65µs ? ?/sec 1.05 143.9±7.37µs ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, no NULLs 1.01 93.9±0.73µs ? ?/sec 1.00 92.9±0.69µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, mandatory, no NULLs 1.00 38.3±0.72µs ? ?/sec 1.03 39.5±0.64µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, half NULLs 1.00 111.8±0.77µs ? ?/sec 1.00 111.3±1.47µs ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, no NULLs 1.04 47.1±0.90µs ? ?/sec 1.00 45.1±0.86µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, mandatory, no NULLs 1.00 88.1±0.99µs ? ?/sec 1.01 89.4±3.75µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, half NULLs 1.00 106.9±1.18µs ? ?/sec 1.00 107.0±0.70µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, no NULLs 1.00 90.4±0.55µs ? ?/sec 1.01 90.9±1.46µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, mandatory, no NULLs 1.00 118.1±3.13µs ? ?/sec 1.00 117.8±1.14µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, half NULLs 1.01 179.3±11.66µs ? ?/sec 1.00 177.5±1.36µs ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, no NULLs 1.00 122.7±2.13µs ? ?/sec 1.00 122.6±1.24µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, mandatory, no NULLs 1.00 34.8±0.51µs ? ?/sec 1.00 34.7±0.35µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, half NULLs 1.01 134.0±0.67µs ? ?/sec 1.00 133.3±1.41µs ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, no NULLs 1.00 39.1±0.50µs ? ?/sec 1.00 39.2±0.34µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, mandatory, no NULLs 1.02 103.8±0.53µs ? ?/sec 1.00 101.6±1.30µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, half NULLs 1.01 173.0±1.23µs ? ?/sec 1.00 171.6±2.60µs ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, no NULLs 1.03 109.0±2.43µs ? ?/sec 1.00 106.1±0.61µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, mandatory, no NULLs 1.00 28.8±0.16µs ? ?/sec 1.01 29.2±0.59µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, half NULLs 1.01 132.0±3.14µs ? ?/sec 1.00 130.3±0.88µs ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, no NULLs 1.01 33.7±0.25µs ? ?/sec 1.00 33.5±0.65µs ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, half NULLs 1.00 120.8±2.08µs ? ?/sec 1.00 121.0±1.70µs ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, no NULLs 1.00 22.5±0.92µs ? ?/sec 1.02 22.9±0.58µs ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, half NULLs 1.00 242.2±3.27µs ? ?/sec 1.00 243.2±6.10µs ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, no NULLs 1.01 119.2±1.70µs ? ?/sec 1.00 118.1±2.27µs ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282#issuecomment-3808866654
rum bemchmark arrow_reader
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] parquet: reduce clone in delta byte array decoder [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #9282:
URL: https://github.com/apache/arrow-rs/pull/9282#issuecomment-3809950095
run benchmark arrow_reader
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org