You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "rluvaton (via GitHub)" <gi...@apache.org> on 2025/12/31 19:30:29 UTC
[PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
rluvaton opened a new pull request, #9078:
URL: https://github.com/apache/arrow-rs/pull/9078
# Which issue does this PR close?
N/A
# Rationale for this change
Making the row length calculation faster which result in faster row conversion
# What changes are included in this PR?
Instead of iterating over the items in the array and getting the length from the byte slice, we use the offsets directly and zip with nulls if necessary
# Are these changes tested?
Existing tests
# Are there any user-facing changes?
Faster encoding
------
The next PR will do the same for view based
and the PR after that for row based
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3732580631
run benchmark row_format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#discussion_r2687791344
##########
arrow-row/src/lib.rs:
##########
@@ -1656,6 +1640,31 @@ fn row_lengths(cols: &[ArrayRef], encoders: &[Encoder]) -> LengthTracker {
tracker
}
+/// Add to [`LengthTracker`] the encoded length of each item in the [`GenericByteArray`]
+fn push_generic_byte_array_lengths<T: ByteArrayType>(
+ tracker: &mut LengthTracker,
+ array: &GenericByteArray<T>,
+) {
+ if let Some(nulls) = array.nulls().filter(|n| n.null_count() > 0) {
+ tracker.push_variable(
+ array
+ .offsets()
+ .lengths()
+ .zip(nulls.iter())
+ .map(|(length, is_valid)| if is_valid { Some(length) } else { None })
+ .map(variable::padded_length),
+ )
+ } else {
+ tracker.push_variable(
+ array
+ .offsets()
+ .lengths()
+ .map(Some)
Review Comment:
I hoped that it would be optimized away, but I see it is not.
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3745498548
@alamb can you please review and hopefully merge if approved with no comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3733106809
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing improve-row-lengths-for-binary (13d64a749e4350ca6d7a943a25c8f5934c55d041) to 5a1e48260b0727ed52d9c070382fa21d69560672 [diff](https://github.com/apache/arrow-rs/compare/5a1e48260b0727ed52d9c070382fa21d69560672..13d64a749e4350ca6d7a943a25c8f5934c55d041)
BENCH_NAME=row_format
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench row_format
BENCH_FILTER=
BENCH_BRANCH_NAME=improve-row-lengths-for-binary
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan merged PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#discussion_r2692356250
##########
arrow-row/src/variable.rs:
##########
@@ -55,11 +55,20 @@ pub fn encoded_len(a: Option<&[u8]>) -> usize {
#[inline]
pub fn padded_length(a: Option<usize>) -> usize {
match a {
- Some(a) if a <= BLOCK_SIZE => 1 + ceil(a, MINI_BLOCK_SIZE) * (MINI_BLOCK_SIZE + 1),
+ Some(a) => non_null_padded_length(a),
Review Comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3702843150
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group improve-row-lengths-for-binary main
----- ------------------------------ ----
append_rows 10 large_list(0) of u64(0) 1.00 626.3±6.97ns ? ?/sec 1.04 651.6±12.20ns ? ?/sec
append_rows 10 list(0) of u64(0) 1.00 669.9±4.76ns ? ?/sec 1.05 702.8±10.88ns ? ?/sec
append_rows 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 351.3±8.17µs ? ?/sec 1.05 368.9±2.34µs ? ?/sec
append_rows 4096 bool(0, 0.5) 1.41 12.1±0.12µs ? ?/sec 1.00 8.6±0.04µs ? ?/sec
append_rows 4096 bool(0.3, 0.5) 1.00 14.9±0.66µs ? ?/sec 1.14 17.0±0.15µs ? ?/sec
append_rows 4096 i64(0) 1.00 7.7±0.18µs ? ?/sec 1.00 7.7±0.20µs ? ?/sec
append_rows 4096 i64(0.3) 1.00 11.1±0.08µs ? ?/sec 1.37 15.3±0.14µs ? ?/sec
append_rows 4096 large_list(0) of u64(0) 1.00 158.2±1.60µs ? ?/sec 1.03 163.1±1.13µs ? ?/sec
append_rows 4096 large_list(0) sliced to 10 of u64(0) 1.00 897.6±30.55ns ? ?/sec 1.02 912.3±11.26ns ? ?/sec
append_rows 4096 list(0) of u64(0) 1.00 150.3±2.54µs ? ?/sec 1.10 165.6±1.29µs ? ?/sec
append_rows 4096 list(0) sliced to 10 of u64(0) 1.00 979.8±7.02ns ? ?/sec 1.08 1059.4±7.95ns ? ?/sec
append_rows 4096 string view(1..100, 0) 1.03 117.6±0.93µs ? ?/sec 1.00 114.5±0.50µs ? ?/sec
append_rows 4096 string view(1..100, 0.5) 1.01 103.4±0.74µs ? ?/sec 1.00 102.6±0.93µs ? ?/sec
append_rows 4096 string view(10, 0) 1.00 44.4±0.67µs ? ?/sec 1.17 51.9±0.23µs ? ?/sec
append_rows 4096 string view(100, 0) 1.04 78.5±1.13µs ? ?/sec 1.00 75.6±1.01µs ? ?/sec
append_rows 4096 string view(100, 0.5) 1.00 85.6±0.70µs ? ?/sec 1.01 86.2±0.38µs ? ?/sec
append_rows 4096 string view(30, 0) 1.00 47.9±0.43µs ? ?/sec 1.13 54.1±0.61µs ? ?/sec
append_rows 4096 string(10, 0) 1.00 36.1±0.13µs ? ?/sec 1.35 48.8±0.37µs ? ?/sec
append_rows 4096 string(100, 0) 1.00 68.7±2.88µs ? ?/sec 1.04 71.3±1.64µs ? ?/sec
append_rows 4096 string(100, 0.5) 1.11 91.1±2.23µs ? ?/sec 1.00 81.9±1.48µs ? ?/sec
append_rows 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 211.5±3.56µs ? ?/sec 1.04 220.6±3.80µs ? ?/sec
append_rows 4096 string(30, 0) 1.00 39.1±0.50µs ? ?/sec 1.27 49.5±0.47µs ? ?/sec
append_rows 4096 string_dictionary(10, 0) 1.00 61.6±0.57µs ? ?/sec 1.21 74.5±0.55µs ? ?/sec
append_rows 4096 string_dictionary(100, 0) 1.00 142.7±8.24µs ? ?/sec 1.01 144.5±1.45µs ? ?/sec
append_rows 4096 string_dictionary(100, 0.5) 1.00 106.1±2.38µs ? ?/sec 1.02 108.6±0.95µs ? ?/sec
append_rows 4096 string_dictionary(30, 0) 1.00 67.0±1.32µs ? ?/sec 1.15 77.0±0.29µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(10, 0) 1.00 24.7±0.16µs ? ?/sec 1.10 27.2±0.12µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(100, 0) 1.00 45.1±0.42µs ? ?/sec 1.04 46.7±1.02µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(30, 0) 1.00 25.2±0.45µs ? ?/sec 1.09 27.3±0.42µs ? ?/sec
append_rows 4096 u64(0) 1.00 7.5±0.10µs ? ?/sec 1.00 7.6±0.16µs ? ?/sec
append_rows 4096 u64(0.3) 1.00 13.8±0.12µs ? ?/sec 1.06 14.6±0.15µs ? ?/sec
convert_columns 10 large_list(0) of u64(0) 1.00 907.2±12.04ns ? ?/sec 1.01 913.0±10.87ns ? ?/sec
convert_columns 10 list(0) of u64(0) 1.00 966.2±17.56ns ? ?/sec 1.00 965.7±6.15ns ? ?/sec
convert_columns 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 354.8±2.64µs ? ?/sec 1.05 372.7±7.47µs ? ?/sec
convert_columns 4096 bool(0, 0.5) 1.38 12.3±0.05µs ? ?/sec 1.00 9.0±0.19µs ? ?/sec
convert_columns 4096 bool(0.3, 0.5) 1.00 15.1±0.42µs ? ?/sec 1.15 17.3±0.20µs ? ?/sec
convert_columns 4096 i64(0) 1.00 7.9±0.10µs ? ?/sec 1.00 7.9±0.24µs ? ?/sec
convert_columns 4096 i64(0.3) 1.00 11.5±0.08µs ? ?/sec 1.35 15.5±0.20µs ? ?/sec
convert_columns 4096 large_list(0) of u64(0) 1.00 160.7±2.10µs ? ?/sec 1.02 163.4±0.77µs ? ?/sec
convert_columns 4096 large_list(0) sliced to 10 of u64(0) 1.00 1165.2±13.51ns ? ?/sec 1.02 1187.7±27.45ns ? ?/sec
convert_columns 4096 list(0) of u64(0) 1.00 150.8±2.98µs ? ?/sec 1.10 166.2±1.00µs ? ?/sec
convert_columns 4096 list(0) sliced to 10 of u64(0) 1.00 1276.1±47.49ns ? ?/sec 1.02 1296.8±24.11ns ? ?/sec
convert_columns 4096 string view(1..100, 0) 1.03 118.1±2.20µs ? ?/sec 1.00 115.1±1.25µs ? ?/sec
convert_columns 4096 string view(1..100, 0.5) 1.00 103.8±1.39µs ? ?/sec 1.00 103.3±0.92µs ? ?/sec
convert_columns 4096 string view(10, 0) 1.00 44.8±0.23µs ? ?/sec 1.18 52.6±0.62µs ? ?/sec
convert_columns 4096 string view(100, 0) 1.03 79.0±2.32µs ? ?/sec 1.00 76.5±0.92µs ? ?/sec
convert_columns 4096 string view(100, 0.5) 1.00 86.2±0.77µs ? ?/sec 1.00 86.2±1.72µs ? ?/sec
convert_columns 4096 string view(30, 0) 1.00 48.5±0.66µs ? ?/sec 1.11 53.9±0.20µs ? ?/sec
convert_columns 4096 string(10, 0) 1.00 36.5±0.34µs ? ?/sec 1.33 48.7±0.45µs ? ?/sec
convert_columns 4096 string(100, 0) 1.00 69.3±0.97µs ? ?/sec 1.04 72.0±0.92µs ? ?/sec
convert_columns 4096 string(100, 0.5) 1.11 91.1±0.83µs ? ?/sec 1.00 82.0±1.25µs ? ?/sec
convert_columns 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 213.1±3.04µs ? ?/sec 1.04 221.4±2.26µs ? ?/sec
convert_columns 4096 string(30, 0) 1.00 39.4±0.39µs ? ?/sec 1.26 49.6±0.22µs ? ?/sec
convert_columns 4096 string_dictionary(10, 0) 1.00 62.8±0.66µs ? ?/sec 1.22 76.6±0.82µs ? ?/sec
convert_columns 4096 string_dictionary(100, 0) 1.00 142.6±1.21µs ? ?/sec 1.03 147.0±4.41µs ? ?/sec
convert_columns 4096 string_dictionary(100, 0.5) 1.00 107.8±0.63µs ? ?/sec 1.01 109.4±2.44µs ? ?/sec
convert_columns 4096 string_dictionary(30, 0) 1.00 68.8±0.57µs ? ?/sec 1.14 78.4±0.80µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(10, 0) 1.00 25.8±0.19µs ? ?/sec 1.09 28.1±0.13µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(100, 0) 1.00 47.1±0.16µs ? ?/sec 1.01 47.3±0.23µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(30, 0) 1.00 26.1±0.69µs ? ?/sec 1.09 28.4±0.59µs ? ?/sec
convert_columns 4096 u64(0) 1.00 7.8±0.16µs ? ?/sec 1.01 7.8±0.13µs ? ?/sec
convert_columns 4096 u64(0.3) 1.00 14.1±0.21µs ? ?/sec 1.05 14.9±0.12µs ? ?/sec
convert_columns_prepared 10 large_list(0) of u64(0) 1.00 686.0±7.06ns ? ?/sec 1.03 708.5±20.14ns ? ?/sec
convert_columns_prepared 10 list(0) of u64(0) 1.00 742.7±18.78ns ? ?/sec 1.02 760.0±9.35ns ? ?/sec
convert_columns_prepared 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 352.8±3.02µs ? ?/sec 1.05 368.7±3.81µs ? ?/sec
convert_columns_prepared 4096 bool(0, 0.5) 1.41 12.3±0.21µs ? ?/sec 1.00 8.7±0.07µs ? ?/sec
convert_columns_prepared 4096 bool(0.3, 0.5) 1.00 15.0±0.17µs ? ?/sec 1.15 17.2±0.34µs ? ?/sec
convert_columns_prepared 4096 i64(0) 1.00 7.7±0.07µs ? ?/sec 1.02 7.9±0.13µs ? ?/sec
convert_columns_prepared 4096 i64(0.3) 1.00 11.2±0.10µs ? ?/sec 1.37 15.4±0.16µs ? ?/sec
convert_columns_prepared 4096 large_list(0) of u64(0) 1.00 159.1±3.82µs ? ?/sec 1.02 162.9±0.86µs ? ?/sec
convert_columns_prepared 4096 large_list(0) sliced to 10 of u64(0) 1.00 976.0±27.13ns ? ?/sec 1.03 1002.9±26.26ns ? ?/sec
convert_columns_prepared 4096 list(0) of u64(0) 1.00 150.7±3.19µs ? ?/sec 1.10 166.2±2.37µs ? ?/sec
convert_columns_prepared 4096 list(0) sliced to 10 of u64(0) 1.00 1079.6±23.87ns ? ?/sec 1.01 1094.0±6.79ns ? ?/sec
convert_columns_prepared 4096 string view(1..100, 0) 1.03 117.8±2.02µs ? ?/sec 1.00 114.7±0.83µs ? ?/sec
convert_columns_prepared 4096 string view(1..100, 0.5) 1.00 103.6±1.46µs ? ?/sec 1.00 103.2±0.87µs ? ?/sec
convert_columns_prepared 4096 string view(10, 0) 1.00 44.4±0.25µs ? ?/sec 1.18 52.2±0.73µs ? ?/sec
convert_columns_prepared 4096 string view(100, 0) 1.04 78.2±1.24µs ? ?/sec 1.00 75.1±0.74µs ? ?/sec
convert_columns_prepared 4096 string view(100, 0.5) 1.00 85.8±0.30µs ? ?/sec 1.00 86.0±0.48µs ? ?/sec
convert_columns_prepared 4096 string view(30, 0) 1.00 48.1±0.96µs ? ?/sec 1.13 54.5±2.11µs ? ?/sec
convert_columns_prepared 4096 string(10, 0) 1.00 36.2±0.46µs ? ?/sec 1.34 48.6±0.33µs ? ?/sec
convert_columns_prepared 4096 string(100, 0) 1.00 68.7±0.84µs ? ?/sec 1.04 71.8±0.83µs ? ?/sec
convert_columns_prepared 4096 string(100, 0.5) 1.11 91.0±0.89µs ? ?/sec 1.00 81.9±0.35µs ? ?/sec
convert_columns_prepared 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 210.5±1.60µs ? ?/sec 1.05 220.9±3.37µs ? ?/sec
convert_columns_prepared 4096 string(30, 0) 1.00 39.3±0.45µs ? ?/sec 1.26 49.5±0.35µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(10, 0) 1.00 61.6±0.65µs ? ?/sec 1.22 75.0±0.98µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(100, 0) 1.00 141.7±1.03µs ? ?/sec 1.03 145.6±1.73µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(100, 0.5) 1.00 106.6±0.39µs ? ?/sec 1.02 108.7±0.88µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(30, 0) 1.00 66.8±0.25µs ? ?/sec 1.15 76.9±0.31µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(10, 0) 1.00 25.1±0.30µs ? ?/sec 1.09 27.4±0.20µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(100, 0) 1.00 45.4±0.21µs ? ?/sec 1.03 46.7±1.80µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(30, 0) 1.00 25.3±0.27µs ? ?/sec 1.08 27.4±0.13µs ? ?/sec
convert_columns_prepared 4096 u64(0) 1.00 7.7±0.12µs ? ?/sec 1.00 7.7±0.17µs ? ?/sec
convert_columns_prepared 4096 u64(0.3) 1.00 14.0±0.20µs ? ?/sec 1.06 14.8±0.17µs ? ?/sec
convert_rows 10 large_list(0) of u64(0) 1.06 1621.2±17.01ns ? ?/sec 1.00 1527.5±8.85ns ? ?/sec
convert_rows 10 list(0) of u64(0) 1.11 1811.2±12.24ns ? ?/sec 1.00 1630.7±36.57ns ? ?/sec
convert_rows 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.03 314.6±8.67µs ? ?/sec 1.00 304.7±12.79µs ? ?/sec
convert_rows 4096 bool(0, 0.5) 1.19 19.7±0.14µs ? ?/sec 1.00 16.5±0.23µs ? ?/sec
convert_rows 4096 bool(0.3, 0.5) 1.19 19.7±0.20µs ? ?/sec 1.00 16.6±0.56µs ? ?/sec
convert_rows 4096 i64(0) 1.00 33.2±0.14µs ? ?/sec 1.23 40.7±13.84µs ? ?/sec
convert_rows 4096 i64(0.3) 1.00 33.3±1.19µs ? ?/sec 1.22 40.8±14.17µs ? ?/sec
convert_rows 4096 large_list(0) of u64(0) 1.01 273.4±2.37µs ? ?/sec 1.00 269.6±4.87µs ? ?/sec
convert_rows 4096 large_list(0) sliced to 10 of u64(0) 1.00 2.1±0.02µs ? ?/sec 1.00 2.1±0.02µs ? ?/sec
convert_rows 4096 list(0) of u64(0) 1.02 272.8±3.74µs ? ?/sec 1.00 268.3±3.44µs ? ?/sec
convert_rows 4096 list(0) sliced to 10 of u64(0) 1.04 2.3±0.03µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
convert_rows 4096 string view(1..100, 0) 1.00 168.9±0.73µs ? ?/sec 1.04 175.3±1.26µs ? ?/sec
convert_rows 4096 string view(1..100, 0.5) 1.00 137.4±0.87µs ? ?/sec 1.02 140.3±1.63µs ? ?/sec
convert_rows 4096 string view(10, 0) 1.00 76.0±1.68µs ? ?/sec 1.12 85.4±7.08µs ? ?/sec
convert_rows 4096 string view(100, 0) 1.00 123.7±1.68µs ? ?/sec 1.05 129.6±2.47µs ? ?/sec
convert_rows 4096 string view(100, 0.5) 1.00 113.5±0.31µs ? ?/sec 1.05 119.2±4.25µs ? ?/sec
convert_rows 4096 string view(30, 0) 1.00 85.1±1.60µs ? ?/sec 1.11 94.4±2.46µs ? ?/sec
convert_rows 4096 string(10, 0) 1.07 64.8±0.19µs ? ?/sec 1.00 60.4±1.39µs ? ?/sec
convert_rows 4096 string(100, 0) 1.04 114.6±1.40µs ? ?/sec 1.00 110.2±0.71µs ? ?/sec
convert_rows 4096 string(100, 0.5) 1.05 108.4±1.08µs ? ?/sec 1.00 103.4±0.51µs ? ?/sec
convert_rows 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.04 315.1±12.59µs ? ?/sec 1.00 304.2±13.21µs ? ?/sec
convert_rows 4096 string(30, 0) 1.08 78.3±3.33µs ? ?/sec 1.00 72.6±0.96µs ? ?/sec
convert_rows 4096 string_dictionary(10, 0) 1.09 66.0±7.15µs ? ?/sec 1.00 60.4±0.29µs ? ?/sec
convert_rows 4096 string_dictionary(100, 0) 1.04 115.1±1.09µs ? ?/sec 1.00 110.4±1.53µs ? ?/sec
convert_rows 4096 string_dictionary(100, 0.5) 1.05 109.4±6.25µs ? ?/sec 1.00 104.1±3.10µs ? ?/sec
convert_rows 4096 string_dictionary(30, 0) 1.08 78.3±0.40µs ? ?/sec 1.00 72.6±0.46µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(10, 0) 1.07 64.8±0.55µs ? ?/sec 1.00 60.5±0.30µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(100, 0) 1.03 114.6±1.56µs ? ?/sec 1.00 110.8±3.98µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(30, 0) 1.08 78.1±0.27µs ? ?/sec 1.00 72.6±0.27µs ? ?/sec
convert_rows 4096 u64(0) 1.01 32.3±0.75µs ? ?/sec 1.00 32.1±0.43µs ? ?/sec
convert_rows 4096 u64(0.3) 1.01 32.3±0.21µs ? ?/sec 1.00 31.9±0.14µs ? ?/sec
iterate rows 1.00 2.6±0.10µs ? ?/sec 1.00 2.6±0.05µs ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3702759760
run benchmark row_format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#discussion_r2687496585
##########
arrow-row/src/lib.rs:
##########
@@ -1656,6 +1640,31 @@ fn row_lengths(cols: &[ArrayRef], encoders: &[Encoder]) -> LengthTracker {
tracker
}
+/// Add to [`LengthTracker`] the encoded length of each item in the [`GenericByteArray`]
+fn push_generic_byte_array_lengths<T: ByteArrayType>(
+ tracker: &mut LengthTracker,
+ array: &GenericByteArray<T>,
+) {
+ if let Some(nulls) = array.nulls().filter(|n| n.null_count() > 0) {
+ tracker.push_variable(
+ array
+ .offsets()
+ .lengths()
+ .zip(nulls.iter())
+ .map(|(length, is_valid)| if is_valid { Some(length) } else { None })
+ .map(variable::padded_length),
Review Comment:
I verified that `variable::encoded_len` is calling `padded_len`
https://github.com/apache/arrow-rs/blob/9213ffd035f32b657965096bf5781d1ca1d5cf67/arrow-row/src/variable.rs#L51-L50
##########
arrow-row/src/lib.rs:
##########
@@ -1656,6 +1640,31 @@ fn row_lengths(cols: &[ArrayRef], encoders: &[Encoder]) -> LengthTracker {
tracker
}
+/// Add to [`LengthTracker`] the encoded length of each item in the [`GenericByteArray`]
+fn push_generic_byte_array_lengths<T: ByteArrayType>(
+ tracker: &mut LengthTracker,
+ array: &GenericByteArray<T>,
+) {
+ if let Some(nulls) = array.nulls().filter(|n| n.null_count() > 0) {
+ tracker.push_variable(
+ array
+ .offsets()
+ .lengths()
+ .zip(nulls.iter())
+ .map(|(length, is_valid)| if is_valid { Some(length) } else { None })
+ .map(variable::padded_length),
+ )
+ } else {
+ tracker.push_variable(
+ array
+ .offsets()
+ .lengths()
+ .map(Some)
Review Comment:
You could potentially avoid another branch by making a version of `padded_length` that takes `usize` instead of `Option<usize>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3702759922
🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing improve-row-lengths-for-binary (ff5b8ab62618d6c4ae7a8cbf60eceec25a53ce3e) to 843bee2c21dbe40a228c37dae7ecd876d73ee5de [diff](https://github.com/apache/arrow-rs/compare/843bee2c21dbe40a228c37dae7ecd876d73ee5de..ff5b8ab62618d6c4ae7a8cbf60eceec25a53ce3e)
BENCH_NAME=row_format
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench row_format
BENCH_FILTER=
BENCH_BRANCH_NAME=improve-row-lengths-for-binary
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3749357307
Very nice!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: improve calculating length performance for `GenericByteArray` in row conversion [arrow-rs]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #9078:
URL: https://github.com/apache/arrow-rs/pull/9078#issuecomment-3733214763
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group improve-row-lengths-for-binary main
----- ------------------------------ ----
append_rows 10 large_list(0) of u64(0) 1.00 618.8±13.52ns ? ?/sec 1.01 624.3±5.53ns ? ?/sec
append_rows 10 list(0) of u64(0) 1.00 671.1±16.99ns ? ?/sec 1.02 683.2±9.13ns ? ?/sec
append_rows 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 349.6±2.61µs ? ?/sec 1.06 369.3±3.29µs ? ?/sec
append_rows 4096 53 columns 1.00 1658.0±22.72µs ? ?/sec 1.06 1755.5±13.53µs ? ?/sec
append_rows 4096 bool(0, 0.5) 1.41 12.1±0.12µs ? ?/sec 1.00 8.6±0.08µs ? ?/sec
append_rows 4096 bool(0.3, 0.5) 1.00 15.4±0.35µs ? ?/sec 1.11 17.0±0.16µs ? ?/sec
append_rows 4096 i64(0) 1.01 7.8±0.26µs ? ?/sec 1.00 7.7±0.16µs ? ?/sec
append_rows 4096 i64(0.3) 1.00 14.1±0.26µs ? ?/sec 1.25 17.6±0.14µs ? ?/sec
append_rows 4096 large_list(0) of u64(0) 1.00 152.9±2.27µs ? ?/sec 1.08 165.6±1.51µs ? ?/sec
append_rows 4096 large_list(0) sliced to 10 of u64(0) 1.00 868.0±13.62ns ? ?/sec 1.02 884.6±21.52ns ? ?/sec
append_rows 4096 list(0) of u64(0) 1.00 155.1±2.43µs ? ?/sec 1.04 161.3±0.78µs ? ?/sec
append_rows 4096 list(0) sliced to 10 of u64(0) 1.00 982.6±17.49ns ? ?/sec 1.02 1003.8±17.90ns ? ?/sec
append_rows 4096 string view(1..100, 0) 1.01 118.3±1.18µs ? ?/sec 1.00 116.7±2.74µs ? ?/sec
append_rows 4096 string view(1..100, 0.5) 1.00 104.1±1.54µs ? ?/sec 1.00 104.6±0.67µs ? ?/sec
append_rows 4096 string view(10, 0) 1.00 44.4±0.74µs ? ?/sec 1.22 54.0±5.07µs ? ?/sec
append_rows 4096 string view(100, 0) 1.02 78.2±1.66µs ? ?/sec 1.00 76.5±3.68µs ? ?/sec
append_rows 4096 string view(100, 0.5) 1.01 87.6±0.84µs ? ?/sec 1.00 86.8±0.49µs ? ?/sec
append_rows 4096 string view(30, 0) 1.00 47.8±0.39µs ? ?/sec 1.17 56.1±0.32µs ? ?/sec
append_rows 4096 string(10, 0) 1.00 36.0±0.33µs ? ?/sec 1.32 47.7±0.51µs ? ?/sec
append_rows 4096 string(100, 0) 1.00 68.3±3.15µs ? ?/sec 1.05 71.9±0.72µs ? ?/sec
append_rows 4096 string(100, 0.5) 1.08 90.7±0.99µs ? ?/sec 1.00 83.8±0.99µs ? ?/sec
append_rows 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 209.5±4.67µs ? ?/sec 1.09 227.8±2.40µs ? ?/sec
append_rows 4096 string(30, 0) 1.00 38.9±0.48µs ? ?/sec 1.28 49.7±1.62µs ? ?/sec
append_rows 4096 string_dictionary(10, 0) 1.00 61.3±1.44µs ? ?/sec 1.22 75.0±0.32µs ? ?/sec
append_rows 4096 string_dictionary(100, 0) 1.00 140.8±2.36µs ? ?/sec 1.03 144.7±1.70µs ? ?/sec
append_rows 4096 string_dictionary(100, 0.5) 1.00 103.2±1.89µs ? ?/sec 1.05 108.1±2.88µs ? ?/sec
append_rows 4096 string_dictionary(30, 0) 1.00 66.7±1.61µs ? ?/sec 1.16 77.1±0.31µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(10, 0) 1.00 25.1±0.19µs ? ?/sec 1.08 27.0±0.06µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(100, 0) 1.00 44.8±0.08µs ? ?/sec 1.04 46.5±0.23µs ? ?/sec
append_rows 4096 string_dictionary_low_cardinality(30, 0) 1.00 25.4±0.16µs ? ?/sec 1.09 27.5±0.19µs ? ?/sec
append_rows 4096 u64(0) 1.00 7.6±0.15µs ? ?/sec 1.00 7.6±0.14µs ? ?/sec
append_rows 4096 u64(0.3) 1.08 14.9±0.12µs ? ?/sec 1.00 13.9±0.14µs ? ?/sec
append_rows 8192 53 columns 1.00 3.6±0.10ms ? ?/sec 1.06 3.8±0.06ms ? ?/sec
convert_columns 10 large_list(0) of u64(0) 1.00 894.0±14.73ns ? ?/sec 1.00 898.0±9.48ns ? ?/sec
convert_columns 10 list(0) of u64(0) 1.00 940.7±6.40ns ? ?/sec 1.01 951.5±5.61ns ? ?/sec
convert_columns 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 355.8±4.27µs ? ?/sec 1.06 377.7±3.06µs ? ?/sec
convert_columns 4096 53 columns 1.00 1655.8±12.42µs ? ?/sec 1.06 1754.7±27.19µs ? ?/sec
convert_columns 4096 bool(0, 0.5) 1.38 12.4±0.16µs ? ?/sec 1.00 8.9±0.13µs ? ?/sec
convert_columns 4096 bool(0.3, 0.5) 1.00 15.6±0.56µs ? ?/sec 1.11 17.3±0.30µs ? ?/sec
convert_columns 4096 i64(0) 1.00 8.0±0.17µs ? ?/sec 1.01 8.1±0.12µs ? ?/sec
convert_columns 4096 i64(0.3) 1.00 14.4±0.10µs ? ?/sec 1.25 18.0±0.20µs ? ?/sec
convert_columns 4096 large_list(0) of u64(0) 1.00 153.4±1.51µs ? ?/sec 1.08 165.6±1.61µs ? ?/sec
convert_columns 4096 large_list(0) sliced to 10 of u64(0) 1.00 1148.7±19.92ns ? ?/sec 1.01 1154.5±7.15ns ? ?/sec
convert_columns 4096 list(0) of u64(0) 1.00 155.0±0.92µs ? ?/sec 1.04 161.6±1.31µs ? ?/sec
convert_columns 4096 list(0) sliced to 10 of u64(0) 1.00 1240.1±30.22ns ? ?/sec 1.04 1291.9±41.23ns ? ?/sec
convert_columns 4096 string view(1..100, 0) 1.00 118.4±1.54µs ? ?/sec 1.00 117.9±4.12µs ? ?/sec
convert_columns 4096 string view(1..100, 0.5) 1.00 104.4±0.54µs ? ?/sec 1.01 105.2±1.80µs ? ?/sec
convert_columns 4096 string view(10, 0) 1.00 44.5±0.60µs ? ?/sec 1.22 54.3±0.20µs ? ?/sec
convert_columns 4096 string view(100, 0) 1.01 78.1±2.46µs ? ?/sec 1.00 77.4±4.39µs ? ?/sec
convert_columns 4096 string view(100, 0.5) 1.01 87.8±0.70µs ? ?/sec 1.00 87.2±1.12µs ? ?/sec
convert_columns 4096 string view(30, 0) 1.00 48.3±0.37µs ? ?/sec 1.19 57.4±0.61µs ? ?/sec
convert_columns 4096 string(10, 0) 1.00 36.3±0.27µs ? ?/sec 1.33 48.5±0.32µs ? ?/sec
convert_columns 4096 string(100, 0) 1.00 69.3±0.61µs ? ?/sec 1.05 73.0±1.10µs ? ?/sec
convert_columns 4096 string(100, 0.5) 1.08 90.9±3.04µs ? ?/sec 1.00 84.3±2.27µs ? ?/sec
convert_columns 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 208.9±1.33µs ? ?/sec 1.09 228.7±4.21µs ? ?/sec
convert_columns 4096 string(30, 0) 1.00 39.3±0.22µs ? ?/sec 1.27 49.7±0.16µs ? ?/sec
convert_columns 4096 string_dictionary(10, 0) 1.00 62.0±0.51µs ? ?/sec 1.24 77.1±1.11µs ? ?/sec
convert_columns 4096 string_dictionary(100, 0) 1.00 142.7±3.03µs ? ?/sec 1.03 146.9±1.86µs ? ?/sec
convert_columns 4096 string_dictionary(100, 0.5) 1.00 104.2±0.37µs ? ?/sec 1.05 109.4±0.65µs ? ?/sec
convert_columns 4096 string_dictionary(30, 0) 1.00 68.5±3.60µs ? ?/sec 1.15 78.8±1.23µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(10, 0) 1.00 25.9±0.06µs ? ?/sec 1.08 28.1±0.47µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(100, 0) 1.00 46.9±0.32µs ? ?/sec 1.03 48.4±1.54µs ? ?/sec
convert_columns 4096 string_dictionary_low_cardinality(30, 0) 1.00 26.6±0.13µs ? ?/sec 1.08 28.6±0.57µs ? ?/sec
convert_columns 4096 u64(0) 1.00 7.8±0.13µs ? ?/sec 1.03 8.0±0.16µs ? ?/sec
convert_columns 4096 u64(0.3) 1.04 15.1±0.20µs ? ?/sec 1.00 14.5±2.05µs ? ?/sec
convert_columns 8192 53 columns 1.00 3.6±0.16ms ? ?/sec 1.04 3.8±0.09ms ? ?/sec
convert_columns_prepared 10 large_list(0) of u64(0) 1.00 674.8±3.78ns ? ?/sec 1.01 683.8±11.35ns ? ?/sec
convert_columns_prepared 10 list(0) of u64(0) 1.00 730.4±7.33ns ? ?/sec 1.02 744.3±35.89ns ? ?/sec
convert_columns_prepared 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.00 348.8±5.61µs ? ?/sec 1.07 372.8±4.97µs ? ?/sec
convert_columns_prepared 4096 53 columns 1.00 1658.8±18.25µs ? ?/sec 1.06 1759.0±27.02µs ? ?/sec
convert_columns_prepared 4096 bool(0, 0.5) 1.41 12.3±0.10µs ? ?/sec 1.00 8.7±0.10µs ? ?/sec
convert_columns_prepared 4096 bool(0.3, 0.5) 1.00 15.4±0.29µs ? ?/sec 1.11 17.1±0.22µs ? ?/sec
convert_columns_prepared 4096 i64(0) 1.00 7.8±0.15µs ? ?/sec 1.01 7.8±0.17µs ? ?/sec
convert_columns_prepared 4096 i64(0.3) 1.00 14.3±0.14µs ? ?/sec 1.25 17.8±0.26µs ? ?/sec
convert_columns_prepared 4096 large_list(0) of u64(0) 1.00 153.4±1.90µs ? ?/sec 1.08 165.9±2.32µs ? ?/sec
convert_columns_prepared 4096 large_list(0) sliced to 10 of u64(0) 1.00 954.2±15.86ns ? ?/sec 1.01 967.8±4.86ns ? ?/sec
convert_columns_prepared 4096 list(0) of u64(0) 1.00 156.5±1.72µs ? ?/sec 1.03 161.7±0.39µs ? ?/sec
convert_columns_prepared 4096 list(0) sliced to 10 of u64(0) 1.00 1067.0±12.45ns ? ?/sec 1.01 1075.1±14.07ns ? ?/sec
convert_columns_prepared 4096 string view(1..100, 0) 1.01 117.9±0.46µs ? ?/sec 1.00 116.5±0.42µs ? ?/sec
convert_columns_prepared 4096 string view(1..100, 0.5) 1.00 104.3±1.28µs ? ?/sec 1.01 105.2±1.20µs ? ?/sec
convert_columns_prepared 4096 string view(10, 0) 1.00 44.4±0.35µs ? ?/sec 1.21 53.7±0.71µs ? ?/sec
convert_columns_prepared 4096 string view(100, 0) 1.02 78.0±0.69µs ? ?/sec 1.00 76.7±1.75µs ? ?/sec
convert_columns_prepared 4096 string view(100, 0.5) 1.01 87.8±1.15µs ? ?/sec 1.00 87.2±0.87µs ? ?/sec
convert_columns_prepared 4096 string view(30, 0) 1.00 47.9±0.24µs ? ?/sec 1.18 56.3±0.58µs ? ?/sec
convert_columns_prepared 4096 string(10, 0) 1.00 36.1±0.79µs ? ?/sec 1.33 48.1±1.96µs ? ?/sec
convert_columns_prepared 4096 string(100, 0) 1.00 68.4±2.05µs ? ?/sec 1.05 72.0±0.60µs ? ?/sec
convert_columns_prepared 4096 string(100, 0.5) 1.08 90.6±1.33µs ? ?/sec 1.00 83.9±1.06µs ? ?/sec
convert_columns_prepared 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.00 211.3±2.42µs ? ?/sec 1.07 226.1±1.30µs ? ?/sec
convert_columns_prepared 4096 string(30, 0) 1.00 39.4±1.83µs ? ?/sec 1.26 49.8±1.12µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(10, 0) 1.00 62.4±3.78µs ? ?/sec 1.21 75.8±1.89µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(100, 0) 1.00 143.0±3.81µs ? ?/sec 1.01 144.4±2.49µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(100, 0.5) 1.00 102.7±1.31µs ? ?/sec 1.06 108.5±4.18µs ? ?/sec
convert_columns_prepared 4096 string_dictionary(30, 0) 1.00 66.6±0.38µs ? ?/sec 1.17 77.8±3.33µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(10, 0) 1.00 25.3±0.26µs ? ?/sec 1.10 27.9±2.85µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(100, 0) 1.00 45.1±0.51µs ? ?/sec 1.04 47.0±0.27µs ? ?/sec
convert_columns_prepared 4096 string_dictionary_low_cardinality(30, 0) 1.00 25.6±0.32µs ? ?/sec 1.08 27.6±0.59µs ? ?/sec
convert_columns_prepared 4096 u64(0) 1.00 7.7±0.12µs ? ?/sec 1.01 7.8±0.18µs ? ?/sec
convert_columns_prepared 4096 u64(0.3) 1.07 14.9±0.13µs ? ?/sec 1.00 13.9±0.17µs ? ?/sec
convert_columns_prepared 8192 53 columns 1.00 3.6±0.06ms ? ?/sec 1.07 3.8±0.09ms ? ?/sec
convert_rows 10 large_list(0) of u64(0) 1.02 1547.5±53.26ns ? ?/sec 1.00 1523.3±8.50ns ? ?/sec
convert_rows 10 list(0) of u64(0) 1.01 1706.8±22.92ns ? ?/sec 1.00 1689.5±26.34ns ? ?/sec
convert_rows 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0), string_dictionary(100, 0), i64(0) 1.08 322.2±3.02µs ? ?/sec 1.00 299.2±4.83µs ? ?/sec
convert_rows 4096 53 columns 1.02 3.0±0.02ms ? ?/sec 1.00 2.9±0.05ms ? ?/sec
convert_rows 4096 bool(0, 0.5) 1.18 19.4±0.22µs ? ?/sec 1.00 16.5±0.47µs ? ?/sec
convert_rows 4096 bool(0.3, 0.5) 1.18 19.4±0.07µs ? ?/sec 1.00 16.5±0.24µs ? ?/sec
convert_rows 4096 i64(0) 1.05 34.7±0.15µs ? ?/sec 1.00 33.2±0.89µs ? ?/sec
convert_rows 4096 i64(0.3) 1.03 34.8±0.59µs ? ?/sec 1.00 33.7±2.70µs ? ?/sec
convert_rows 4096 large_list(0) of u64(0) 1.02 276.0±4.79µs ? ?/sec 1.00 271.4±4.09µs ? ?/sec
convert_rows 4096 large_list(0) sliced to 10 of u64(0) 1.03 2.0±0.04µs ? ?/sec 1.00 1954.3±10.37ns ? ?/sec
convert_rows 4096 list(0) of u64(0) 1.02 275.5±8.48µs ? ?/sec 1.00 270.4±6.19µs ? ?/sec
convert_rows 4096 list(0) sliced to 10 of u64(0) 1.05 2.3±0.02µs ? ?/sec 1.00 2.2±0.09µs ? ?/sec
convert_rows 4096 string view(1..100, 0) 1.00 171.7±11.53µs ? ?/sec 1.02 175.1±1.16µs ? ?/sec
convert_rows 4096 string view(1..100, 0.5) 1.00 136.6±1.14µs ? ?/sec 1.03 140.4±1.07µs ? ?/sec
convert_rows 4096 string view(10, 0) 1.00 75.5±0.40µs ? ?/sec 1.11 83.6±1.34µs ? ?/sec
convert_rows 4096 string view(100, 0) 1.00 123.4±1.31µs ? ?/sec 1.05 129.0±2.55µs ? ?/sec
convert_rows 4096 string view(100, 0.5) 1.00 113.1±0.26µs ? ?/sec 1.04 118.0±0.72µs ? ?/sec
convert_rows 4096 string view(30, 0) 1.00 86.8±1.05µs ? ?/sec 1.07 93.2±0.70µs ? ?/sec
convert_rows 4096 string(10, 0) 1.09 65.7±0.34µs ? ?/sec 1.00 60.3±1.10µs ? ?/sec
convert_rows 4096 string(100, 0) 1.05 115.3±1.15µs ? ?/sec 1.00 109.9±1.32µs ? ?/sec
convert_rows 4096 string(100, 0.5) 1.06 110.3±0.94µs ? ?/sec 1.00 104.0±1.40µs ? ?/sec
convert_rows 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) 1.03 316.6±11.27µs ? ?/sec 1.00 305.9±5.69µs ? ?/sec
convert_rows 4096 string(30, 0) 1.07 79.0±1.40µs ? ?/sec 1.00 73.5±0.67µs ? ?/sec
convert_rows 4096 string_dictionary(10, 0) 1.09 66.0±0.93µs ? ?/sec 1.00 60.5±0.33µs ? ?/sec
convert_rows 4096 string_dictionary(100, 0) 1.05 115.9±1.40µs ? ?/sec 1.00 110.0±0.55µs ? ?/sec
convert_rows 4096 string_dictionary(100, 0.5) 1.06 110.2±0.37µs ? ?/sec 1.00 103.8±0.39µs ? ?/sec
convert_rows 4096 string_dictionary(30, 0) 1.06 79.0±0.90µs ? ?/sec 1.00 74.3±1.76µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(10, 0) 1.09 65.8±0.27µs ? ?/sec 1.00 60.3±0.55µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(100, 0) 1.05 115.3±0.96µs ? ?/sec 1.00 109.7±0.55µs ? ?/sec
convert_rows 4096 string_dictionary_low_cardinality(30, 0) 1.07 79.1±0.61µs ? ?/sec 1.00 74.1±4.73µs ? ?/sec
convert_rows 4096 u64(0) 1.00 32.1±1.09µs ? ?/sec 1.02 32.8±1.76µs ? ?/sec
convert_rows 4096 u64(0.3) 1.00 32.2±1.50µs ? ?/sec 1.01 32.5±0.20µs ? ?/sec
convert_rows 8192 53 columns 1.00 7.1±0.20ms ? ?/sec 1.02 7.2±0.15ms ? ?/sec
iterate rows 1.00 2.6±0.02µs ? ?/sec 1.00 2.6±0.01µs ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org