You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "neilconway (via GitHub)" <gi...@apache.org> on 2026/09/19 15:52:49 UTC
[PR] perf: Size builder capacity using visible array span, not total size [datafusion]
neilconway opened a new pull request, #25502:
URL: https://github.com/apache/datafusion/pull/25502
## Which issue does this PR close?
- N/A
## Rationale for this change
Many UDFs used the raw size of an input array's values buffer to reserve capacity in a builder or similar data structure. This can result in significantly over-allocating space for sliced array inputs. We use the new offset span helpers added in #25481.
`array_element` had a similar but slightly distinct allocation mistake: it reserved space for every child element, but it only returns a single value per input row.
Along the way, refactor `array_add` to use a shared array math kernel.
## What changes are included in this PR?
* Update ~31 UDFs to size buffer allocations using visible input ranges, not raw input size
* Add unit tests checking that allocated capacity for various UDFs matches visible input size
* Reuse the shared array-math kernel for `array_add` and consolidate test fixtures
The following UDFs are improved:
- `array_add`
- `array_append`
- `array_compact`
- `array_concat`
- `array_element`
- `array_normalize`
- `array_pop_back`
- `array_pop_front`
- `array_prepend`
- `array_remove`
- `array_remove_all`
- `array_remove_n`
- `array_replace`
- `array_replace_all`
- `array_replace_n`
- `array_reverse` (List/LargeList inputs; ListView is more complex)
- `array_scale`
- `array_slice` (List/LargeList inputs; ListView is more complex)
- `array_subtract`
- `concat` (DataFusion)
- `concat_ws` (DataFusion)
- `initcap`
- `string_to_array`
- `translate`
- `concat` (Spark)
- `shuffle` (Spark)
- `slice` (Spark)
- `try_url_decode` (Spark)
- `unhex` (Spark)
- `url_decode` (Spark)
- `url_encode` (Spark)
## What is the testing strategy for this PR?
Existing tests pass. Added new unit tests to check that the behavior of various UDFs doesn't change for sliced vs. unsliced inputs; also checked that the allocated capacity for the output array reflects the visible span of the input, not the raw input buffer size.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5743415460
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/25502?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `93.55828%` with `21 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 82.39%. Comparing base ([`24a7f13`](https://app.codecov.io/gh/apache/datafusion/commit/24a7f13f51230def9ceb0eeae920fd3840324e13?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`812bf71`](https://app.codecov.io/gh/apache/datafusion/commit/812bf717f6826f7b2d275573a90e9cb6149d9ca6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/25502?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [datafusion/functions-nested/src/utils.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions-nested%2Fsrc%2Futils.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMtbmVzdGVkL3NyYy91dGlscy5ycw==) | 85.71% | [1 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/functions-nested/src/string.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions-nested%2Fsrc%2Fstring.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMtbmVzdGVkL3NyYy9zdHJpbmcucnM=) | 93.18% | [0 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/functions/src/unicode/translate.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions%2Fsrc%2Funicode%2Ftranslate.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMvc3JjL3VuaWNvZGUvdHJhbnNsYXRlLnJz) | 92.30% | [0 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/functions/src/strings.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions%2Fsrc%2Fstrings.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMvc3JjL3N0cmluZ3MucnM=) | 91.30% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/functions/src/unicode/initcap.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions%2Fsrc%2Funicode%2Finitcap.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMvc3JjL3VuaWNvZGUvaW5pdGNhcC5ycw==) | 80.00% | [0 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/spark/src/function/array/shuffle.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Fspark%2Fsrc%2Ffunction%2Farray%2Fshuffle.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9zcGFyay9zcmMvZnVuY3Rpb24vYXJyYXkvc2h1ZmZsZS5ycw==) | 88.23% | [0 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/spark/src/function/utils.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Fspark%2Fsrc%2Ffunction%2Futils.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9zcGFyay9zcmMvZnVuY3Rpb24vdXRpbHMucnM=) | 88.88% | [0 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/functions/src/utils.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Ffunctions%2Fsrc%2Futils.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9mdW5jdGlvbnMvc3JjL3V0aWxzLnJz) | 90.90% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/spark/src/function/math/unhex.rs](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&filepath=datafusion%2Fspark%2Fsrc%2Ffunction%2Fmath%2Funhex.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9zcGFyay9zcmMvZnVuY3Rpb24vbWF0aC91bmhleC5ycw==) | 87.50% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/25502?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #25502 +/- ##
========================================
Coverage 82.38% 82.39%
========================================
Files 1138 1138
Lines 433739 433974 +235
Branches 433739 433974 +235
========================================
+ Hits 357338 357558 +220
+ Misses 54847 54842 -5
- Partials 21554 21574 +20
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/25502?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on code in PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#discussion_r4057047366
##########
datafusion/functions-nested/src/utils.rs:
##########
@@ -602,4 +604,52 @@ mod tests {
assert!(needs_norm_scale(f64::INFINITY, 1));
assert!(needs_norm_scale(f64::NAN, 1));
}
+
+ /// Tests the array function supplied via `run` on small slices of a large
Review Comment:
`test_sliced_list` is maybe a little too generic :)
I went with `check_sliced_list_behavior`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749686443
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark string_expressions
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group HEAD neilc_fix-sliced-builder-capacity
----- ---- ---------------------------------
ascii/scalar_utf8 1.00 122.7±1.05ns ? ?/sec 1.00 122.6±1.10ns ? ?/sec
ascii/scalar_utf8view 1.00 123.5±1.11ns ? ?/sec 1.00 123.3±1.17ns ? ?/sec
ascii/string_ascii_only (null_density=0) 1.00 6.4±0.01µs ? ?/sec 1.00 6.4±0.01µs ? ?/sec
ascii/string_ascii_only (null_density=0.5) 1.08 7.4±0.01µs ? ?/sec 1.00 6.9±0.00µs ? ?/sec
ascii/string_utf8 (null_density=0) 1.00 8.7±0.01µs ? ?/sec 1.00 8.7±0.01µs ? ?/sec
ascii/string_utf8 (null_density=0.5) 1.09 8.8±0.01µs ? ?/sec 1.00 8.0±0.00µs ? ?/sec
ascii/string_view_ascii_only (null_density=0) 1.00 7.1±0.01µs ? ?/sec 1.00 7.1±0.07µs ? ?/sec
ascii/string_view_ascii_only (null_density=0.5) 1.12 9.0±0.01µs ? ?/sec 1.00 8.1±0.01µs ? ?/sec
ascii/string_view_utf8 (null_density=0) 1.00 13.6±0.13µs ? ?/sec 1.08 14.6±0.16µs ? ?/sec
ascii/string_view_utf8 (null_density=0.5) 1.12 10.5±0.01µs ? ?/sec 1.00 9.4±0.01µs ? ?/sec
chr/array 1.00 7.0±0.52µs ? ?/sec 1.02 7.2±0.53µs ? ?/sec
chr/scalar 1.00 135.5±1.12ns ? ?/sec 1.09 147.7±7.18ns ? ?/sec
concat function/concat/1024 1.00 14.2±0.02µs ? ?/sec 1.03 14.6±0.02µs ? ?/sec
concat function/concat/4096 1.00 56.6±0.17µs ? ?/sec 1.01 57.3±0.09µs ? ?/sec
concat function/concat/8192 1.00 111.6±0.21µs ? ?/sec 1.03 114.4±0.11µs ? ?/sec
concat function/concat/scalar 1.02 833.2±1.63ns ? ?/sec 1.00 815.8±4.93ns ? ?/sec
concat function/concat_view/1024 1.00 30.6±0.27µs ? ?/sec 1.00 30.8±0.20µs ? ?/sec
concat function/concat_view/4096 1.00 144.2±1.33µs ? ?/sec 1.00 144.8±1.12µs ? ?/sec
concat function/concat_view/8192 1.00 307.6±2.99µs ? ?/sec 1.00 306.8±2.15µs ? ?/sec
concat_ws function/concat_ws/1024 1.00 20.4±0.07µs ? ?/sec 1.01 20.6±0.06µs ? ?/sec
concat_ws function/concat_ws/4096 1.00 79.1±0.09µs ? ?/sec 1.01 80.2±0.19µs ? ?/sec
concat_ws function/concat_ws/8192 1.00 157.7±0.91µs ? ?/sec 1.01 159.3±0.23µs ? ?/sec
concat_ws function/concat_ws/scalar 1.00 973.2±3.38ns ? ?/sec 1.00 969.9±3.27ns ? ?/sec
contains_StringArray_array_search 1.00 219.4±4.34µs ? ?/sec 1.00 220.4±4.68µs ? ?/sec
contains_StringArray_scalar_search 1.00 73.1±0.15µs ? ?/sec 1.00 73.4±0.17µs ? ?/sec
contains_StringViewArray_array_search 1.00 232.7±5.68µs ? ?/sec 1.02 237.5±5.50µs ? ?/sec
contains_StringViewArray_scalar_search 1.00 79.8±0.25µs ? ?/sec 1.00 79.4±0.14µs ? ?/sec
contains_StringViewArray_scalar_strlen_128 1.00 79.6±0.12µs ? ?/sec 1.01 80.1±0.12µs ? ?/sec
contains_StringViewArray_scalar_strlen_32 1.01 41.0±0.06µs ? ?/sec 1.00 40.6±0.05µs ? ?/sec
contains_StringViewArray_scalar_strlen_512 1.00 231.9±0.76µs ? ?/sec 1.00 231.4±1.00µs ? ?/sec
contains_StringViewArray_scalar_strlen_8 1.00 47.2±0.03µs ? ?/sec 1.00 47.3±0.12µs ? ?/sec
ends_with_StringArray_array_suffix 1.01 43.3±0.66µs ? ?/sec 1.00 43.0±0.53µs ? ?/sec
ends_with_StringArray_scalar_suffix 1.02 11.5±0.03µs ? ?/sec 1.00 11.4±0.05µs ? ?/sec
ends_with_StringViewArray_array_suffix 1.01 56.6±0.51µs ? ?/sec 1.00 56.1±0.71µs ? ?/sec
ends_with_StringViewArray_scalar_strlen_128 1.00 17.4±0.02µs ? ?/sec 1.02 17.7±0.02µs ? ?/sec
ends_with_StringViewArray_scalar_strlen_32 1.00 16.2±0.02µs ? ?/sec 1.00 16.1±0.01µs ? ?/sec
ends_with_StringViewArray_scalar_strlen_512 1.01 17.9±0.03µs ? ?/sec 1.00 17.8±0.04µs ? ?/sec
ends_with_StringViewArray_scalar_strlen_8 1.00 17.9±0.01µs ? ?/sec 1.00 17.9±0.01µs ? ?/sec
ends_with_StringViewArray_scalar_suffix 1.03 18.1±0.04µs ? ?/sec 1.00 17.6±0.05µs ? ?/sec
long strings, long trim/btrim large_string [size=4096, len=64, remaining=8] 1.00 808.6±0.36µs ? ?/sec 1.00 807.2±0.55µs ? ?/sec
long strings, long trim/btrim string [size=4096, len=64, remaining=8] 1.00 809.1±0.45µs ? ?/sec 1.00 808.1±1.05µs ? ?/sec
long strings, long trim/btrim string_view [size=4096, len=64, remaining=8] 1.00 819.2±0.88µs ? ?/sec 1.00 818.6±0.17µs ? ?/sec
long strings, long trim/ltrim large_string [size=4096, len=64, remaining=8] 1.00 423.4±0.93µs ? ?/sec 1.00 421.9±1.19µs ? ?/sec
long strings, long trim/ltrim string [size=4096, len=64, remaining=8] 1.00 424.1±0.59µs ? ?/sec 1.00 424.0±0.90µs ? ?/sec
long strings, long trim/ltrim string_view [size=4096, len=64, remaining=8] 1.00 434.4±1.06µs ? ?/sec 1.00 432.9±0.88µs ? ?/sec
long strings, long trim/rtrim large_string [size=4096, len=64, remaining=8] 1.00 397.3±0.10µs ? ?/sec 1.00 397.5±0.09µs ? ?/sec
long strings, long trim/rtrim string [size=4096, len=64, remaining=8] 1.00 396.3±0.08µs ? ?/sec 1.00 396.5±0.11µs ? ?/sec
long strings, long trim/rtrim string_view [size=4096, len=64, remaining=8] 1.00 405.2±0.10µs ? ?/sec 1.00 405.6±0.18µs ? ?/sec
long strings, short trim/btrim large_string [size=4096, len=64, remaining=60] 1.01 97.6±0.02µs ? ?/sec 1.00 96.9±0.02µs ? ?/sec
long strings, short trim/btrim string [size=4096, len=64, remaining=60] 1.01 97.9±0.04µs ? ?/sec 1.00 96.8±0.02µs ? ?/sec
long strings, short trim/btrim string_view [size=4096, len=64, remaining=60] 1.00 107.2±0.03µs ? ?/sec 1.00 107.4±0.07µs ? ?/sec
long strings, short trim/ltrim large_string [size=4096, len=64, remaining=60] 1.00 54.0±0.01µs ? ?/sec 1.00 54.0±0.03µs ? ?/sec
long strings, short trim/ltrim string [size=4096, len=64, remaining=60] 1.00 55.4±0.01µs ? ?/sec 1.00 55.5±0.04µs ? ?/sec
long strings, short trim/ltrim string_view [size=4096, len=64, remaining=60] 1.00 63.4±0.03µs ? ?/sec 1.03 65.6±0.40µs ? ?/sec
long strings, short trim/rtrim large_string [size=4096, len=64, remaining=60] 1.00 55.6±0.05µs ? ?/sec 1.00 55.6±0.01µs ? ?/sec
long strings, short trim/rtrim string [size=4096, len=64, remaining=60] 1.00 54.5±0.01µs ? ?/sec 1.00 54.5±0.02µs ? ?/sec
long strings, short trim/rtrim string_view [size=4096, len=64, remaining=60] 1.00 59.9±0.04µs ? ?/sec 1.02 61.1±0.06µs ? ?/sec
lower_all_values_are_ascii: 1024 1.00 2.1±0.00µs ? ?/sec 1.00 2.0±0.00µs ? ?/sec
lower_all_values_are_ascii: 4096 1.00 7.5±0.01µs ? ?/sec 1.00 7.5±0.01µs ? ?/sec
lower_all_values_are_ascii: 8192 1.03 15.1±0.86µs ? ?/sec 1.00 14.7±0.02µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 10, null_density: 0, mixed: false 1.03 59.6±0.11µs ? ?/sec 1.00 57.7±0.05µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 10, null_density: 0, mixed: true 1.01 72.3±0.61µs ? ?/sec 1.00 71.5±0.23µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 10, null_density: 0.1, mixed: false 1.01 55.9±0.04µs ? ?/sec 1.00 55.6±0.04µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 10, null_density: 0.1, mixed: true 1.00 65.9±0.17µs ? ?/sec 1.04 68.3±0.12µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 128, null_density: 0, mixed: false 1.04 70.1±0.65µs ? ?/sec 1.00 67.4±0.17µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 128, null_density: 0, mixed: true 1.01 83.8±0.46µs ? ?/sec 1.00 83.1±0.39µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 128, null_density: 0.1, mixed: false 1.00 67.4±0.15µs ? ?/sec 1.02 68.6±0.12µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 128, null_density: 0.1, mixed: true 1.01 75.9±0.67µs ? ?/sec 1.00 75.4±0.20µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 64, null_density: 0, mixed: false 1.00 48.8±0.08µs ? ?/sec 1.03 50.2±0.10µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 64, null_density: 0, mixed: true 1.02 77.6±0.09µs ? ?/sec 1.00 76.2±0.26µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 64, null_density: 0.1, mixed: false 1.01 48.8±0.08µs ? ?/sec 1.00 48.5±0.04µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 4096, str_len: 64, null_density: 0.1, mixed: true 1.00 68.3±0.17µs ? ?/sec 1.00 68.3±0.15µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 10, null_density: 0, mixed: false 1.00 115.6±0.08µs ? ?/sec 1.00 115.2±0.09µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 10, null_density: 0, mixed: true 1.00 160.6±0.26µs ? ?/sec 1.00 160.1±0.36µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 10, null_density: 0.1, mixed: false 1.01 112.1±0.08µs ? ?/sec 1.00 111.5±0.41µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 10, null_density: 0.1, mixed: true 1.00 147.6±0.22µs ? ?/sec 1.00 147.3±0.28µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 128, null_density: 0, mixed: false 1.00 134.6±0.31µs ? ?/sec 1.02 136.8±0.27µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 128, null_density: 0, mixed: true 1.00 195.6±0.53µs ? ?/sec 1.00 195.8±0.45µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 128, null_density: 0.1, mixed: false 1.00 137.4±0.38µs ? ?/sec 1.00 137.7±0.29µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 128, null_density: 0.1, mixed: true 1.00 180.8±0.44µs ? ?/sec 1.00 181.1±0.43µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 64, null_density: 0, mixed: false 1.00 97.3±0.19µs ? ?/sec 1.02 99.3±1.61µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 64, null_density: 0, mixed: true 1.00 181.9±0.53µs ? ?/sec 1.00 181.3±0.41µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 64, null_density: 0.1, mixed: false 1.00 100.0±0.24µs ? ?/sec 1.02 102.0±0.16µs ? ?/sec
lower_all_values_are_ascii_string_views: size: 8192, str_len: 64, null_density: 0.1, mixed: true 1.00 166.7±0.41µs ? ?/sec 1.00 166.6±0.41µs ? ?/sec
lower_sliced_ascii: parent=65536, slice=128, str_len=32 1.01 525.8±1.49ns ? ?/sec 1.00 522.8±1.92ns ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 10, non_ascii_density: 0.1, null_density: 0, mixed: false 1.05 206.7±0.61µs ? ?/sec 1.00 196.3±0.70µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 10, non_ascii_density: 0.1, null_density: 0, mixed: true 1.00 199.4±0.68µs ? ?/sec 1.00 199.5±0.72µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 10, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.00 182.5±0.97µs ? ?/sec 1.01 184.8±0.60µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 10, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.00 180.1±0.49µs ? ?/sec 1.03 185.8±0.51µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 128, non_ascii_density: 0.1, null_density: 0, mixed: false 1.02 202.6±0.59µs ? ?/sec 1.00 199.4±0.65µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 128, non_ascii_density: 0.1, null_density: 0, mixed: true 1.00 205.7±0.83µs ? ?/sec 1.01 207.4±0.79µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 128, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.00 180.9±0.59µs ? ?/sec 1.02 183.7±0.56µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 128, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.03 179.7±0.52µs ? ?/sec 1.00 174.7±0.59µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 64, non_ascii_density: 0.1, null_density: 0, mixed: false 1.02 200.9±0.63µs ? ?/sec 1.00 196.3±0.66µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 64, non_ascii_density: 0.1, null_density: 0, mixed: true 1.03 201.3±0.99µs ? ?/sec 1.00 195.4±0.62µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 64, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.06 188.5±0.60µs ? ?/sec 1.00 177.5±0.59µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 4096, str_len: 64, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.00 176.2±0.55µs ? ?/sec 1.04 183.0±0.65µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 10, non_ascii_density: 0.1, null_density: 0, mixed: false 1.02 398.0±1.35µs ? ?/sec 1.00 391.5±1.47µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 10, non_ascii_density: 0.1, null_density: 0, mixed: true 1.01 406.1±1.22µs ? ?/sec 1.00 402.1±2.87µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 10, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.00 365.6±1.26µs ? ?/sec 1.01 368.2±1.31µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 10, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.00 364.2±1.30µs ? ?/sec 1.02 370.0±1.16µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 128, non_ascii_density: 0.1, null_density: 0, mixed: false 1.00 406.1±1.19µs ? ?/sec 1.01 410.4±1.30µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 128, non_ascii_density: 0.1, null_density: 0, mixed: true 1.00 403.9±1.69µs ? ?/sec 1.00 403.5±1.31µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 128, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.01 374.2±1.89µs ? ?/sec 1.00 369.4±1.12µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 128, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.00 361.9±1.00µs ? ?/sec 1.03 373.6±1.25µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 64, non_ascii_density: 0.1, null_density: 0, mixed: false 1.00 399.6±1.27µs ? ?/sec 1.01 403.6±1.44µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 64, non_ascii_density: 0.1, null_density: 0, mixed: true 1.00 397.9±1.18µs ? ?/sec 1.01 403.5±1.84µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 64, non_ascii_density: 0.1, null_density: 0.1, mixed: false 1.00 363.1±1.04µs ? ?/sec 1.00 364.0±1.05µs ? ?/sec
lower_some_values_are_nonascii_string_views: size: 8192, str_len: 64, non_ascii_density: 0.1, null_density: 0.1, mixed: true 1.00 364.2±0.99µs ? ?/sec 1.00 364.5±1.13µs ? ?/sec
lower_the_first_value_is_nonascii: 1024 1.00 26.6±0.11µs ? ?/sec 1.00 26.6±0.14µs ? ?/sec
lower_the_first_value_is_nonascii: 4096 1.01 107.9±0.47µs ? ?/sec 1.00 107.2±0.47µs ? ?/sec
lower_the_first_value_is_nonascii: 8192 1.00 215.3±0.95µs ? ?/sec 1.00 215.3±1.06µs ? ?/sec
lower_the_middle_value_is_nonascii: 1024 1.00 26.8±0.10µs ? ?/sec 1.00 26.8±0.11µs ? ?/sec
lower_the_middle_value_is_nonascii: 4096 1.00 108.5±0.97µs ? ?/sec 1.00 108.5±0.48µs ? ?/sec
lower_the_middle_value_is_nonascii: 8192 1.01 219.2±0.99µs ? ?/sec 1.00 217.0±0.91µs ? ?/sec
repeat 1073741824 times/repeat_string overflow [size=1024, repeat_times=1073741824] 1.00 799.2±0.60ns ? ?/sec 1.00 797.8±0.54ns ? ?/sec
repeat 1073741824 times/repeat_string overflow [size=4096, repeat_times=1073741824] 1.00 798.6±0.84ns ? ?/sec 1.00 801.5±0.94ns ? ?/sec
repeat 3 times/repeat_large_string [size=1024, repeat_times=3] 1.01 17.2±0.01µs ? ?/sec 1.00 17.0±0.02µs ? ?/sec
repeat 3 times/repeat_large_string [size=4096, repeat_times=3] 1.01 65.0±0.05µs ? ?/sec 1.00 64.6±0.11µs ? ?/sec
repeat 3 times/repeat_string [size=1024, repeat_times=3] 1.01 17.2±0.10µs ? ?/sec 1.00 17.0±0.11µs ? ?/sec
repeat 3 times/repeat_string [size=4096, repeat_times=3] 1.00 64.8±0.39µs ? ?/sec 1.00 64.6±0.40µs ? ?/sec
repeat 3 times/repeat_string_view [size=1024, repeat_times=3] 1.00 18.2±0.02µs ? ?/sec 1.02 18.6±0.01µs ? ?/sec
repeat 3 times/repeat_string_view [size=4096, repeat_times=3] 1.00 69.8±0.08µs ? ?/sec 1.03 71.8±0.11µs ? ?/sec
repeat 30 times/repeat_large_string [size=1024, repeat_times=30] 1.00 53.6±0.05µs ? ?/sec 1.00 53.7±0.04µs ? ?/sec
repeat 30 times/repeat_large_string [size=4096, repeat_times=30] 1.01 255.3±0.66µs ? ?/sec 1.00 251.6±0.15µs ? ?/sec
repeat 30 times/repeat_string [size=1024, repeat_times=30] 1.00 53.6±0.06µs ? ?/sec 1.01 54.0±0.10µs ? ?/sec
repeat 30 times/repeat_string [size=4096, repeat_times=30] 1.01 254.8±0.91µs ? ?/sec 1.00 251.7±0.72µs ? ?/sec
repeat 30 times/repeat_string_view [size=1024, repeat_times=30] 1.00 55.1±0.08µs ? ?/sec 1.02 56.4±0.02µs ? ?/sec
repeat 30 times/repeat_string_view [size=4096, repeat_times=30] 1.00 252.0±0.30µs ? ?/sec 1.04 260.9±0.76µs ? ?/sec
repeat/scalar_utf8 1.00 149.7±1.21ns ? ?/sec 1.02 152.0±5.26ns ? ?/sec
repeat/scalar_utf8view 1.00 149.3±1.10ns ? ?/sec 1.03 153.0±4.41ns ? ?/sec
replace size=1024/replace_string [size=1024, str_len=128, nulls=0.2] 1.04 48.0±0.41µs ? ?/sec 1.00 46.3±0.34µs ? ?/sec
replace size=1024/replace_string [size=1024, str_len=128, nulls=0] 1.03 63.8±0.64µs ? ?/sec 1.00 61.9±0.50µs ? ?/sec
replace size=1024/replace_string [size=1024, str_len=32, nulls=0.2] 1.02 23.0±0.03µs ? ?/sec 1.00 22.4±0.07µs ? ?/sec
replace size=1024/replace_string [size=1024, str_len=32, nulls=0] 1.00 28.5±0.06µs ? ?/sec 1.00 28.5±0.11µs ? ?/sec
replace size=1024/replace_string_ascii_single [size=1024, str_len=128, nulls=0.2] 1.02 11.9±0.01µs ? ?/sec 1.00 11.7±0.03µs ? ?/sec
replace size=1024/replace_string_ascii_single [size=1024, str_len=128, nulls=0] 1.01 12.9±0.01µs ? ?/sec 1.00 12.8±0.01µs ? ?/sec
replace size=1024/replace_string_ascii_single [size=1024, str_len=32, nulls=0.2] 1.01 9.0±0.03µs ? ?/sec 1.00 8.9±0.01µs ? ?/sec
replace size=1024/replace_string_ascii_single [size=1024, str_len=32, nulls=0] 1.00 9.6±0.01µs ? ?/sec 1.00 9.6±0.01µs ? ?/sec
replace size=1024/replace_string_view [size=1024, str_len=128, nulls=0.2] 1.02 48.9±0.25µs ? ?/sec 1.00 47.9±0.63µs ? ?/sec
replace size=1024/replace_string_view [size=1024, str_len=128, nulls=0] 1.02 63.3±0.25µs ? ?/sec 1.00 61.8±0.47µs ? ?/sec
replace size=1024/replace_string_view [size=1024, str_len=32, nulls=0.2] 1.02 23.5±0.07µs ? ?/sec 1.00 23.1±0.09µs ? ?/sec
replace size=1024/replace_string_view [size=1024, str_len=32, nulls=0] 1.01 28.6±0.04µs ? ?/sec 1.00 28.5±0.08µs ? ?/sec
replace size=4096/replace_string [size=4096, str_len=128, nulls=0.2] 1.03 245.2±0.59µs ? ?/sec 1.00 238.3±1.23µs ? ?/sec
replace size=4096/replace_string [size=4096, str_len=128, nulls=0] 1.00 299.5±0.66µs ? ?/sec 1.00 298.8±1.48µs ? ?/sec
replace size=4096/replace_string [size=4096, str_len=32, nulls=0.2] 1.04 111.4±0.28µs ? ?/sec 1.00 106.7±0.93µs ? ?/sec
replace size=4096/replace_string [size=4096, str_len=32, nulls=0] 1.00 137.9±0.46µs ? ?/sec 1.00 137.4±1.13µs ? ?/sec
replace size=4096/replace_string_ascii_single [size=4096, str_len=128, nulls=0.2] 1.00 39.7±0.01µs ? ?/sec 1.05 41.8±0.04µs ? ?/sec
replace size=4096/replace_string_ascii_single [size=4096, str_len=128, nulls=0] 1.00 44.2±0.05µs ? ?/sec 1.02 45.0±0.12µs ? ?/sec
replace size=4096/replace_string_ascii_single [size=4096, str_len=32, nulls=0.2] 1.00 28.4±0.07µs ? ?/sec 1.00 28.5±0.15µs ? ?/sec
replace size=4096/replace_string_ascii_single [size=4096, str_len=32, nulls=0] 1.00 30.6±0.01µs ? ?/sec 1.00 30.6±0.01µs ? ?/sec
replace size=4096/replace_string_view [size=4096, str_len=128, nulls=0.2] 1.02 247.8±0.68µs ? ?/sec 1.00 243.7±1.60µs ? ?/sec
replace size=4096/replace_string_view [size=4096, str_len=128, nulls=0] 1.02 300.9±0.68µs ? ?/sec 1.00 296.4±1.63µs ? ?/sec
replace size=4096/replace_string_view [size=4096, str_len=32, nulls=0.2] 1.02 113.7±0.33µs ? ?/sec 1.00 111.9±0.78µs ? ?/sec
replace size=4096/replace_string_view [size=4096, str_len=32, nulls=0] 1.01 136.8±0.49µs ? ?/sec 1.00 134.8±0.80µs ? ?/sec
replace_scalar from="ab" [size=8192, str_len=16] 1.00 163.5±0.51µs ? ?/sec 1.00 163.7±0.62µs ? ?/sec
replace_scalar from="ab" [size=8192, str_len=32] 1.00 161.2±1.63µs ? ?/sec 1.00 161.1±1.53µs ? ?/sec
replace_scalar from="ab" [size=8192, str_len=64] 1.01 174.6±1.72µs ? ?/sec 1.00 173.6±1.82µs ? ?/sec
replace_scalar from="the" [size=8192, str_len=16] 1.00 153.4±0.32µs ? ?/sec 1.00 153.3±0.52µs ? ?/sec
replace_scalar from="the" [size=8192, str_len=32] 1.00 156.4±2.95µs ? ?/sec 1.01 157.4±2.83µs ? ?/sec
replace_scalar from="the" [size=8192, str_len=64] 1.00 172.5±1.93µs ? ?/sec 1.00 171.6±2.12µs ? ?/sec
short strings (len <= 12)/btrim large_string [size=4096, len=12, remaining=8] 1.01 96.1±0.02µs ? ?/sec 1.00 95.3±0.02µs ? ?/sec
short strings (len <= 12)/btrim string [size=4096, len=12, remaining=8] 1.01 96.3±0.02µs ? ?/sec 1.00 95.3±0.02µs ? ?/sec
short strings (len <= 12)/btrim string_view [size=4096, len=12, remaining=8] 1.01 106.9±0.06µs ? ?/sec 1.00 106.3±0.08µs ? ?/sec
short strings (len <= 12)/ltrim large_string [size=4096, len=12, remaining=8] 1.00 52.8±0.01µs ? ?/sec 1.01 53.1±0.02µs ? ?/sec
short strings (len <= 12)/ltrim string [size=4096, len=12, remaining=8] 1.00 54.1±0.04µs ? ?/sec 1.00 54.1±0.06µs ? ?/sec
short strings (len <= 12)/ltrim string_view [size=4096, len=12, remaining=8] 1.00 62.4±0.07µs ? ?/sec 1.01 63.3±0.32µs ? ?/sec
short strings (len <= 12)/rtrim large_string [size=4096, len=12, remaining=8] 1.00 54.4±0.03µs ? ?/sec 1.00 54.3±0.02µs ? ?/sec
short strings (len <= 12)/rtrim string [size=4096, len=12, remaining=8] 1.00 53.3±0.08µs ? ?/sec 1.00 53.2±0.02µs ? ?/sec
short strings (len <= 12)/rtrim string_view [size=4096, len=12, remaining=8] 1.00 60.1±0.05µs ? ?/sec 1.00 60.2±0.03µs ? ?/sec
split_part/array_utf8_multi_char/pos_middle 1.00 430.0±1.90µs ? ?/sec 1.01 434.2±1.62µs ? ?/sec
split_part/array_utf8_single_char/pos_middle 1.00 441.4±1.87µs ? ?/sec 1.02 449.2±1.75µs ? ?/sec
split_part/scalar_utf8_long_strings/pos_middle 1.00 1829.7±0.76µs ? ?/sec 1.00 1830.5±0.84µs ? ?/sec
split_part/scalar_utf8_multi_char/pos_middle 1.00 340.5±0.37µs ? ?/sec 1.01 343.8±0.75µs ? ?/sec
split_part/scalar_utf8_single_char/pos_first 1.00 70.4±0.13µs ? ?/sec 1.00 70.2±0.21µs ? ?/sec
split_part/scalar_utf8_single_char/pos_middle 1.00 218.6±0.12µs ? ?/sec 1.00 218.5±0.14µs ? ?/sec
split_part/scalar_utf8_single_char/pos_negative 1.00 81.5±0.10µs ? ?/sec 1.00 81.7±0.17µs ? ?/sec
split_part/scalar_utf8view_long_parts/pos_middle 1.00 323.5±0.65µs ? ?/sec 1.00 323.0±0.26µs ? ?/sec
split_part/scalar_utf8view_very_long_parts/pos_first 1.01 113.3±0.38µs ? ?/sec 1.00 112.3±0.50µs ? ?/sec
starts_with_StringArray_array_prefix 1.01 42.7±0.94µs ? ?/sec 1.00 42.5±0.85µs ? ?/sec
starts_with_StringArray_scalar_prefix 1.03 12.4±0.09µs ? ?/sec 1.00 12.1±0.08µs ? ?/sec
starts_with_StringViewArray_array_prefix 1.00 55.3±0.50µs ? ?/sec 1.00 55.3±0.73µs ? ?/sec
starts_with_StringViewArray_scalar_prefix 1.00 14.9±0.04µs ? ?/sec 1.01 15.1±0.21µs ? ?/sec
starts_with_StringViewArray_scalar_strlen_128 1.01 15.1±0.04µs ? ?/sec 1.00 14.9±0.05µs ? ?/sec
starts_with_StringViewArray_scalar_strlen_32 1.01 15.0±0.01µs ? ?/sec 1.00 14.9±0.01µs ? ?/sec
starts_with_StringViewArray_scalar_strlen_512 1.00 15.1±0.01µs ? ?/sec 1.00 15.0±0.05µs ? ?/sec
starts_with_StringViewArray_scalar_strlen_8 1.01 15.1±0.01µs ? ?/sec 1.00 14.9±0.01µs ? ?/sec
to_hex size=1024/i32_random 1.00 8.2±0.08µs ? ?/sec 1.02 8.4±0.08µs ? ?/sec
to_hex size=1024/i64_large_values 1.00 8.6±0.04µs ? ?/sec 1.01 8.7±0.04µs ? ?/sec
to_hex size=1024/i64_random 1.00 8.6±0.04µs ? ?/sec 1.01 8.7±0.04µs ? ?/sec
to_hex size=4096/i32_random 1.00 31.6±0.34µs ? ?/sec 1.03 32.5±0.35µs ? ?/sec
to_hex size=4096/i64_large_values 1.00 33.4±0.18µs ? ?/sec 1.06 35.3±0.76µs ? ?/sec
to_hex size=4096/i64_random 1.00 33.4±0.22µs ? ?/sec 1.02 34.1±0.23µs ? ?/sec
to_hex size=8192/i32_random 1.00 63.5±0.55µs ? ?/sec 1.04 65.7±0.56µs ? ?/sec
to_hex size=8192/i64_large_values 1.00 68.7±0.64µs ? ?/sec 1.00 68.6±1.63µs ? ?/sec
to_hex size=8192/i64_random 1.00 66.8±0.40µs ? ?/sec 1.02 68.0±0.46µs ? ?/sec
to_hex/scalar_i32 1.01 109.8±1.17ns ? ?/sec 1.00 109.2±0.36ns ? ?/sec
to_hex/scalar_i64 1.00 110.2±0.65ns ? ?/sec 1.00 110.2±0.29ns ? ?/sec
trim spaces, heavy padding/btrim large_string [size=4096, len=120, pad=56] 1.00 165.0±0.04µs ? ?/sec 1.00 165.2±0.16µs ? ?/sec
trim spaces, heavy padding/btrim string [size=4096, len=120, pad=56] 1.00 164.2±0.02µs ? ?/sec 1.00 164.2±0.05µs ? ?/sec
trim spaces, heavy padding/btrim string_view [size=4096, len=120, pad=56] 1.00 177.4±0.04µs ? ?/sec 1.00 177.6±0.07µs ? ?/sec
trim spaces, heavy padding/ltrim large_string [size=4096, len=64, pad=56] 1.00 88.7±0.03µs ? ?/sec 1.00 88.8±0.02µs ? ?/sec
trim spaces, heavy padding/ltrim string [size=4096, len=64, pad=56] 1.00 89.0±0.11µs ? ?/sec 1.00 89.0±0.02µs ? ?/sec
trim spaces, heavy padding/ltrim string_view [size=4096, len=64, pad=56] 1.00 100.9±0.02µs ? ?/sec 1.01 101.6±0.12µs ? ?/sec
trim spaces, heavy padding/rtrim large_string [size=4096, len=64, pad=56] 1.00 89.4±0.02µs ? ?/sec 1.00 89.8±0.04µs ? ?/sec
trim spaces, heavy padding/rtrim string [size=4096, len=64, pad=56] 1.00 88.7±0.02µs ? ?/sec 1.00 89.0±0.23µs ? ?/sec
trim spaces, heavy padding/rtrim string_view [size=4096, len=64, pad=56] 1.00 104.3±0.04µs ? ?/sec 1.01 104.9±0.13µs ? ?/sec
trim spaces, long strings/btrim large_string [size=4096, len=68, pad=4] 1.00 31.7±0.01µs ? ?/sec 1.00 31.6±0.02µs ? ?/sec
trim spaces, long strings/btrim string [size=4096, len=68, pad=4] 1.00 32.2±0.01µs ? ?/sec 1.00 32.1±0.01µs ? ?/sec
trim spaces, long strings/btrim string_view [size=4096, len=68, pad=4] 1.00 37.2±0.03µs ? ?/sec 1.03 38.3±0.10µs ? ?/sec
trim spaces, long strings/ltrim large_string [size=4096, len=64, pad=4] 1.00 23.5±0.02µs ? ?/sec 1.00 23.4±0.01µs ? ?/sec
trim spaces, long strings/ltrim string [size=4096, len=64, pad=4] 1.00 23.6±0.01µs ? ?/sec 1.00 23.5±0.01µs ? ?/sec
trim spaces, long strings/ltrim string_view [size=4096, len=64, pad=4] 1.00 26.3±0.02µs ? ?/sec 1.04 27.3±0.02µs ? ?/sec
trim spaces, long strings/rtrim large_string [size=4096, len=64, pad=4] 1.00 23.2±0.04µs ? ?/sec 1.00 23.2±0.02µs ? ?/sec
trim spaces, long strings/rtrim string [size=4096, len=64, pad=4] 1.01 22.2±0.05µs ? ?/sec 1.00 22.1±0.01µs ? ?/sec
trim spaces, long strings/rtrim string_view [size=4096, len=64, pad=4] 1.00 27.7±0.01µs ? ?/sec 1.01 28.0±0.06µs ? ?/sec
trim spaces, short strings (len <= 12)/btrim large_string [size=4096, len=16, pad=4] 1.00 30.4±0.01µs ? ?/sec 1.00 30.5±0.01µs ? ?/sec
trim spaces, short strings (len <= 12)/btrim string [size=4096, len=16, pad=4] 1.00 30.9±0.01µs ? ?/sec 1.00 31.0±0.01µs ? ?/sec
trim spaces, short strings (len <= 12)/btrim string_view [size=4096, len=16, pad=4] 1.00 37.4±0.03µs ? ?/sec 1.01 38.0±0.17µs ? ?/sec
trim spaces, short strings (len <= 12)/ltrim large_string [size=4096, len=12, pad=4] 1.00 22.0±0.05µs ? ?/sec 1.00 21.9±0.02µs ? ?/sec
trim spaces, short strings (len <= 12)/ltrim string [size=4096, len=12, pad=4] 1.00 22.0±0.01µs ? ?/sec 1.00 22.1±0.01µs ? ?/sec
trim spaces, short strings (len <= 12)/ltrim string_view [size=4096, len=12, pad=4] 1.01 27.0±0.01µs ? ?/sec 1.00 26.8±0.04µs ? ?/sec
trim spaces, short strings (len <= 12)/rtrim large_string [size=4096, len=12, pad=4] 1.00 21.7±0.01µs ? ?/sec 1.00 21.7±0.01µs ? ?/sec
trim spaces, short strings (len <= 12)/rtrim string [size=4096, len=12, pad=4] 1.00 20.5±0.01µs ? ?/sec 1.01 20.7±0.01µs ? ?/sec
trim spaces, short strings (len <= 12)/rtrim string_view [size=4096, len=12, pad=4] 1.00 27.0±0.04µs ? ?/sec 1.00 27.1±0.06µs ? ?/sec
upper_unicode_large_utf8 1.00 428.6±0.58µs ? ?/sec 1.00 428.5±0.41µs ? ?/sec
upper_unicode_utf8 1.00 428.8±0.38µs ? ?/sec 1.00 429.3±1.53µs ? ?/sec
uuid 1.00 36.0±0.61µs ? ?/sec 1.00 35.9±0.82µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**string_expressions — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 2815.6s |
| Peak memory | 74.4 MiB |
| Avg memory | 40.4 MiB |
| CPU user | 3239.8s |
| CPU sys | 2.7s |
| Peak spill | 0 B |
**string_expressions — branch**
| Metric | Value |
|--------|-------|
| Wall time | 2755.6s |
| Peak memory | 52.5 MiB |
| Avg memory | 39.5 MiB |
| CPU user | 3232.7s |
| CPU sys | 2.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902
run benchmark array_slice string_to_array string_expressions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749253071
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5749235902-2525-f92wq 6.12.94+ #1 SMP Tue Aug 4 08:44:15 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark string_to_array
```
</details>
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749240933
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5749235902-2524-nc9nv 6.12.94+ #1 SMP Tue Aug 4 08:44:15 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark array_slice
```
</details>
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749252491
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5749235902-2526-5z8kl 6.12.94+ #1 SMP Tue Aug 4 08:44:15 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark string_expressions
```
</details>
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749315344
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark string_to_array
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group HEAD neilc_fix-sliced-builder-capacity
----- ---- ---------------------------------
string_to_array_columnar_delim/100 1.00 1101.8±16.57µs ? ?/sec 1.00 1096.5±13.29µs ? ?/sec
string_to_array_columnar_delim/20 1.00 239.3±3.40µs ? ?/sec 1.00 240.4±2.69µs ? ?/sec
string_to_array_columnar_delim/5 1.00 71.9±0.75µs ? ?/sec 1.00 72.1±0.59µs ? ?/sec
string_to_array_multi_char_delim/100 1.00 1229.2±5.01µs ? ?/sec 1.00 1225.2±5.06µs ? ?/sec
string_to_array_multi_char_delim/20 1.01 254.9±1.12µs ? ?/sec 1.00 252.5±1.14µs ? ?/sec
string_to_array_multi_char_delim/5 1.00 71.0±0.60µs ? ?/sec 1.00 70.8±0.53µs ? ?/sec
string_to_array_null_delim/100 1.00 435.8±24.12µs ? ?/sec 1.02 443.1±17.32µs ? ?/sec
string_to_array_null_delim/20 1.00 89.8±4.73µs ? ?/sec 1.02 91.5±3.65µs ? ?/sec
string_to_array_null_delim/5 1.00 25.8±0.39µs ? ?/sec 1.01 26.0±0.34µs ? ?/sec
string_to_array_single_char_delim/100 1.00 1018.4±0.87µs ? ?/sec 1.00 1017.5±1.14µs ? ?/sec
string_to_array_single_char_delim/20 1.01 209.3±0.13µs ? ?/sec 1.00 206.4±0.24µs ? ?/sec
string_to_array_single_char_delim/5 1.01 54.5±0.06µs ? ?/sec 1.00 53.7±0.24µs ? ?/sec
string_to_array_with_null_str/100 1.05 1247.3±5.48µs ? ?/sec 1.00 1184.3±5.56µs ? ?/sec
string_to_array_with_null_str/20 1.04 247.3±1.20µs ? ?/sec 1.00 237.4±0.95µs ? ?/sec
string_to_array_with_null_str/5 1.04 61.3±0.12µs ? ?/sec 1.00 59.1±0.22µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**string_to_array — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 340.1s |
| Peak memory | 30.7 MiB |
| Avg memory | 11.9 MiB |
| CPU user | 184.9s |
| CPU sys | 0.2s |
| Peak spill | 0 B |
**string_to_array — branch**
| Metric | Value |
|--------|-------|
| Wall time | 465.1s |
| Peak memory | 30.8 MiB |
| Avg memory | 8.6 MiB |
| CPU user | 185.3s |
| CPU sys | 0.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#discussion_r4057061133
##########
datafusion/functions-nested/src/utils.rs:
##########
@@ -602,4 +604,52 @@ mod tests {
assert!(needs_norm_scale(f64::INFINITY, 1));
assert!(needs_norm_scale(f64::NAN, 1));
}
+
+ /// Tests the array function supplied via `run` on small slices of a large
Review Comment:
sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#issuecomment-5749328216
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/25502#issuecomment-5749235902)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing neilc/fix-sliced-builder-capacity (812bf717f6826f7b2d275573a90e9cb6149d9ca6) to 24a7f13 (merge-base) [diff](https://github.com/apache/datafusion/compare/24a7f13f51230def9ceb0eeae920fd3840324e13..812bf717f6826f7b2d275573a90e9cb6149d9ca6)
<details><summary>Run configuration</summary>
```yaml
run benchmark array_slice
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group HEAD neilc_fix-sliced-builder-capacity
----- ---- ---------------------------------
array_slice: input List(Int64), array args 1.00 86.4±0.39ms ? ?/sec 1.03 88.7±0.27ms ? ?/sec
array_slice: input List(Int64), array args, no stride 1.00 41.2±0.48ms ? ?/sec 1.06 43.6±0.35ms ? ?/sec
array_slice: input List(Int64), scalar args, no stride 1.00 110.5±0.70ms ? ?/sec 1.05 115.6±0.68ms ? ?/sec
array_slice: input List(Int64), scalar args, stride=-1 1.00 328.2±1.71ms ? ?/sec 1.00 328.5±2.33ms ? ?/sec
array_slice: input List(Int64), scalar args, stride=-2 1.00 191.3±0.70ms ? ?/sec 1.00 190.5±0.83ms ? ?/sec
array_slice: input List(Int64), scalar args, stride=1 1.01 115.2±0.67ms ? ?/sec 1.00 114.4±0.65ms ? ?/sec
array_slice: input List(Int64), scalar args, stride=2 1.00 285.2±1.14ms ? ?/sec 1.00 285.4±1.19ms ? ?/sec
array_slice: input ListView(Int64), array args 1.00 95.4±0.60ms ? ?/sec 1.00 95.2±0.52ms ? ?/sec
array_slice: input ListView(Int64), array args, no stride 1.01 42.8±0.62ms ? ?/sec 1.00 42.2±0.41ms ? ?/sec
array_slice: input ListView(Int64), scalar args, no stride 1.00 112.9±0.73ms ? ?/sec 1.00 113.5±0.66ms ? ?/sec
array_slice: input ListView(Int64), scalar args, stride=-1 1.00 346.8±0.85ms ? ?/sec 1.00 345.9±1.22ms ? ?/sec
array_slice: input ListView(Int64), scalar args, stride=-2 1.00 203.7±0.73ms ? ?/sec 1.00 203.7±0.74ms ? ?/sec
array_slice: input ListView(Int64), scalar args, stride=1 1.01 124.9±0.58ms ? ?/sec 1.00 124.1±0.81ms ? ?/sec
array_slice: input ListView(Int64), scalar args, stride=2 1.00 298.5±0.92ms ? ?/sec 1.00 299.0±1.29ms ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**array_slice — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 650.1s |
| Peak memory | 728.1 MiB |
| Avg memory | 261.6 MiB |
| CPU user | 255.3s |
| CPU sys | 88.2s |
| Peak spill | 0 B |
**array_slice — branch**
| Metric | Value |
|--------|-------|
| Wall time | 480.1s |
| Peak memory | 727.7 MiB |
| Avg memory | 348.0 MiB |
| CPU user | 256.4s |
| CPU sys | 88.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway merged PR #25502:
URL: https://github.com/apache/datafusion/pull/25502
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Size builder capacity using visible array span, not total size [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #25502:
URL: https://github.com/apache/datafusion/pull/25502#discussion_r4056724677
##########
datafusion/functions-nested/src/array_add.rs:
##########
@@ -120,84 +116,29 @@ impl ScalarUDFImpl for ArrayAdd {
fn array_add_inner(args: &[ArrayRef]) -> Result<ArrayRef> {
let [array1, array2] = take_function_args("array_add", args)?;
+ let add = |a: f64, b: f64| a + b;
match (array1.data_type(), array2.data_type()) {
- (List(_), List(_)) => general_array_add::<i32>(array1, array2),
- (LargeList(_), LargeList(_)) => general_array_add::<i64>(array1, array2),
+ (List(_), List(_)) => {
+ array_math_binary_op::<i32, _>("array_add", array1, array2, add)
+ }
+ (LargeList(_), LargeList(_)) => {
+ array_math_binary_op::<i64, _>("array_add", array1, array2, add)
+ }
(arg_type1, arg_type2) => exec_err!(
"array_add received unexpected types after coercion: {arg_type1} and {arg_type2}"
),
}
}
-fn general_array_add<O: OffsetSizeTrait>(
Review Comment:
this is a nice reuse of existing function
##########
datafusion/functions-nested/src/utils.rs:
##########
@@ -602,4 +604,52 @@ mod tests {
assert!(needs_norm_scale(f64::INFINITY, 1));
assert!(needs_norm_scale(f64::NAN, 1));
}
+
+ /// Tests the array function supplied via `run` on small slices of a large
Review Comment:
this is a clever function
The name made it sound to me like it only checks capacity.
What do you think about calling it something like `test_sliced_list()` or something more generic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org