You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/06/23 19:27:00 UTC
[PR] IN LIST: reduce bitmap allocations [datafusion]
alamb opened a new pull request, #23136:
URL: https://github.com/apache/datafusion/pull/23136
## Which issue does this PR close?
- Part of #19241.
- Stacked on #23011.
## Rationale for this change
Avoid extra bitmap allocations while building IN LIST static-filter results.
## What changes are included in this PR?
- Pass ArrayRef into StaticFilter::contains so filters can move owned null buffers where possible.
- Use in-place boolean buffer operations when combining result and validity bitmaps.
## Are these changes tested?
- cargo fmt --all
- cargo test -p datafusion-physical-expr bitmap_filter_u8 --lib
- cargo test -p datafusion-physical-expr in_list_int_types --lib
## Are there any user-facing changes?
No. This is an internal performance cleanup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4782745971
run benchmark in_list_strategy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4793784618
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/23136#issuecomment-4793395665)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group HEAD codex_in-list-in-place-buffers
----- ---- ------------------------------
dictionary/i32/dict=10/list=16 1.00 7.6±0.01µs ? ?/sec 1.01 7.7±0.01µs ? ?/sec
dictionary/i32/dict=100/list=16 1.00 7.7±0.02µs ? ?/sec 1.02 7.9±0.01µs ? ?/sec
dictionary/i32/dict=100/list=16/NOT_IN 1.00 7.7±0.00µs ? ?/sec 1.01 7.9±0.01µs ? ?/sec
dictionary/i32/dict=100/list=4 1.00 7.7±0.00µs ? ?/sec 1.02 7.9±0.01µs ? ?/sec
dictionary/i32/dict=100/list=64 1.00 7.7±0.01µs ? ?/sec 1.01 7.9±0.01µs ? ?/sec
dictionary/i32/dict=1000/list=16 1.01 9.2±0.01µs ? ?/sec 1.00 9.1±0.01µs ? ?/sec
dictionary/utf8_long/dict=100/list=16 1.00 8.3±0.01µs ? ?/sec 1.01 8.4±0.01µs ? ?/sec
dictionary/utf8_short/dict=50/list=32 1.00 8.1±0.01µs ? ?/sec 1.01 8.2±0.01µs ? ?/sec
dictionary/utf8_short/dict=50/list=8 1.00 8.0±0.05µs ? ?/sec 1.01 8.1±0.01µs ? ?/sec
dictionary/utf8_short/dict=500/list=20 1.00 9.6±0.03µs ? ?/sec 1.02 9.9±0.01µs ? ?/sec
f32/large_list/list=64/match=0% 1.02 16.1±0.01µs ? ?/sec 1.00 15.9±0.02µs ? ?/sec
f32/large_list/list=64/match=50% 1.23 22.2±0.26µs ? ?/sec 1.00 18.0±0.04µs ? ?/sec
f32/small_list/list=32/match=0% 1.00 16.0±0.01µs ? ?/sec 1.01 16.3±0.01µs ? ?/sec
f32/small_list/list=32/match=50% 1.21 23.6±0.19µs ? ?/sec 1.00 19.6±0.33µs ? ?/sec
f32/small_list/list=4/match=0% 1.00 16.2±0.01µs ? ?/sec 1.02 16.5±0.05µs ? ?/sec
f32/small_list/list=4/match=50% 1.01 27.8±0.18µs ? ?/sec 1.00 27.6±3.71µs ? ?/sec
fixed_size_binary/fsb16/list=10000/match=0% 1.00 25.9±0.31µs ? ?/sec 1.13 29.3±0.48µs ? ?/sec
fixed_size_binary/fsb16/list=10000/match=50% 1.00 54.5±0.28µs ? ?/sec 1.00 54.7±0.40µs ? ?/sec
fixed_size_binary/fsb16/list=256/match=0% 1.00 24.9±0.75µs ? ?/sec 1.08 26.9±0.04µs ? ?/sec
fixed_size_binary/fsb16/list=256/match=50% 1.00 48.9±0.51µs ? ?/sec 1.02 49.8±0.19µs ? ?/sec
fixed_size_binary/fsb16/list=4/match=0% 1.00 23.2±0.68µs ? ?/sec 1.13 26.1±0.04µs ? ?/sec
fixed_size_binary/fsb16/list=4/match=50% 1.01 55.0±0.59µs ? ?/sec 1.00 54.3±0.41µs ? ?/sec
fixed_size_binary/fsb16/list=64/match=0% 1.00 22.8±0.03µs ? ?/sec 1.14 26.0±0.04µs ? ?/sec
fixed_size_binary/fsb16/list=64/match=50% 1.00 54.1±0.42µs ? ?/sec 1.02 55.1±0.41µs ? ?/sec
narrow_integer/i16/list=256/match=0% 1.00 12.2±0.01µs ? ?/sec 1.23 15.0±0.08µs ? ?/sec
narrow_integer/i16/list=256/match=50% 1.00 18.4±0.18µs ? ?/sec 1.26 23.2±0.54µs ? ?/sec
narrow_integer/i16/list=4/match=0% 1.00 12.7±0.05µs ? ?/sec 1.18 15.0±0.07µs ? ?/sec
narrow_integer/i16/list=4/match=50% 1.00 22.6±0.21µs ? ?/sec 1.07 24.1±0.52µs ? ?/sec
narrow_integer/i16/list=64/match=0% 1.00 12.4±0.14µs ? ?/sec 1.37 17.0±0.05µs ? ?/sec
narrow_integer/i16/list=64/match=50% 1.30 22.7±0.37µs ? ?/sec 1.00 17.5±1.97µs ? ?/sec
narrow_integer/u8/list=16/match=0% 1.00 5.2±0.00µs ? ?/sec 1.00 5.2±0.00µs ? ?/sec
narrow_integer/u8/list=16/match=50% 1.00 5.2±0.01µs ? ?/sec 1.00 5.2±0.00µs ? ?/sec
narrow_integer/u8/list=4/match=0% 1.00 5.2±0.04µs ? ?/sec 1.00 5.2±0.01µs ? ?/sec
narrow_integer/u8/list=4/match=50% 1.00 5.2±0.00µs ? ?/sec 1.00 5.2±0.00µs ? ?/sec
nulls/narrow_integer/u8/list=16/match=50%/nulls=20% 1.00 5.3±0.01µs ? ?/sec 1.00 5.3±0.00µs ? ?/sec
nulls/primitive/i32/large_list/list=64/match=50%/nulls=20% 1.00 22.8±0.32µs ? ?/sec 1.02 23.3±0.24µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20% 1.12 28.4±0.26µs ? ?/sec 1.00 25.5±0.24µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%/NOT_IN 1.14 25.2±0.25µs ? ?/sec 1.00 22.1±0.29µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=50% 1.41 21.7±0.06µs ? ?/sec 1.00 15.4±0.19µs ? ?/sec
nulls/utf8/long_24b/list=16/match=50%/nulls=20% 1.00 68.9±0.20µs ? ?/sec 1.03 70.8±0.56µs ? ?/sec
nulls/utf8/short_8b/list=16/match=50%/nulls=20% 1.00 58.5±0.40µs ? ?/sec 1.05 61.3±0.49µs ? ?/sec
nulls/utf8view/long_24b/list=16/match=50%/nulls=20% 1.00 78.8±0.16µs ? ?/sec 1.01 79.7±0.31µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=20% 1.00 40.9±0.09µs ? ?/sec 1.16 47.4±0.26µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=20%/NOT_IN 1.00 41.1±1.01µs ? ?/sec 1.16 47.5±0.21µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=50% 1.00 29.2±0.09µs ? ?/sec 1.31 38.1±0.21µs ? ?/sec
primitive/i32/large_list/list=256/match=0% 1.00 12.0±0.08µs ? ?/sec 1.03 12.4±0.02µs ? ?/sec
primitive/i32/large_list/list=256/match=50% 1.18 27.6±0.33µs ? ?/sec 1.00 23.5±0.18µs ? ?/sec
primitive/i32/large_list/list=64/match=0% 1.00 12.2±0.09µs ? ?/sec 1.07 13.0±0.03µs ? ?/sec
primitive/i32/large_list/list=64/match=50% 1.23 29.7±0.27µs ? ?/sec 1.00 24.2±0.22µs ? ?/sec
primitive/i32/small_list/list=16/match=50%/NOT_IN 1.21 27.8±0.37µs ? ?/sec 1.00 22.9±2.78µs ? ?/sec
primitive/i32/small_list/list=32/match=0% 1.00 11.9±0.01µs ? ?/sec 1.03 12.2±0.02µs ? ?/sec
primitive/i32/small_list/list=32/match=50% 1.33 28.2±0.27µs ? ?/sec 1.00 21.2±2.14µs ? ?/sec
primitive/i32/small_list/list=4/match=0% 1.00 12.1±0.01µs ? ?/sec 1.04 12.5±0.03µs ? ?/sec
primitive/i32/small_list/list=4/match=50% 1.28 32.8±0.28µs ? ?/sec 1.00 25.7±0.15µs ? ?/sec
primitive/i64/large_list/list=128/match=0% 1.00 12.1±0.05µs ? ?/sec 1.00 12.1±0.12µs ? ?/sec
primitive/i64/large_list/list=128/match=50% 1.00 18.5±0.14µs ? ?/sec 1.04 19.2±0.14µs ? ?/sec
primitive/i64/large_list/list=32/match=0% 1.08 13.1±0.04µs ? ?/sec 1.00 12.1±0.01µs ? ?/sec
primitive/i64/large_list/list=32/match=50% 1.00 20.7±0.52µs ? ?/sec 1.11 23.0±0.53µs ? ?/sec
primitive/i64/small_list/list=16/match=0% 1.00 12.0±0.01µs ? ?/sec 1.01 12.2±0.01µs ? ?/sec
primitive/i64/small_list/list=16/match=50% 1.00 23.7±0.33µs ? ?/sec 1.00 23.7±0.71µs ? ?/sec
primitive/i64/small_list/list=4/match=0% 1.00 12.1±0.01µs ? ?/sec 1.00 12.0±0.02µs ? ?/sec
primitive/i64/small_list/list=4/match=50% 1.02 22.9±0.48µs ? ?/sec 1.00 22.4±0.39µs ? ?/sec
timestamp_ns/large_list/list=32/match=0% 1.00 17.2±0.03µs ? ?/sec 1.18 20.4±0.03µs ? ?/sec
timestamp_ns/large_list/list=32/match=50% 1.00 29.6±0.65µs ? ?/sec 1.11 33.0±0.38µs ? ?/sec
timestamp_ns/small_list/list=16/match=0% 1.00 18.1±0.10µs ? ?/sec 1.12 20.3±0.03µs ? ?/sec
timestamp_ns/small_list/list=16/match=50% 1.00 29.0±0.41µs ? ?/sec 1.09 31.5±0.38µs ? ?/sec
timestamp_ns/small_list/list=4/match=0% 1.00 17.1±0.01µs ? ?/sec 1.20 20.4±0.02µs ? ?/sec
timestamp_ns/small_list/list=4/match=50% 1.00 29.7±0.14µs ? ?/sec 1.20 35.8±0.40µs ? ?/sec
utf8/long_24b/list=256/match=0% 1.00 33.7±0.04µs ? ?/sec 1.11 37.3±0.10µs ? ?/sec
utf8/long_24b/list=256/match=50% 1.01 71.4±0.31µs ? ?/sec 1.00 70.4±0.34µs ? ?/sec
utf8/long_24b/list=4/match=0% 1.00 34.2±0.04µs ? ?/sec 1.08 37.1±0.03µs ? ?/sec
utf8/long_24b/list=4/match=50% 1.04 73.9±0.57µs ? ?/sec 1.00 71.0±0.21µs ? ?/sec
utf8/long_24b/list=64/match=0% 1.00 34.3±0.27µs ? ?/sec 1.06 36.3±0.06µs ? ?/sec
utf8/long_24b/list=64/match=50% 1.05 72.6±0.64µs ? ?/sec 1.00 69.2±0.20µs ? ?/sec
utf8/mixed_len/list=16/match=0% 1.00 36.5±0.08µs ? ?/sec 1.12 40.9±0.10µs ? ?/sec
utf8/mixed_len/list=16/match=50% 1.07 106.5±1.69µs ? ?/sec 1.00 99.8±1.06µs ? ?/sec
utf8/mixed_len/list=64/match=0% 1.00 37.7±0.11µs ? ?/sec 1.09 41.2±0.10µs ? ?/sec
utf8/mixed_len/list=64/match=50% 1.16 115.4±0.45µs ? ?/sec 1.00 99.4±1.30µs ? ?/sec
utf8/shared_prefix/pfx=12/list=32/match=50% 1.03 72.7±0.44µs ? ?/sec 1.00 70.8±0.38µs ? ?/sec
utf8/short_8b/list=16/match=50%/NOT_IN 1.00 61.4±0.43µs ? ?/sec 1.02 62.9±0.30µs ? ?/sec
utf8/short_8b/list=256/match=0% 1.00 26.8±0.51µs ? ?/sec 1.10 29.3±0.13µs ? ?/sec
utf8/short_8b/list=256/match=50% 1.00 62.0±0.43µs ? ?/sec 1.02 63.5±0.29µs ? ?/sec
utf8/short_8b/list=4/match=0% 1.00 26.3±0.08µs ? ?/sec 1.11 29.3±0.03µs ? ?/sec
utf8/short_8b/list=4/match=50% 1.00 62.4±0.31µs ? ?/sec 1.03 64.3±0.79µs ? ?/sec
utf8/short_8b/list=64/match=0% 1.00 26.4±0.35µs ? ?/sec 1.10 29.1±0.03µs ? ?/sec
utf8/short_8b/list=64/match=50% 1.00 62.4±0.60µs ? ?/sec 1.00 62.7±0.69µs ? ?/sec
utf8view/len_12b/list=16/match=0% 1.00 17.8±0.02µs ? ?/sec 1.18 21.1±0.02µs ? ?/sec
utf8view/len_12b/list=16/match=50% 1.19 48.1±0.24µs ? ?/sec 1.00 40.3±0.16µs ? ?/sec
utf8view/len_12b/list=64/match=0% 1.00 17.9±0.04µs ? ?/sec 1.18 21.1±0.02µs ? ?/sec
utf8view/len_12b/list=64/match=50% 1.16 46.4±0.20µs ? ?/sec 1.00 40.0±0.28µs ? ?/sec
utf8view/long_24b/list=16/match=0% 1.00 41.5±0.05µs ? ?/sec 1.05 43.6±0.04µs ? ?/sec
utf8view/long_24b/list=16/match=50% 1.01 86.7±0.20µs ? ?/sec 1.00 85.6±0.17µs ? ?/sec
utf8view/long_24b/list=256/match=0% 1.00 40.6±0.43µs ? ?/sec 1.06 43.1±0.08µs ? ?/sec
utf8view/long_24b/list=256/match=50% 1.00 84.5±0.21µs ? ?/sec 1.02 86.6±0.27µs ? ?/sec
utf8view/long_24b/list=4/match=0% 1.00 40.3±0.05µs ? ?/sec 1.10 44.3±0.30µs ? ?/sec
utf8view/long_24b/list=4/match=50% 1.00 86.3±0.22µs ? ?/sec 1.00 86.3±0.31µs ? ?/sec
utf8view/long_24b/list=64/match=0% 1.00 40.3±0.05µs ? ?/sec 1.07 43.2±0.07µs ? ?/sec
utf8view/long_24b/list=64/match=50% 1.00 83.5±0.14µs ? ?/sec 1.04 86.7±1.24µs ? ?/sec
utf8view/mixed_len/list=16/match=0% 1.00 30.2±0.08µs ? ?/sec 1.12 33.9±0.07µs ? ?/sec
utf8view/mixed_len/list=16/match=50% 1.09 73.3±0.50µs ? ?/sec 1.00 67.2±1.58µs ? ?/sec
utf8view/mixed_len/list=64/match=0% 1.00 34.6±0.11µs ? ?/sec 1.08 37.5±0.15µs ? ?/sec
utf8view/mixed_len/list=64/match=50% 1.08 85.9±0.56µs ? ?/sec 1.00 79.6±0.37µs ? ?/sec
utf8view/shared_prefix/pfx=12/list=32/match=0% 1.00 42.4±0.78µs ? ?/sec 1.08 45.7±0.10µs ? ?/sec
utf8view/shared_prefix/pfx=12/list=32/match=50% 1.00 83.7±1.10µs ? ?/sec 1.05 87.9±0.31µs ? ?/sec
utf8view/shared_prefix/pfx=16/list=64/match=0% 1.00 40.3±0.09µs ? ?/sec 1.10 44.5±0.57µs ? ?/sec
utf8view/shared_prefix/pfx=16/list=64/match=50% 1.00 83.8±0.31µs ? ?/sec 1.03 86.0±0.37µs ? ?/sec
utf8view/shared_prefix/pfx=8/list=16/match=0% 1.00 29.9±0.05µs ? ?/sec 1.09 32.7±0.08µs ? ?/sec
utf8view/shared_prefix/pfx=8/list=16/match=50% 1.00 70.8±0.28µs ? ?/sec 1.05 74.3±0.16µs ? ?/sec
utf8view/short_8b/list=16/match=0% 1.00 17.6±0.02µs ? ?/sec 1.17 20.7±0.06µs ? ?/sec
utf8view/short_8b/list=16/match=50% 1.09 43.4±0.21µs ? ?/sec 1.00 39.9±0.36µs ? ?/sec
utf8view/short_8b/list=256/match=0% 1.00 17.8±0.05µs ? ?/sec 1.16 20.7±0.10µs ? ?/sec
utf8view/short_8b/list=256/match=50% 1.12 43.1±0.22µs ? ?/sec 1.00 38.5±0.32µs ? ?/sec
utf8view/short_8b/list=4/match=0% 1.00 17.8±0.03µs ? ?/sec 1.18 21.0±0.03µs ? ?/sec
utf8view/short_8b/list=4/match=50% 1.17 46.5±0.22µs ? ?/sec 1.00 39.7±0.29µs ? ?/sec
utf8view/short_8b/list=64/match=0% 1.00 18.0±0.03µs ? ?/sec 1.17 21.0±0.03µs ? ?/sec
utf8view/short_8b/list=64/match=50% 1.06 41.0±0.12µs ? ?/sec 1.00 38.5±0.27µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**in_list_strategy — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 1330.3s |
| Peak memory | 40.7 MiB |
| Avg memory | 28.5 MiB |
| CPU user | 1411.0s |
| CPU sys | 1.1s |
| Peak spill | 0 B |
**in_list_strategy — branch**
| Metric | Value |
|--------|-------|
| Wall time | 1320.3s |
| Peak memory | 41.9 MiB |
| Avg memory | 29.4 MiB |
| CPU user | 1416.5s |
| CPU sys | 1.2s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #23136: IN LIST: reduce bitmap allocations
URL: https://github.com/apache/datafusion/pull/23136
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4793966589
no obvious wins hwere, so closing it down
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4793414926
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/23136#issuecomment-4793395665)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4793395665-665-xjk4c 6.12.68+ #1 SMP Sat May 2 07:49:07 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing codex/in-list-in-place-buffers (500eccd9298482607c8436fbadb18f7493366ba8) to d0f8c34 (merge-base) [diff](https://github.com/apache/datafusion/compare/d0f8c340e888e42b151d5c8964692f37c9655251..500eccd9298482607c8436fbadb18f7493366ba8) using: in_list_strategy
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4782774862
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/23136#issuecomment-4782745971)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4782745971-634-28bfb 6.12.68+ #1 SMP Sat May 2 07:49:07 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing codex/in-list-in-place-buffers (7408fb5f0dfe9789b517910d7cc27acace2ad15c) to 60c407c (merge-base) [diff](https://github.com/apache/datafusion/compare/60c407cbfbdc589c5cf751dea07abf5b335c8e66..7408fb5f0dfe9789b517910d7cc27acace2ad15c) using: in_list_strategy
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4783081520
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/23136#issuecomment-4782745971)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group HEAD codex_in-list-in-place-buffers
----- ---- ------------------------------
dictionary/i32/dict=10/list=16 1.00 7.6±0.01µs ? ?/sec 1.01 7.7±0.03µs ? ?/sec
dictionary/i32/dict=100/list=16 1.00 7.7±0.01µs ? ?/sec 1.02 7.8±0.01µs ? ?/sec
dictionary/i32/dict=100/list=16/NOT_IN 1.00 7.7±0.01µs ? ?/sec 1.01 7.9±0.01µs ? ?/sec
dictionary/i32/dict=100/list=4 1.00 7.7±0.01µs ? ?/sec 1.01 7.8±0.01µs ? ?/sec
dictionary/i32/dict=100/list=64 1.00 7.8±0.01µs ? ?/sec 1.01 7.8±0.01µs ? ?/sec
dictionary/i32/dict=1000/list=16 1.00 9.0±0.01µs ? ?/sec 1.01 9.1±0.01µs ? ?/sec
dictionary/utf8_long/dict=100/list=16 1.00 8.3±0.01µs ? ?/sec 1.01 8.4±0.01µs ? ?/sec
dictionary/utf8_short/dict=50/list=32 1.00 8.1±0.01µs ? ?/sec 1.01 8.2±0.01µs ? ?/sec
dictionary/utf8_short/dict=50/list=8 1.00 8.0±0.06µs ? ?/sec 1.01 8.1±0.01µs ? ?/sec
dictionary/utf8_short/dict=500/list=20 1.00 9.7±0.01µs ? ?/sec 1.02 9.9±0.08µs ? ?/sec
f32/large_list/list=64/match=0% 1.02 16.1±0.01µs ? ?/sec 1.00 15.7±0.03µs ? ?/sec
f32/large_list/list=64/match=50% 1.00 23.5±0.28µs ? ?/sec 1.15 27.1±0.88µs ? ?/sec
f32/small_list/list=32/match=0% 1.01 16.0±0.01µs ? ?/sec 1.00 15.8±0.09µs ? ?/sec
f32/small_list/list=32/match=50% 1.00 26.8±0.26µs ? ?/sec 1.06 28.5±0.58µs ? ?/sec
f32/small_list/list=4/match=0% 1.00 16.0±0.02µs ? ?/sec 1.02 16.3±0.01µs ? ?/sec
f32/small_list/list=4/match=50% 1.00 27.0±0.17µs ? ?/sec 1.08 29.1±0.65µs ? ?/sec
fixed_size_binary/fsb16/list=10000/match=0% 1.00 26.9±0.03µs ? ?/sec 1.03 27.8±0.04µs ? ?/sec
fixed_size_binary/fsb16/list=10000/match=50% 1.07 57.6±0.70µs ? ?/sec 1.00 54.1±0.16µs ? ?/sec
fixed_size_binary/fsb16/list=256/match=0% 1.00 24.5±0.06µs ? ?/sec 1.10 26.8±0.04µs ? ?/sec
fixed_size_binary/fsb16/list=256/match=50% 1.05 52.0±0.76µs ? ?/sec 1.00 49.7±0.61µs ? ?/sec
fixed_size_binary/fsb16/list=4/match=0% 1.00 24.3±0.11µs ? ?/sec 1.06 25.9±0.06µs ? ?/sec
fixed_size_binary/fsb16/list=4/match=50% 1.01 54.2±0.89µs ? ?/sec 1.00 53.4±0.81µs ? ?/sec
fixed_size_binary/fsb16/list=64/match=0% 1.00 23.8±0.07µs ? ?/sec 1.11 26.4±0.59µs ? ?/sec
fixed_size_binary/fsb16/list=64/match=50% 1.04 54.8±0.60µs ? ?/sec 1.00 52.8±0.49µs ? ?/sec
narrow_integer/i16/list=256/match=0% 1.00 12.7±0.06µs ? ?/sec 1.19 15.1±0.15µs ? ?/sec
narrow_integer/i16/list=256/match=50% 1.14 18.2±0.12µs ? ?/sec 1.00 16.0±0.09µs ? ?/sec
narrow_integer/i16/list=4/match=0% 1.00 12.7±0.04µs ? ?/sec 1.20 15.1±0.04µs ? ?/sec
narrow_integer/i16/list=4/match=50% 1.00 21.5±0.13µs ? ?/sec 1.12 24.0±0.29µs ? ?/sec
narrow_integer/i16/list=64/match=0% 1.00 13.0±0.05µs ? ?/sec 1.20 15.6±0.06µs ? ?/sec
narrow_integer/i16/list=64/match=50% 1.00 18.1±0.17µs ? ?/sec 1.03 18.7±2.57µs ? ?/sec
narrow_integer/u8/list=16/match=0% 2.43 12.7±0.04µs ? ?/sec 1.00 5.2±0.01µs ? ?/sec
narrow_integer/u8/list=16/match=50% 4.79 25.0±0.13µs ? ?/sec 1.00 5.2±0.01µs ? ?/sec
narrow_integer/u8/list=4/match=0% 2.31 12.3±0.04µs ? ?/sec 1.00 5.3±0.07µs ? ?/sec
narrow_integer/u8/list=4/match=50% 5.02 26.2±0.54µs ? ?/sec 1.00 5.2±0.02µs ? ?/sec
nulls/narrow_integer/u8/list=16/match=50%/nulls=20% 4.77 25.3±0.19µs ? ?/sec 1.00 5.3±0.01µs ? ?/sec
nulls/primitive/i32/large_list/list=64/match=50%/nulls=20% 1.25 24.8±0.23µs ? ?/sec 1.00 19.7±2.87µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20% 1.87 28.5±0.34µs ? ?/sec 1.00 15.3±0.09µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%/NOT_IN 1.12 25.4±0.17µs ? ?/sec 1.00 22.7±0.18µs ? ?/sec
nulls/primitive/i32/small_list/list=16/match=50%/nulls=50% 1.56 22.1±0.08µs ? ?/sec 1.00 14.1±0.20µs ? ?/sec
nulls/utf8/long_24b/list=16/match=50%/nulls=20% 1.03 72.3±0.55µs ? ?/sec 1.00 69.9±0.26µs ? ?/sec
nulls/utf8/short_8b/list=16/match=50%/nulls=20% 1.04 63.2±0.49µs ? ?/sec 1.00 60.7±0.40µs ? ?/sec
nulls/utf8view/long_24b/list=16/match=50%/nulls=20% 1.12 86.6±0.24µs ? ?/sec 1.00 77.0±0.54µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=20% 1.00 45.4±0.21µs ? ?/sec 1.05 47.6±0.30µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=20%/NOT_IN 1.00 45.9±0.35µs ? ?/sec 1.04 47.5±0.29µs ? ?/sec
nulls/utf8view/short_8b/list=16/match=50%/nulls=50% 1.00 35.9±0.15µs ? ?/sec 1.05 37.7±0.17µs ? ?/sec
primitive/i32/large_list/list=256/match=0% 1.00 12.3±0.07µs ? ?/sec 1.00 12.3±0.07µs ? ?/sec
primitive/i32/large_list/list=256/match=50% 1.19 25.8±0.37µs ? ?/sec 1.00 21.6±2.24µs ? ?/sec
primitive/i32/large_list/list=64/match=0% 1.00 12.1±0.01µs ? ?/sec 1.18 14.3±0.04µs ? ?/sec
primitive/i32/large_list/list=64/match=50% 1.46 24.5±0.27µs ? ?/sec 1.00 16.8±0.44µs ? ?/sec
primitive/i32/small_list/list=16/match=50%/NOT_IN 1.00 21.2±0.21µs ? ?/sec 1.10 23.4±0.29µs ? ?/sec
primitive/i32/small_list/list=32/match=0% 1.00 12.0±0.03µs ? ?/sec 1.05 12.5±0.03µs ? ?/sec
primitive/i32/small_list/list=32/match=50% 1.03 25.4±0.45µs ? ?/sec 1.00 24.8±0.18µs ? ?/sec
primitive/i32/small_list/list=4/match=0% 1.00 12.0±0.01µs ? ?/sec 1.04 12.4±0.03µs ? ?/sec
primitive/i32/small_list/list=4/match=50% 1.28 33.4±0.20µs ? ?/sec 1.00 26.0±0.19µs ? ?/sec
primitive/i64/large_list/list=128/match=0% 1.08 13.0±0.09µs ? ?/sec 1.00 12.1±0.03µs ? ?/sec
primitive/i64/large_list/list=128/match=50% 1.07 19.8±0.15µs ? ?/sec 1.00 18.5±0.21µs ? ?/sec
primitive/i64/large_list/list=32/match=0% 1.00 12.1±0.03µs ? ?/sec 1.00 12.1±0.17µs ? ?/sec
primitive/i64/large_list/list=32/match=50% 1.37 24.4±0.21µs ? ?/sec 1.00 17.8±0.14µs ? ?/sec
primitive/i64/small_list/list=16/match=0% 1.00 12.1±0.04µs ? ?/sec 1.00 12.0±0.01µs ? ?/sec
primitive/i64/small_list/list=16/match=50% 1.00 20.4±0.13µs ? ?/sec 1.15 23.5±0.18µs ? ?/sec
primitive/i64/small_list/list=4/match=0% 1.00 12.1±0.01µs ? ?/sec 1.00 12.2±0.01µs ? ?/sec
primitive/i64/small_list/list=4/match=50% 1.03 23.0±0.27µs ? ?/sec 1.00 22.2±0.41µs ? ?/sec
timestamp_ns/large_list/list=32/match=0% 1.00 17.9±0.22µs ? ?/sec 1.13 20.2±0.02µs ? ?/sec
timestamp_ns/large_list/list=32/match=50% 1.04 33.8±0.56µs ? ?/sec 1.00 32.3±3.24µs ? ?/sec
timestamp_ns/small_list/list=16/match=0% 1.00 17.8±0.04µs ? ?/sec 1.13 20.2±0.02µs ? ?/sec
timestamp_ns/small_list/list=16/match=50% 1.21 36.1±0.41µs ? ?/sec 1.00 30.0±0.13µs ? ?/sec
timestamp_ns/small_list/list=4/match=0% 1.00 17.9±0.04µs ? ?/sec 1.13 20.2±0.01µs ? ?/sec
timestamp_ns/small_list/list=4/match=50% 1.05 32.2±0.24µs ? ?/sec 1.00 30.6±0.35µs ? ?/sec
utf8/long_24b/list=256/match=0% 1.00 35.0±0.15µs ? ?/sec 1.03 36.1±0.03µs ? ?/sec
utf8/long_24b/list=256/match=50% 1.03 71.9±0.32µs ? ?/sec 1.00 69.5±0.67µs ? ?/sec
utf8/long_24b/list=4/match=0% 1.00 34.6±0.04µs ? ?/sec 1.06 36.6±0.41µs ? ?/sec
utf8/long_24b/list=4/match=50% 1.01 72.6±0.25µs ? ?/sec 1.00 71.9±0.47µs ? ?/sec
utf8/long_24b/list=64/match=0% 1.00 35.7±0.04µs ? ?/sec 1.01 36.2±0.32µs ? ?/sec
utf8/long_24b/list=64/match=50% 1.04 71.7±0.36µs ? ?/sec 1.00 69.1±0.16µs ? ?/sec
utf8/mixed_len/list=16/match=0% 1.00 38.0±0.11µs ? ?/sec 1.07 40.8±0.13µs ? ?/sec
utf8/mixed_len/list=16/match=50% 1.07 107.6±0.65µs ? ?/sec 1.00 100.3±0.53µs ? ?/sec
utf8/mixed_len/list=64/match=0% 1.00 39.1±0.14µs ? ?/sec 1.05 41.0±0.31µs ? ?/sec
utf8/mixed_len/list=64/match=50% 1.20 118.9±0.69µs ? ?/sec 1.00 99.2±0.88µs ? ?/sec
utf8/shared_prefix/pfx=12/list=32/match=50% 1.02 71.2±0.33µs ? ?/sec 1.00 69.8±0.28µs ? ?/sec
utf8/short_8b/list=16/match=50%/NOT_IN 1.06 68.2±0.13µs ? ?/sec 1.00 64.5±0.22µs ? ?/sec
utf8/short_8b/list=256/match=0% 1.00 27.2±0.03µs ? ?/sec 1.07 29.1±0.02µs ? ?/sec
utf8/short_8b/list=256/match=50% 1.01 65.4±0.91µs ? ?/sec 1.00 64.6±0.57µs ? ?/sec
utf8/short_8b/list=4/match=0% 1.00 27.2±0.03µs ? ?/sec 1.08 29.5±0.02µs ? ?/sec
utf8/short_8b/list=4/match=50% 1.03 67.2±0.45µs ? ?/sec 1.00 65.5±0.23µs ? ?/sec
utf8/short_8b/list=64/match=0% 1.00 27.2±0.03µs ? ?/sec 1.07 28.9±0.02µs ? ?/sec
utf8/short_8b/list=64/match=50% 1.05 65.7±0.53µs ? ?/sec 1.00 62.8±0.57µs ? ?/sec
utf8view/len_12b/list=16/match=0% 1.00 18.6±0.03µs ? ?/sec 1.14 21.1±0.02µs ? ?/sec
utf8view/len_12b/list=16/match=50% 1.13 46.1±0.40µs ? ?/sec 1.00 40.6±0.21µs ? ?/sec
utf8view/len_12b/list=64/match=0% 1.00 18.7±0.03µs ? ?/sec 1.14 21.4±0.27µs ? ?/sec
utf8view/len_12b/list=64/match=50% 1.15 46.1±0.40µs ? ?/sec 1.00 40.1±0.40µs ? ?/sec
utf8view/long_24b/list=16/match=0% 1.00 40.9±0.07µs ? ?/sec 1.07 43.7±0.13µs ? ?/sec
utf8view/long_24b/list=16/match=50% 1.00 84.5±0.31µs ? ?/sec 1.03 86.9±0.25µs ? ?/sec
utf8view/long_24b/list=256/match=0% 1.00 41.1±0.16µs ? ?/sec 1.06 43.4±0.07µs ? ?/sec
utf8view/long_24b/list=256/match=50% 1.00 84.5±1.14µs ? ?/sec 1.03 86.6±0.19µs ? ?/sec
utf8view/long_24b/list=4/match=0% 1.00 41.0±0.04µs ? ?/sec 1.06 43.4±0.04µs ? ?/sec
utf8view/long_24b/list=4/match=50% 1.00 83.2±0.29µs ? ?/sec 1.04 86.7±0.11µs ? ?/sec
utf8view/long_24b/list=64/match=0% 1.00 40.7±0.28µs ? ?/sec 1.09 44.5±0.12µs ? ?/sec
utf8view/long_24b/list=64/match=50% 1.00 84.9±0.44µs ? ?/sec 1.01 85.6±0.22µs ? ?/sec
utf8view/mixed_len/list=16/match=0% 1.01 34.4±0.11µs ? ?/sec 1.00 33.9±0.10µs ? ?/sec
utf8view/mixed_len/list=16/match=50% 1.47 95.9±1.35µs ? ?/sec 1.00 65.3±0.36µs ? ?/sec
utf8view/mixed_len/list=64/match=0% 1.00 35.3±1.64µs ? ?/sec 1.06 37.3±0.15µs ? ?/sec
utf8view/mixed_len/list=64/match=50% 1.34 107.1±0.39µs ? ?/sec 1.00 79.9±0.65µs ? ?/sec
utf8view/shared_prefix/pfx=12/list=32/match=0% 1.00 43.0±0.03µs ? ?/sec 1.06 45.4±0.09µs ? ?/sec
utf8view/shared_prefix/pfx=12/list=32/match=50% 1.00 84.9±0.70µs ? ?/sec 1.03 87.7±0.34µs ? ?/sec
utf8view/shared_prefix/pfx=16/list=64/match=0% 1.00 40.4±0.05µs ? ?/sec 1.09 44.1±0.45µs ? ?/sec
utf8view/shared_prefix/pfx=16/list=64/match=50% 1.00 83.8±0.31µs ? ?/sec 1.04 87.0±1.33µs ? ?/sec
utf8view/shared_prefix/pfx=8/list=16/match=0% 1.00 30.0±0.05µs ? ?/sec 1.09 32.6±0.06µs ? ?/sec
utf8view/shared_prefix/pfx=8/list=16/match=50% 1.00 71.4±0.20µs ? ?/sec 1.03 73.4±0.37µs ? ?/sec
utf8view/short_8b/list=16/match=0% 1.00 18.1±0.03µs ? ?/sec 1.14 20.6±0.02µs ? ?/sec
utf8view/short_8b/list=16/match=50% 1.04 41.1±0.30µs ? ?/sec 1.00 39.4±0.28µs ? ?/sec
utf8view/short_8b/list=256/match=0% 1.00 18.3±0.04µs ? ?/sec 1.13 20.8±0.02µs ? ?/sec
utf8view/short_8b/list=256/match=50% 1.07 41.1±0.25µs ? ?/sec 1.00 38.4±0.17µs ? ?/sec
utf8view/short_8b/list=4/match=0% 1.00 18.6±0.03µs ? ?/sec 1.14 21.2±0.39µs ? ?/sec
utf8view/short_8b/list=4/match=50% 1.12 44.7±0.39µs ? ?/sec 1.00 39.8±0.14µs ? ?/sec
utf8view/short_8b/list=64/match=0% 1.00 18.4±0.05µs ? ?/sec 1.13 20.8±0.02µs ? ?/sec
utf8view/short_8b/list=64/match=50% 1.03 40.8±0.37µs ? ?/sec 1.00 39.5±0.57µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**in_list_strategy — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 1220.3s |
| Peak memory | 43.0 MiB |
| Avg memory | 32.0 MiB |
| CPU user | 1406.5s |
| CPU sys | 1.1s |
| Peak spill | 0 B |
**in_list_strategy — branch**
| Metric | Value |
|--------|-------|
| Wall time | 1260.3s |
| Peak memory | 41.2 MiB |
| Avg memory | 30.3 MiB |
| CPU user | 1419.2s |
| CPU sys | 1.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] IN LIST: reduce bitmap allocations [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #23136:
URL: https://github.com/apache/datafusion/pull/23136#issuecomment-4793395665
run benchmark in_list_strategy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org