You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Dandandan (via GitHub)" <gi...@apache.org> on 2026/04/13 17:43:59 UTC
[PR] feat: two-generation early emission for partial aggregation [datafusion]
Dandandan opened a new pull request, #21595:
URL: https://github.com/apache/datafusion/pull/21595
## Which issue does this PR close?
Related to improving GROUP BY performance on large datasets (e.g., ClickBench).
## Rationale for this change
The `AggregateExec(Partial)` accumulates ALL input rows into a single hash table before emitting. For high-cardinality GROUP BY queries (e.g., `GROUP BY URL` with 15M distinct values), the hash table grows far beyond CPU cache capacity, causing poor cache performance during hash table lookups.
## What changes are included in this PR?
**Two-generation early emission:** When the partial aggregate's hash table exceeds a configurable size threshold (default: 4MB), it uses a generational scheme:
1. **Hot table fills up** → emit the **cold batch** (previous generation) downstream
2. **Promote** current hot table state → cold batch
3. **Reset** hot hash table and continue reading input
This gives recurring groups a second chance to be merged locally before being sent downstream. At end-of-input, the remaining hot state and cold batch are concatenated and emitted together.
**Benefits:**
- Hash table stays small (~4MB) → fits in L3 cache → fewer cache misses
- Two generations means groups appearing across batch windows get merged locally
- Low-cardinality GROUP BY: threshold never exceeded → no change in behavior
- High-cardinality GROUP BY: hash table stays cache-friendly, still aggregates (unlike `skip_partial_aggregation` which stops aggregating entirely)
**New config:** `datafusion.execution.partial_aggregation_max_table_size` (default: 4MB, 0 to disable)
## Are these changes tested?
All existing aggregate SLT tests pass with the default threshold of 4MB enabled. The feature is designed to be transparent — correctness is maintained because `FinalPartitioned` correctly merges multiple partial emissions.
## Are there any user-facing changes?
New configuration option `datafusion.execution.partial_aggregation_max_table_size`. The default (4MB) is enabled automatically. Set to 0 to disable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238907792
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.27 / 4.66 ±6.56 / 17.78 ms │ 1.26 / 4.57 ±6.49 / 17.55 ms │ no change │
│ QQuery 1 │ 14.50 / 15.06 ±0.33 / 15.41 ms │ 14.61 / 15.20 ±0.31 / 15.46 ms │ no change │
│ QQuery 2 │ 44.79 / 45.46 ±0.37 / 45.81 ms │ 44.75 / 45.12 ±0.29 / 45.61 ms │ no change │
│ QQuery 3 │ 44.81 / 46.33 ±1.26 / 47.78 ms │ 45.00 / 47.38 ±1.37 / 48.96 ms │ no change │
│ QQuery 4 │ 311.10 / 316.98 ±6.48 / 329.53 ms │ 308.66 / 321.41 ±9.70 / 334.14 ms │ no change │
│ QQuery 5 │ 361.54 / 371.12 ±5.01 / 376.03 ms │ 702.97 / 725.57 ±12.37 / 736.70 ms │ 1.96x slower │
│ QQuery 6 │ 5.70 / 6.29 ±0.38 / 6.78 ms │ 5.64 / 6.55 ±0.91 / 8.16 ms │ no change │
│ QQuery 7 │ 17.41 / 17.75 ±0.22 / 18.07 ms │ 18.05 / 19.58 ±1.70 / 22.12 ms │ 1.10x slower │
│ QQuery 8 │ 450.70 / 456.71 ±5.50 / 466.25 ms │ 456.29 / 466.36 ±11.65 / 488.65 ms │ no change │
│ QQuery 9 │ 715.15 / 719.40 ±2.15 / 721.04 ms │ 8540.98 / 8652.81 ±71.14 / 8761.11 ms │ 12.03x slower │
│ QQuery 10 │ 95.13 / 98.27 ±3.48 / 105.01 ms │ 93.02 / 96.53 ±3.25 / 102.34 ms │ no change │
│ QQuery 11 │ 110.62 / 112.31 ±1.55 / 114.18 ms │ 105.61 / 109.58 ±2.71 / 113.77 ms │ no change │
│ QQuery 12 │ 361.06 / 367.14 ±5.68 / 377.84 ms │ 321.87 / 336.57 ±8.13 / 345.30 ms │ +1.09x faster │
│ QQuery 13 │ 480.46 / 500.82 ±17.11 / 530.98 ms │ 489.15 / 499.46 ±7.44 / 511.68 ms │ no change │
│ QQuery 14 │ 369.17 / 376.75 ±4.99 / 384.14 ms │ 334.97 / 352.59 ±12.50 / 367.65 ms │ +1.07x faster │
│ QQuery 15 │ 391.98 / 406.81 ±14.96 / 434.07 ms │ 400.51 / 408.19 ±6.06 / 418.91 ms │ no change │
│ QQuery 16 │ 760.59 / 789.63 ±23.98 / 833.57 ms │ 756.10 / 779.32 ±27.47 / 831.02 ms │ no change │
│ QQuery 17 │ 756.49 / 812.20 ±44.12 / 874.27 ms │ 738.34 / 756.23 ±13.33 / 769.66 ms │ +1.07x faster │
│ QQuery 18 │ 1452.08 / 1507.11 ±48.63 / 1568.16 ms │ 1207.73 / 1243.47 ±26.69 / 1266.44 ms │ +1.21x faster │
│ QQuery 19 │ 37.27 / 38.10 ±1.01 / 39.90 ms │ 38.83 / 41.59 ±2.68 / 45.05 ms │ 1.09x slower │
│ QQuery 20 │ 726.07 / 735.91 ±9.65 / 748.06 ms │ 732.88 / 740.42 ±11.28 / 762.20 ms │ no change │
│ QQuery 21 │ 777.57 / 780.48 ±1.75 / 782.66 ms │ 782.13 / 787.57 ±6.49 / 800.15 ms │ no change │
│ QQuery 22 │ 1173.98 / 1180.86 ±5.39 / 1189.62 ms │ 1171.18 / 1180.03 ±5.59 / 1188.60 ms │ no change │
│ QQuery 23 │ 3249.46 / 3292.44 ±29.21 / 3325.08 ms │ 3259.86 / 3277.78 ±19.36 / 3314.79 ms │ no change │
│ QQuery 24 │ 109.20 / 110.48 ±1.07 / 111.86 ms │ 105.95 / 108.42 ±1.83 / 110.71 ms │ no change │
│ QQuery 25 │ 144.21 / 146.89 ±1.97 / 149.44 ms │ 143.89 / 144.97 ±1.14 / 146.65 ms │ no change │
│ QQuery 26 │ 107.06 / 109.21 ±1.69 / 112.05 ms │ 106.21 / 108.45 ±1.45 / 110.74 ms │ no change │
│ QQuery 27 │ 865.53 / 875.48 ±8.75 / 889.86 ms │ 865.58 / 869.87 ±3.64 / 874.77 ms │ no change │
│ QQuery 28 │ 3351.93 / 3396.91 ±23.20 / 3415.11 ms │ 3582.54 / 3618.49 ±49.35 / 3715.36 ms │ 1.07x slower │
│ QQuery 29 │ 51.82 / 58.98 ±7.68 / 73.54 ms │ 51.96 / 55.25 ±2.01 / 57.99 ms │ +1.07x faster │
│ QQuery 30 │ 387.60 / 394.09 ±4.64 / 399.45 ms │ 343.32 / 364.21 ±20.33 / 399.01 ms │ +1.08x faster │
│ QQuery 31 │ 382.17 / 403.08 ±16.46 / 425.88 ms │ 381.36 / 393.97 ±9.11 / 408.88 ms │ no change │
│ QQuery 32 │ 1119.56 / 1156.52 ±33.74 / 1212.83 ms │ 1238.03 / 1326.41 ±47.99 / 1376.39 ms │ 1.15x slower │
│ QQuery 33 │ 1567.32 / 1588.34 ±13.11 / 1602.84 ms │ 1664.59 / 1718.42 ±43.01 / 1795.50 ms │ 1.08x slower │
│ QQuery 34 │ 1570.63 / 1586.44 ±15.50 / 1607.52 ms │ 1644.22 / 1661.30 ±9.27 / 1671.44 ms │ no change │
│ QQuery 35 │ 443.73 / 448.88 ±5.34 / 458.84 ms │ 405.96 / 417.94 ±6.84 / 425.58 ms │ +1.07x faster │
│ QQuery 36 │ 126.64 / 128.66 ±1.95 / 131.06 ms │ 98.55 / 100.34 ±2.17 / 104.60 ms │ +1.28x faster │
│ QQuery 37 │ 49.07 / 52.20 ±2.80 / 56.47 ms │ 48.11 / 49.69 ±1.26 / 51.78 ms │ no change │
│ QQuery 38 │ 77.80 / 79.65 ±0.93 / 80.32 ms │ 78.03 / 79.60 ±1.14 / 80.69 ms │ no change │
│ QQuery 39 │ 226.98 / 235.16 ±5.08 / 241.51 ms │ 176.30 / 180.24 ±3.53 / 186.77 ms │ +1.30x faster │
│ QQuery 40 │ 24.26 / 28.22 ±2.21 / 30.59 ms │ 24.25 / 26.64 ±1.80 / 29.00 ms │ +1.06x faster │
│ QQuery 41 │ 21.70 / 22.48 ±0.77 / 23.84 ms │ 21.39 / 22.40 ±1.06 / 24.39 ms │ no change │
│ QQuery 42 │ 20.40 / 21.39 ±0.61 / 22.22 ms │ 21.27 / 21.89 ±0.69 / 23.21 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 23841.66ms │
│ Total Time (early-group-emission) │ 32182.36ms │
│ Average Time (HEAD) │ 554.46ms │
│ Average Time (early-group-emission) │ 748.43ms │
│ Queries Faster │ 10 │
│ Queries Slower │ 7 │
│ Queries with No Change │ 26 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 120.5s |
| Peak memory | 42.6 GiB |
| Avg memory | 32.6 GiB |
| CPU user | 1135.1s |
| CPU sys | 90.2s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 161.9s |
| Peak memory | 39.0 GiB |
| Avg memory | 27.0 GiB |
| CPU user | 1567.2s |
| CPU sys | 84.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238495826
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238481303-1169-2w5nv 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2006bca0b43a2509f533c2c056ac77aab89b86d3) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2006bca0b43a2509f533c2c056ac77aab89b86d3) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4239135011
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.35 / 4.79 ±6.63 / 18.04 ms │ 1.19 / 4.45 ±6.37 / 17.19 ms │ +1.08x faster │
│ QQuery 1 │ 14.91 / 15.39 ±0.30 / 15.77 ms │ 14.42 / 14.89 ±0.49 / 15.83 ms │ no change │
│ QQuery 2 │ 45.87 / 46.13 ±0.23 / 46.46 ms │ 44.04 / 44.54 ±0.26 / 44.75 ms │ no change │
│ QQuery 3 │ 45.72 / 47.66 ±1.36 / 49.32 ms │ 43.24 / 44.68 ±1.05 / 46.42 ms │ +1.07x faster │
│ QQuery 4 │ 326.18 / 329.96 ±2.81 / 333.23 ms │ 287.43 / 308.28 ±21.45 / 341.67 ms │ +1.07x faster │
│ QQuery 5 │ 365.93 / 378.64 ±9.77 / 390.07 ms │ 744.24 / 816.41 ±73.08 / 907.24 ms │ 2.16x slower │
│ QQuery 6 │ 5.06 / 6.15 ±1.29 / 8.63 ms │ 5.78 / 6.89 ±1.04 / 8.80 ms │ 1.12x slower │
│ QQuery 7 │ 17.82 / 18.25 ±0.45 / 18.97 ms │ 18.15 / 20.53 ±2.29 / 24.37 ms │ 1.13x slower │
│ QQuery 8 │ 406.77 / 418.08 ±17.32 / 452.41 ms │ 444.99 / 462.64 ±9.42 / 471.12 ms │ 1.11x slower │
│ QQuery 9 │ 626.42 / 649.98 ±20.65 / 688.21 ms │ 439.34 / 446.77 ±7.29 / 459.78 ms │ +1.45x faster │
│ QQuery 10 │ 98.32 / 100.24 ±2.69 / 105.58 ms │ 88.12 / 89.14 ±0.64 / 90.02 ms │ +1.12x faster │
│ QQuery 11 │ 110.86 / 111.60 ±0.61 / 112.53 ms │ 99.37 / 103.84 ±2.67 / 106.82 ms │ +1.07x faster │
│ QQuery 12 │ 383.74 / 388.48 ±3.56 / 394.12 ms │ 302.76 / 314.41 ±19.41 / 353.11 ms │ +1.24x faster │
│ QQuery 13 │ 464.59 / 509.19 ±29.63 / 537.01 ms │ 496.81 / 515.32 ±11.47 / 527.32 ms │ no change │
│ QQuery 14 │ 350.35 / 364.48 ±10.59 / 375.52 ms │ 351.48 / 358.30 ±8.12 / 373.25 ms │ no change │
│ QQuery 15 │ 343.49 / 357.96 ±11.43 / 375.12 ms │ 375.83 / 404.40 ±23.03 / 435.69 ms │ 1.13x slower │
│ QQuery 16 │ 719.50 / 760.99 ±31.46 / 800.46 ms │ 696.01 / 725.07 ±18.59 / 753.83 ms │ no change │
│ QQuery 17 │ 757.16 / 781.81 ±14.13 / 800.38 ms │ 763.01 / 771.88 ±6.63 / 780.08 ms │ no change │
│ QQuery 18 │ 1363.91 / 1426.81 ±43.91 / 1484.29 ms │ 1071.81 / 1095.71 ±21.06 / 1126.27 ms │ +1.30x faster │
│ QQuery 19 │ 34.71 / 75.44 ±76.16 / 227.69 ms │ 35.20 / 36.25 ±1.63 / 39.50 ms │ +2.08x faster │
│ QQuery 20 │ 720.57 / 746.46 ±32.45 / 810.29 ms │ 709.92 / 715.09 ±6.54 / 727.89 ms │ no change │
│ QQuery 21 │ 785.42 / 790.46 ±3.98 / 796.56 ms │ 769.39 / 778.92 ±7.34 / 791.09 ms │ no change │
│ QQuery 22 │ 1126.84 / 1132.37 ±8.87 / 1149.96 ms │ 1124.07 / 1155.00 ±25.78 / 1186.86 ms │ no change │
│ QQuery 23 │ 3087.58 / 3146.18 ±72.50 / 3288.68 ms │ 3052.37 / 3103.53 ±77.97 / 3258.49 ms │ no change │
│ QQuery 24 │ 108.03 / 109.82 ±1.53 / 112.56 ms │ 108.99 / 111.39 ±2.40 / 114.86 ms │ no change │
│ QQuery 25 │ 143.71 / 146.16 ±1.30 / 147.31 ms │ 142.10 / 145.11 ±2.10 / 148.57 ms │ no change │
│ QQuery 26 │ 106.01 / 108.76 ±2.82 / 113.63 ms │ 107.74 / 110.35 ±2.52 / 114.44 ms │ no change │
│ QQuery 27 │ 850.21 / 862.46 ±15.74 / 893.23 ms │ 846.33 / 860.77 ±14.73 / 887.13 ms │ no change │
│ QQuery 28 │ 3286.94 / 3378.21 ±50.28 / 3418.40 ms │ 3425.02 / 3554.94 ±73.86 / 3630.23 ms │ 1.05x slower │
│ QQuery 29 │ 50.91 / 55.49 ±4.76 / 63.98 ms │ 51.90 / 57.69 ±6.87 / 70.18 ms │ no change │
│ QQuery 30 │ 363.07 / 369.34 ±5.75 / 377.46 ms │ 354.41 / 374.62 ±18.83 / 408.27 ms │ no change │
│ QQuery 31 │ 361.63 / 377.83 ±9.67 / 386.06 ms │ 364.83 / 381.23 ±13.72 / 400.09 ms │ no change │
│ QQuery 32 │ 1208.39 / 1256.99 ±42.94 / 1309.45 ms │ 1034.98 / 1043.61 ±6.87 / 1053.91 ms │ +1.20x faster │
│ QQuery 33 │ 1456.56 / 1681.09 ±121.85 / 1786.67 ms │ 1526.56 / 1625.12 ±77.84 / 1702.06 ms │ no change │
│ QQuery 34 │ 1443.36 / 1450.91 ±6.87 / 1459.17 ms │ 1487.41 / 1543.74 ±50.46 / 1636.60 ms │ 1.06x slower │
│ QQuery 35 │ 375.87 / 382.75 ±5.07 / 390.90 ms │ 356.94 / 368.45 ±8.83 / 379.08 ms │ no change │
│ QQuery 36 │ 118.54 / 120.59 ±1.28 / 121.75 ms │ 93.54 / 97.19 ±2.45 / 99.59 ms │ +1.24x faster │
│ QQuery 37 │ 48.83 / 50.50 ±1.26 / 52.14 ms │ 45.24 / 46.06 ±0.86 / 47.62 ms │ +1.10x faster │
│ QQuery 38 │ 73.42 / 75.30 ±1.46 / 76.74 ms │ 74.35 / 75.14 ±0.74 / 76.37 ms │ no change │
│ QQuery 39 │ 202.90 / 215.56 ±10.84 / 234.49 ms │ 165.49 / 169.44 ±3.17 / 173.17 ms │ +1.27x faster │
│ QQuery 40 │ 23.81 / 25.52 ±0.89 / 26.38 ms │ 22.42 / 24.81 ±1.99 / 27.87 ms │ no change │
│ QQuery 41 │ 19.19 / 20.07 ±0.65 / 21.06 ms │ 19.01 / 21.31 ±2.20 / 25.49 ms │ 1.06x slower │
│ QQuery 42 │ 19.03 / 19.59 ±0.30 / 19.93 ms │ 18.82 / 19.38 ±0.37 / 19.77 ms │ no change │
└───────────┴────────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 23314.42ms │
│ Total Time (early-group-emission) │ 22967.24ms │
│ Average Time (HEAD) │ 542.20ms │
│ Average Time (early-group-emission) │ 534.12ms │
│ Queries Faster │ 13 │
│ Queries Slower │ 8 │
│ Queries with No Change │ 22 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 117.7s |
| Peak memory | 41.9 GiB |
| Avg memory | 29.2 GiB |
| CPU user | 1096.8s |
| CPU sys | 100.1s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 115.8s |
| Peak memory | 40.0 GiB |
| Avg memory | 30.8 GiB |
| CPU user | 1118.6s |
| CPU sys | 68.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241510771
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241504219-1192-tzbzd 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238957125
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238944023-1179-spwms 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (1390b5ece2b27816c913beb8b77aacf335207181) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..1390b5ece2b27816c913beb8b77aacf335207181) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241507297
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241497116-1189-vpb5q 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241511007
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241504219-1191-fpwcr 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241683717
run benchmark clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4239066018
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.19 / 4.41 ±6.35 / 17.10 ms │ 1.21 / 4.43 ±6.33 / 17.09 ms │ no change │
│ QQuery 1 │ 14.05 / 14.36 ±0.21 / 14.66 ms │ 14.21 / 14.51 ±0.39 / 15.27 ms │ no change │
│ QQuery 2 │ 43.59 / 44.08 ±0.38 / 44.65 ms │ 43.46 / 43.73 ±0.22 / 44.09 ms │ no change │
│ QQuery 3 │ 41.51 / 43.84 ±2.29 / 47.46 ms │ 41.47 / 44.47 ±3.10 / 48.80 ms │ no change │
│ QQuery 4 │ 287.16 / 291.09 ±4.15 / 298.97 ms │ 287.50 / 300.37 ±11.09 / 317.89 ms │ no change │
│ QQuery 5 │ 340.07 / 344.68 ±3.50 / 349.45 ms │ 667.21 / 703.23 ±57.08 / 816.72 ms │ 2.04x slower │
│ QQuery 6 │ 5.45 / 5.88 ±0.61 / 7.10 ms │ 5.37 / 5.79 ±0.59 / 6.93 ms │ no change │
│ QQuery 7 │ 16.65 / 17.14 ±0.89 / 18.92 ms │ 16.68 / 19.04 ±3.29 / 25.46 ms │ 1.11x slower │
│ QQuery 8 │ 412.30 / 428.08 ±15.97 / 450.90 ms │ 430.34 / 441.30 ±7.61 / 448.84 ms │ no change │
│ QQuery 9 │ 620.14 / 634.89 ±10.82 / 650.40 ms │ 633.77 / 646.18 ±9.96 / 664.33 ms │ no change │
│ QQuery 10 │ 92.72 / 93.97 ±1.20 / 96.21 ms │ 88.45 / 91.78 ±2.68 / 95.96 ms │ no change │
│ QQuery 11 │ 104.52 / 106.02 ±1.04 / 107.17 ms │ 100.03 / 103.20 ±2.55 / 106.34 ms │ no change │
│ QQuery 12 │ 334.56 / 337.44 ±2.22 / 340.80 ms │ 307.79 / 314.01 ±4.78 / 321.21 ms │ +1.07x faster │
│ QQuery 13 │ 446.99 / 461.93 ±10.95 / 479.25 ms │ 452.21 / 469.71 ±13.86 / 485.62 ms │ no change │
│ QQuery 14 │ 339.89 / 348.73 ±4.47 / 351.66 ms │ 314.05 / 328.27 ±9.21 / 340.68 ms │ +1.06x faster │
│ QQuery 15 │ 337.28 / 359.82 ±18.52 / 389.46 ms │ 373.89 / 384.10 ±7.54 / 394.76 ms │ 1.07x slower │
│ QQuery 16 │ 695.99 / 723.24 ±21.98 / 760.18 ms │ 692.63 / 714.52 ±19.28 / 743.88 ms │ no change │
│ QQuery 17 │ 701.08 / 709.07 ±6.94 / 721.38 ms │ 688.04 / 697.83 ±7.42 / 710.44 ms │ no change │
│ QQuery 18 │ 1385.44 / 1458.11 ±43.18 / 1504.57 ms │ 1119.06 / 1158.95 ±20.22 / 1172.59 ms │ +1.26x faster │
│ QQuery 19 │ 35.14 / 39.71 ±6.51 / 52.55 ms │ 35.02 / 36.91 ±1.44 / 38.62 ms │ +1.08x faster │
│ QQuery 20 │ 708.34 / 728.83 ±19.25 / 755.78 ms │ 712.05 / 718.45 ±4.46 / 723.54 ms │ no change │
│ QQuery 21 │ 753.90 / 756.99 ±2.32 / 760.58 ms │ 760.73 / 771.11 ±10.29 / 789.80 ms │ no change │
│ QQuery 22 │ 1119.54 / 1129.75 ±7.54 / 1142.72 ms │ 1127.43 / 1129.93 ±2.16 / 1133.52 ms │ no change │
│ QQuery 23 │ 3077.54 / 3110.76 ±26.00 / 3149.22 ms │ 3051.78 / 3070.93 ±18.75 / 3103.14 ms │ no change │
│ QQuery 24 │ 98.48 / 100.78 ±1.68 / 103.55 ms │ 100.60 / 104.83 ±3.71 / 111.49 ms │ no change │
│ QQuery 25 │ 138.56 / 139.69 ±1.13 / 141.67 ms │ 138.77 / 140.68 ±1.27 / 142.16 ms │ no change │
│ QQuery 26 │ 98.67 / 102.41 ±2.50 / 105.12 ms │ 98.78 / 103.46 ±3.14 / 106.77 ms │ no change │
│ QQuery 27 │ 847.89 / 853.74 ±5.83 / 863.91 ms │ 853.02 / 858.66 ±2.93 / 860.73 ms │ no change │
│ QQuery 28 │ 3276.37 / 3294.57 ±13.14 / 3310.48 ms │ 3420.53 / 3528.44 ±66.91 / 3606.54 ms │ 1.07x slower │
│ QQuery 29 │ 49.74 / 55.29 ±5.14 / 62.21 ms │ 50.95 / 57.03 ±6.19 / 68.41 ms │ no change │
│ QQuery 30 │ 358.30 / 365.84 ±5.05 / 372.61 ms │ 352.06 / 356.64 ±3.21 / 362.07 ms │ no change │
│ QQuery 31 │ 362.59 / 373.90 ±10.26 / 390.72 ms │ 356.00 / 372.15 ±17.94 / 406.26 ms │ no change │
│ QQuery 32 │ 1210.79 / 1247.49 ±29.56 / 1292.81 ms │ 1108.46 / 1176.99 ±50.55 / 1254.51 ms │ +1.06x faster │
│ QQuery 33 │ 1505.47 / 1558.31 ±29.26 / 1585.22 ms │ 1553.82 / 1593.27 ±35.11 / 1636.61 ms │ no change │
│ QQuery 34 │ 1498.13 / 1513.60 ±12.54 / 1528.69 ms │ 1516.59 / 1551.85 ±31.39 / 1594.73 ms │ no change │
│ QQuery 35 │ 371.99 / 380.20 ±6.55 / 386.25 ms │ 371.31 / 376.08 ±4.72 / 385.12 ms │ no change │
│ QQuery 36 │ 118.29 / 120.97 ±1.88 / 123.02 ms │ 94.94 / 96.49 ±1.16 / 97.94 ms │ +1.25x faster │
│ QQuery 37 │ 47.07 / 47.96 ±0.98 / 49.79 ms │ 45.62 / 45.93 ±0.28 / 46.45 ms │ no change │
│ QQuery 38 │ 75.12 / 76.25 ±0.82 / 77.22 ms │ 76.34 / 77.44 ±1.08 / 78.93 ms │ no change │
│ QQuery 39 │ 210.31 / 214.83 ±2.97 / 218.67 ms │ 166.97 / 171.25 ±3.35 / 176.30 ms │ +1.25x faster │
│ QQuery 40 │ 23.33 / 24.24 ±1.16 / 26.37 ms │ 24.51 / 25.20 ±0.48 / 25.86 ms │ no change │
│ QQuery 41 │ 20.27 / 20.98 ±0.58 / 22.01 ms │ 19.43 / 20.71 ±1.14 / 22.71 ms │ no change │
│ QQuery 42 │ 19.68 / 19.93 ±0.27 / 20.41 ms │ 19.18 / 20.04 ±0.63 / 20.83 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22703.81ms │
│ Total Time (early-group-emission) │ 22889.87ms │
│ Average Time (HEAD) │ 528.00ms │
│ Average Time (early-group-emission) │ 532.32ms │
│ Queries Faster │ 7 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 32 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 114.6s |
| Peak memory | 37.5 GiB |
| Avg memory | 27.0 GiB |
| CPU user | 1067.1s |
| CPU sys | 98.0s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 115.3s |
| Peak memory | 34.2 GiB |
| Avg memory | 25.9 GiB |
| CPU user | 1090.3s |
| CPU sys | 78.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238961704
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238944023-1178-rkg7v 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (1390b5ece2b27816c913beb8b77aacf335207181) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..1390b5ece2b27816c913beb8b77aacf335207181) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241757014
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241683717)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.20 / 4.50 ±6.45 / 17.41 ms │ 1.19 / 4.42 ±6.33 / 17.08 ms │ no change │
│ QQuery 1 │ 14.47 / 14.82 ±0.35 / 15.49 ms │ 14.07 / 14.54 ±0.26 / 14.87 ms │ no change │
│ QQuery 2 │ 44.36 / 45.02 ±0.49 / 45.66 ms │ 44.32 / 44.69 ±0.25 / 45.07 ms │ no change │
│ QQuery 3 │ 41.44 / 42.37 ±1.13 / 44.35 ms │ 43.86 / 46.04 ±1.70 / 47.96 ms │ 1.09x slower │
│ QQuery 4 │ 285.27 / 292.96 ±7.83 / 306.97 ms │ 295.08 / 305.99 ±7.04 / 316.98 ms │ no change │
│ QQuery 5 │ 342.85 / 344.85 ±1.94 / 348.12 ms │ 341.63 / 366.76 ±18.67 / 392.22 ms │ 1.06x slower │
│ QQuery 6 │ 5.57 / 6.14 ±0.62 / 7.28 ms │ 4.98 / 5.73 ±0.64 / 6.76 ms │ +1.07x faster │
│ QQuery 7 │ 16.65 / 16.97 ±0.17 / 17.12 ms │ 16.49 / 17.06 ±1.03 / 19.11 ms │ no change │
│ QQuery 8 │ 401.90 / 421.07 ±11.30 / 434.57 ms │ 418.30 / 428.75 ±5.86 / 434.46 ms │ no change │
│ QQuery 9 │ 636.20 / 654.14 ±10.18 / 665.06 ms │ 439.63 / 443.62 ±3.44 / 447.81 ms │ +1.47x faster │
│ QQuery 10 │ 94.19 / 95.88 ±1.15 / 97.54 ms │ 93.30 / 95.72 ±2.00 / 98.64 ms │ no change │
│ QQuery 11 │ 107.36 / 107.67 ±0.37 / 108.38 ms │ 106.37 / 108.60 ±1.44 / 110.64 ms │ no change │
│ QQuery 12 │ 336.64 / 344.03 ±8.81 / 360.28 ms │ 336.10 / 352.48 ±13.84 / 370.54 ms │ no change │
│ QQuery 13 │ 452.77 / 468.59 ±10.52 / 478.55 ms │ 492.16 / 502.64 ±8.96 / 519.22 ms │ 1.07x slower │
│ QQuery 14 │ 344.66 / 348.88 ±3.29 / 353.02 ms │ 343.84 / 351.43 ±5.01 / 358.64 ms │ no change │
│ QQuery 15 │ 355.30 / 371.69 ±15.50 / 400.29 ms │ 332.33 / 341.33 ±8.02 / 353.17 ms │ +1.09x faster │
│ QQuery 16 │ 711.89 / 724.46 ±16.25 / 756.15 ms │ 692.72 / 721.60 ±24.24 / 755.37 ms │ no change │
│ QQuery 17 │ 713.35 / 717.07 ±4.03 / 724.87 ms │ 658.93 / 688.01 ±31.68 / 744.74 ms │ no change │
│ QQuery 18 │ 1425.50 / 1476.72 ±33.21 / 1514.64 ms │ 1233.55 / 1302.35 ±38.62 / 1342.87 ms │ +1.13x faster │
│ QQuery 19 │ 37.25 / 40.19 ±5.51 / 51.21 ms │ 36.47 / 43.24 ±10.08 / 62.89 ms │ 1.08x slower │
│ QQuery 20 │ 711.07 / 736.09 ±21.22 / 765.42 ms │ 722.95 / 732.72 ±11.69 / 755.66 ms │ no change │
│ QQuery 21 │ 761.12 / 766.37 ±5.18 / 773.02 ms │ 761.07 / 766.46 ±2.99 / 769.55 ms │ no change │
│ QQuery 22 │ 1129.56 / 1135.20 ±3.62 / 1139.98 ms │ 1131.31 / 1138.17 ±5.63 / 1147.42 ms │ no change │
│ QQuery 23 │ 3099.11 / 3124.57 ±21.36 / 3162.90 ms │ 3075.91 / 3109.09 ±25.14 / 3151.05 ms │ no change │
│ QQuery 24 │ 101.95 / 105.71 ±4.32 / 113.94 ms │ 100.03 / 103.82 ±2.16 / 106.02 ms │ no change │
│ QQuery 25 │ 137.12 / 141.45 ±2.33 / 144.11 ms │ 136.69 / 139.98 ±2.88 / 143.75 ms │ no change │
│ QQuery 26 │ 101.13 / 103.92 ±2.11 / 106.76 ms │ 100.44 / 101.77 ±1.26 / 103.55 ms │ no change │
│ QQuery 27 │ 848.88 / 852.46 ±3.39 / 858.56 ms │ 850.91 / 860.20 ±9.45 / 878.33 ms │ no change │
│ QQuery 28 │ 3283.12 / 3307.14 ±13.43 / 3321.27 ms │ 3318.30 / 3351.09 ±35.09 / 3416.65 ms │ no change │
│ QQuery 29 │ 51.21 / 54.20 ±3.56 / 60.52 ms │ 51.62 / 54.09 ±3.61 / 61.05 ms │ no change │
│ QQuery 30 │ 362.78 / 370.22 ±5.14 / 376.96 ms │ 337.92 / 357.06 ±18.63 / 389.81 ms │ no change │
│ QQuery 31 │ 370.26 / 378.20 ±7.81 / 392.55 ms │ 363.58 / 373.18 ±6.65 / 383.57 ms │ no change │
│ QQuery 32 │ 1185.98 / 1257.18 ±42.77 / 1316.50 ms │ 1159.57 / 1233.37 ±37.67 / 1260.20 ms │ no change │
│ QQuery 33 │ 1524.62 / 1581.61 ±33.17 / 1616.34 ms │ 1552.48 / 1618.04 ±79.59 / 1771.75 ms │ no change │
│ QQuery 34 │ 1484.92 / 1535.30 ±33.21 / 1588.75 ms │ 1542.27 / 1562.12 ±15.49 / 1583.72 ms │ no change │
│ QQuery 35 │ 381.79 / 385.53 ±2.45 / 388.30 ms │ 312.69 / 337.30 ±13.30 / 350.37 ms │ +1.14x faster │
│ QQuery 36 │ 119.24 / 121.93 ±1.92 / 124.26 ms │ 113.49 / 115.58 ±1.90 / 118.42 ms │ +1.06x faster │
│ QQuery 37 │ 48.19 / 50.98 ±1.47 / 52.19 ms │ 47.38 / 49.50 ±1.15 / 50.82 ms │ no change │
│ QQuery 38 │ 78.00 / 79.49 ±1.01 / 80.92 ms │ 77.53 / 78.91 ±0.97 / 80.47 ms │ no change │
│ QQuery 39 │ 212.76 / 224.53 ±7.54 / 236.27 ms │ 196.46 / 202.86 ±4.27 / 208.42 ms │ +1.11x faster │
│ QQuery 40 │ 26.76 / 29.00 ±2.26 / 33.19 ms │ 23.80 / 25.10 ±1.07 / 26.91 ms │ +1.16x faster │
│ QQuery 41 │ 21.02 / 22.69 ±0.91 / 23.55 ms │ 18.50 / 20.22 ±1.13 / 22.00 ms │ +1.12x faster │
│ QQuery 42 │ 19.12 / 20.40 ±0.66 / 20.96 ms │ 18.97 / 19.47 ±0.50 / 20.25 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22922.16ms │
│ Total Time (early-group-emission) │ 22535.06ms │
│ Average Time (HEAD) │ 533.07ms │
│ Average Time (early-group-emission) │ 524.07ms │
│ Queries Faster │ 9 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 30 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 115.6s |
| Peak memory | 35.4 GiB |
| Avg memory | 27.3 GiB |
| CPU user | 1073.0s |
| CPU sys | 100.0s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 113.6s |
| Peak memory | 36.1 GiB |
| Avg memory | 28.9 GiB |
| CPU user | 1033.0s |
| CPU sys | 86.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238596357
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.58 / 7.02 ±0.77 / 8.55 ms │ 6.84 / 7.36 ±0.80 / 8.95 ms │ no change │
│ QQuery 2 │ 145.44 / 146.08 ±0.68 / 147.39 ms │ 144.76 / 146.09 ±1.16 / 147.97 ms │ no change │
│ QQuery 3 │ 113.09 / 114.00 ±0.63 / 114.82 ms │ 115.09 / 115.62 ±0.39 / 116.19 ms │ no change │
│ QQuery 4 │ 1258.16 / 1281.15 ±15.04 / 1301.13 ms │ 1366.10 / 1376.70 ±8.79 / 1391.41 ms │ 1.07x slower │
│ QQuery 5 │ 170.61 / 172.94 ±1.68 / 175.11 ms │ 173.20 / 174.49 ±0.85 / 175.80 ms │ no change │
│ QQuery 6 │ 836.35 / 856.55 ±19.42 / 888.55 ms │ 838.38 / 868.47 ±23.54 / 904.99 ms │ no change │
│ QQuery 7 │ 339.93 / 342.15 ±1.39 / 343.97 ms │ 339.86 / 341.65 ±1.70 / 344.36 ms │ no change │
│ QQuery 8 │ 116.39 / 117.24 ±0.52 / 117.83 ms │ 115.87 / 117.54 ±1.21 / 119.17 ms │ no change │
│ QQuery 9 │ 101.22 / 102.91 ±2.08 / 106.87 ms │ 101.56 / 105.19 ±1.82 / 106.32 ms │ no change │
│ QQuery 10 │ 107.30 / 108.61 ±1.16 / 110.42 ms │ 107.11 / 107.74 ±0.60 / 108.67 ms │ no change │
│ QQuery 11 │ 882.49 / 921.10 ±21.13 / 941.45 ms │ 934.64 / 951.31 ±11.16 / 967.05 ms │ no change │
│ QQuery 12 │ 45.40 / 46.66 ±1.01 / 48.07 ms │ 45.13 / 47.77 ±1.61 / 49.56 ms │ no change │
│ QQuery 13 │ 395.02 / 397.93 ±1.67 / 399.79 ms │ 398.40 / 401.15 ±2.07 / 403.52 ms │ no change │
│ QQuery 14 │ 988.74 / 1000.54 ±9.23 / 1012.38 ms │ 994.08 / 1008.21 ±15.65 / 1038.71 ms │ no change │
│ QQuery 15 │ 15.51 / 16.31 ±0.75 / 17.54 ms │ 15.87 / 16.79 ±0.94 / 18.60 ms │ no change │
│ QQuery 16 │ 7.01 / 7.27 ±0.18 / 7.52 ms │ 7.30 / 7.85 ±0.71 / 9.21 ms │ 1.08x slower │
│ QQuery 17 │ 227.18 / 228.34 ±1.34 / 230.17 ms │ 225.04 / 227.45 ±1.72 / 230.11 ms │ no change │
│ QQuery 18 │ 126.62 / 128.20 ±1.29 / 130.20 ms │ 127.36 / 128.16 ±0.46 / 128.71 ms │ no change │
│ QQuery 19 │ 154.96 / 156.82 ±1.41 / 158.87 ms │ 155.08 / 156.95 ±1.16 / 158.72 ms │ no change │
│ QQuery 20 │ 13.43 / 14.30 ±0.68 / 15.48 ms │ 13.82 / 14.98 ±0.80 / 16.05 ms │ no change │
│ QQuery 21 │ 18.75 / 19.31 ±0.39 / 19.96 ms │ 19.58 / 19.90 ±0.18 / 20.09 ms │ no change │
│ QQuery 22 │ 479.81 / 483.75 ±2.61 / 487.40 ms │ 1154.90 / 1174.66 ±15.26 / 1201.27 ms │ 2.43x slower │
│ QQuery 23 │ 860.73 / 868.37 ±6.14 / 879.00 ms │ 950.00 / 964.87 ±12.04 / 983.05 ms │ 1.11x slower │
│ QQuery 24 │ 379.99 / 382.20 ±1.57 / 384.33 ms │ 376.97 / 381.03 ±4.49 / 388.21 ms │ no change │
│ QQuery 25 │ 334.93 / 338.45 ±2.06 / 341.37 ms │ 337.60 / 339.41 ±1.10 / 340.77 ms │ no change │
│ QQuery 26 │ 80.84 / 83.03 ±2.37 / 87.45 ms │ 79.73 / 81.87 ±1.40 / 83.80 ms │ no change │
│ QQuery 27 │ 6.79 / 7.31 ±0.47 / 8.00 ms │ 6.94 / 7.15 ±0.22 / 7.45 ms │ no change │
│ QQuery 28 │ 147.16 / 148.11 ±0.66 / 149.18 ms │ 148.55 / 149.55 ±1.10 / 151.57 ms │ no change │
│ QQuery 29 │ 279.41 / 280.28 ±0.78 / 281.48 ms │ 279.15 / 280.30 ±0.79 / 281.24 ms │ no change │
│ QQuery 30 │ 42.11 / 44.02 ±1.08 / 45.40 ms │ 43.92 / 44.98 ±0.68 / 45.87 ms │ no change │
│ QQuery 31 │ 168.81 / 171.08 ±1.87 / 173.35 ms │ 169.49 / 172.33 ±1.76 / 174.19 ms │ no change │
│ QQuery 32 │ 56.09 / 57.31 ±0.76 / 58.36 ms │ 57.51 / 65.96 ±15.84 / 97.63 ms │ 1.15x slower │
│ QQuery 33 │ 138.00 / 139.91 ±1.52 / 142.52 ms │ 138.15 / 141.23 ±2.58 / 144.72 ms │ no change │
│ QQuery 34 │ 6.83 / 7.08 ±0.23 / 7.48 ms │ 6.93 / 7.41 ±0.44 / 8.16 ms │ no change │
│ QQuery 35 │ 106.90 / 108.77 ±1.21 / 110.66 ms │ 106.36 / 107.56 ±0.70 / 108.44 ms │ no change │
│ QQuery 36 │ 6.49 / 6.77 ±0.28 / 7.27 ms │ 6.39 / 6.66 ±0.21 / 7.00 ms │ no change │
│ QQuery 37 │ 8.21 / 8.64 ±0.24 / 8.88 ms │ 8.55 / 8.82 ±0.19 / 9.13 ms │ no change │
│ QQuery 38 │ 83.60 / 87.27 ±2.84 / 91.80 ms │ 83.41 / 86.85 ±4.14 / 94.79 ms │ no change │
│ QQuery 39 │ 122.89 / 125.98 ±2.03 / 128.50 ms │ 124.74 / 125.87 ±1.13 / 127.77 ms │ no change │
│ QQuery 40 │ 109.68 / 115.15 ±4.46 / 122.25 ms │ 107.24 / 112.77 ±5.49 / 123.03 ms │ no change │
│ QQuery 41 │ 14.05 / 15.16 ±0.82 / 16.50 ms │ 14.32 / 14.94 ±0.51 / 15.58 ms │ no change │
│ QQuery 42 │ 107.42 / 109.24 ±1.32 / 111.34 ms │ 106.62 / 108.48 ±1.16 / 109.93 ms │ no change │
│ QQuery 43 │ 5.78 / 5.99 ±0.20 / 6.37 ms │ 5.89 / 6.05 ±0.17 / 6.38 ms │ no change │
│ QQuery 44 │ 12.04 / 12.43 ±0.50 / 13.33 ms │ 11.37 / 12.15 ±0.88 / 13.63 ms │ no change │
│ QQuery 45 │ 50.00 / 50.82 ±0.52 / 51.37 ms │ 49.45 / 50.08 ±0.43 / 50.76 ms │ no change │
│ QQuery 46 │ 8.18 / 8.43 ±0.23 / 8.79 ms │ 8.35 / 8.56 ±0.18 / 8.79 ms │ no change │
│ QQuery 47 │ 681.88 / 686.26 ±2.86 / 689.40 ms │ 696.60 / 702.50 ±4.09 / 707.02 ms │ no change │
│ QQuery 48 │ 284.49 / 290.51 ±3.19 / 292.89 ms │ 283.99 / 290.08 ±4.09 / 295.57 ms │ no change │
│ QQuery 49 │ 248.84 / 252.76 ±2.77 / 256.68 ms │ 248.68 / 249.51 ±0.62 / 250.35 ms │ no change │
│ QQuery 50 │ 214.26 / 223.86 ±5.43 / 230.28 ms │ 216.73 / 221.58 ±4.12 / 228.95 ms │ no change │
│ QQuery 51 │ 179.74 / 181.87 ±1.93 / 185.26 ms │ 193.04 / 197.40 ±3.06 / 201.16 ms │ 1.09x slower │
│ QQuery 52 │ 107.21 / 108.16 ±0.85 / 109.55 ms │ 107.18 / 109.75 ±1.70 / 112.19 ms │ no change │
│ QQuery 53 │ 101.85 / 103.68 ±1.13 / 105.10 ms │ 101.88 / 103.64 ±1.71 / 106.62 ms │ no change │
│ QQuery 54 │ 144.64 / 145.98 ±1.06 / 147.84 ms │ 145.32 / 147.26 ±1.90 / 150.73 ms │ no change │
│ QQuery 55 │ 105.55 / 106.98 ±1.46 / 109.01 ms │ 106.77 / 108.00 ±1.36 / 110.19 ms │ no change │
│ QQuery 56 │ 141.58 / 142.34 ±0.74 / 143.27 ms │ 141.41 / 142.26 ±0.46 / 142.72 ms │ no change │
│ QQuery 57 │ 170.84 / 172.86 ±1.42 / 174.45 ms │ 170.22 / 173.04 ±2.07 / 175.95 ms │ no change │
│ QQuery 58 │ 290.16 / 298.62 ±7.58 / 310.11 ms │ 292.78 / 299.08 ±8.26 / 314.96 ms │ no change │
│ QQuery 59 │ 196.29 / 200.77 ±4.14 / 208.30 ms │ 197.99 / 201.24 ±4.84 / 210.73 ms │ no change │
│ QQuery 60 │ 143.30 / 145.09 ±1.18 / 146.35 ms │ 143.25 / 144.39 ±1.01 / 146.22 ms │ no change │
│ QQuery 61 │ 13.05 / 13.19 ±0.14 / 13.39 ms │ 13.17 / 13.51 ±0.26 / 13.92 ms │ no change │
│ QQuery 62 │ 876.75 / 901.87 ±19.22 / 935.84 ms │ 905.43 / 945.73 ±21.64 / 967.82 ms │ no change │
│ QQuery 63 │ 104.02 / 106.60 ±3.56 / 113.68 ms │ 104.05 / 105.19 ±0.64 / 105.90 ms │ no change │
│ QQuery 64 │ 670.43 / 677.27 ±4.24 / 683.30 ms │ 685.25 / 692.86 ±7.36 / 706.20 ms │ no change │
│ QQuery 65 │ 247.11 / 250.12 ±2.33 / 254.13 ms │ 257.30 / 262.05 ±2.84 / 265.47 ms │ no change │
│ QQuery 66 │ 227.02 / 250.67 ±17.24 / 272.10 ms │ 252.29 / 259.34 ±5.18 / 267.69 ms │ no change │
│ QQuery 67 │ 301.46 / 312.38 ±6.70 / 321.65 ms │ 303.49 / 316.73 ±11.79 / 338.43 ms │ no change │
│ QQuery 68 │ 8.56 / 9.64 ±0.93 / 10.65 ms │ 9.03 / 10.56 ±1.19 / 12.58 ms │ 1.10x slower │
│ QQuery 69 │ 102.99 / 106.31 ±2.17 / 108.80 ms │ 101.98 / 103.31 ±1.06 / 104.90 ms │ no change │
│ QQuery 70 │ 332.11 / 346.18 ±9.56 / 359.12 ms │ 350.68 / 362.89 ±9.16 / 375.54 ms │ no change │
│ QQuery 71 │ 133.31 / 134.55 ±1.01 / 136.35 ms │ 135.94 / 138.18 ±1.69 / 139.89 ms │ no change │
│ QQuery 72 │ 609.15 / 619.26 ±5.77 / 625.82 ms │ 615.09 / 627.52 ±7.73 / 638.38 ms │ no change │
│ QQuery 73 │ 6.86 / 8.25 ±1.21 / 10.09 ms │ 6.85 / 8.31 ±1.55 / 10.85 ms │ no change │
│ QQuery 74 │ 537.95 / 542.66 ±5.10 / 552.12 ms │ 617.59 / 628.04 ±6.51 / 636.54 ms │ 1.16x slower │
│ QQuery 75 │ 274.18 / 277.96 ±2.65 / 280.81 ms │ 275.33 / 278.90 ±2.96 / 282.51 ms │ no change │
│ QQuery 76 │ 130.46 / 132.35 ±1.38 / 133.93 ms │ 131.24 / 132.62 ±0.95 / 133.97 ms │ no change │
│ QQuery 77 │ 185.65 / 190.32 ±4.81 / 199.45 ms │ 189.72 / 192.00 ±2.26 / 195.16 ms │ no change │
│ QQuery 78 │ 337.82 / 340.17 ±1.74 / 343.02 ms │ 340.69 / 342.43 ±2.10 / 346.42 ms │ no change │
│ QQuery 79 │ 229.16 / 231.87 ±2.59 / 236.41 ms │ 234.40 / 237.98 ±2.05 / 239.89 ms │ no change │
│ QQuery 80 │ 319.99 / 322.28 ±2.16 / 325.83 ms │ 319.43 / 323.25 ±3.48 / 329.72 ms │ no change │
│ QQuery 81 │ 26.23 / 26.58 ±0.33 / 27.17 ms │ 26.54 / 27.28 ±0.37 / 27.49 ms │ no change │
│ QQuery 82 │ 198.46 / 200.39 ±1.39 / 202.73 ms │ 202.36 / 204.26 ±1.40 / 206.45 ms │ no change │
│ QQuery 83 │ 37.70 / 38.76 ±0.91 / 40.27 ms │ 38.05 / 39.30 ±1.00 / 41.14 ms │ no change │
│ QQuery 84 │ 48.23 / 48.77 ±0.39 / 49.31 ms │ 48.14 / 49.43 ±0.77 / 50.40 ms │ no change │
│ QQuery 85 │ 147.11 / 149.02 ±1.14 / 150.22 ms │ 147.28 / 150.63 ±2.50 / 154.46 ms │ no change │
│ QQuery 86 │ 38.31 / 39.53 ±1.00 / 41.11 ms │ 38.84 / 40.72 ±1.57 / 42.56 ms │ no change │
│ QQuery 87 │ 82.40 / 86.72 ±2.86 / 91.18 ms │ 83.84 / 89.16 ±3.46 / 94.52 ms │ no change │
│ QQuery 88 │ 100.38 / 101.24 ±1.02 / 102.86 ms │ 100.78 / 101.31 ±0.48 / 102.08 ms │ no change │
│ QQuery 89 │ 116.22 / 118.50 ±1.78 / 120.93 ms │ 120.31 / 121.03 ±0.70 / 122.30 ms │ no change │
│ QQuery 90 │ 23.18 / 23.63 ±0.30 / 23.99 ms │ 23.95 / 25.09 ±0.74 / 26.26 ms │ 1.06x slower │
│ QQuery 91 │ 62.71 / 64.95 ±1.47 / 66.85 ms │ 64.41 / 65.49 ±0.92 / 66.83 ms │ no change │
│ QQuery 92 │ 56.01 / 57.84 ±1.22 / 59.55 ms │ 58.84 / 60.35 ±1.66 / 63.58 ms │ no change │
│ QQuery 93 │ 185.39 / 186.63 ±0.86 / 187.95 ms │ 183.86 / 187.61 ±2.31 / 190.28 ms │ no change │
│ QQuery 94 │ 61.03 / 61.97 ±0.71 / 62.85 ms │ 61.02 / 62.13 ±0.81 / 63.48 ms │ no change │
│ QQuery 95 │ 127.94 / 129.03 ±0.98 / 130.68 ms │ 125.72 / 128.05 ±1.29 / 129.23 ms │ no change │
│ QQuery 96 │ 74.29 / 74.99 ±0.50 / 75.56 ms │ 70.32 / 73.91 ±1.90 / 75.96 ms │ no change │
│ QQuery 97 │ 124.82 / 126.39 ±1.26 / 128.49 ms │ 128.02 / 129.13 ±0.87 / 130.09 ms │ no change │
│ QQuery 98 │ 151.22 / 154.06 ±1.68 / 156.23 ms │ 150.31 / 153.73 ±2.04 / 156.07 ms │ no change │
│ QQuery 99 │ 10775.04 / 10840.18 ±74.01 / 10980.32 ms │ 10772.60 / 10817.84 ±49.08 / 10894.67 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31213.74ms │
│ Total Time (early-group-emission) │ 32398.49ms │
│ Average Time (HEAD) │ 315.29ms │
│ Average Time (early-group-emission) │ 327.26ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 9 │
│ Queries with No Change │ 90 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 156.4s |
| Peak memory | 5.2 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 257.2s |
| CPU sys | 16.8s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 162.3s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 269.7s |
| CPU sys | 17.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241511493
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241504219-1190-hjzcv 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241566424
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.23 / 4.57 ±6.52 / 17.60 ms │ 1.22 / 4.61 ±6.60 / 17.81 ms │ no change │
│ QQuery 1 │ 14.99 / 15.36 ±0.20 / 15.59 ms │ 14.47 / 14.98 ±0.32 / 15.37 ms │ no change │
│ QQuery 2 │ 44.50 / 44.88 ±0.45 / 45.72 ms │ 44.99 / 45.17 ±0.17 / 45.46 ms │ no change │
│ QQuery 3 │ 44.89 / 46.51 ±1.12 / 47.81 ms │ 44.60 / 46.45 ±1.52 / 49.03 ms │ no change │
│ QQuery 4 │ 299.12 / 308.59 ±5.58 / 314.79 ms │ 306.22 / 317.92 ±8.41 / 326.21 ms │ no change │
│ QQuery 5 │ 356.90 / 360.13 ±3.05 / 365.94 ms │ 345.88 / 364.74 ±15.37 / 380.60 ms │ no change │
│ QQuery 6 │ 6.82 / 8.49 ±2.25 / 12.93 ms │ 6.14 / 6.60 ±0.53 / 7.50 ms │ +1.29x faster │
│ QQuery 7 │ 17.63 / 18.07 ±0.47 / 18.97 ms │ 17.55 / 19.15 ±1.18 / 20.73 ms │ 1.06x slower │
│ QQuery 8 │ 443.78 / 453.06 ±5.80 / 459.87 ms │ 425.57 / 434.59 ±6.73 / 445.95 ms │ no change │
│ QQuery 9 │ 724.22 / 731.46 ±5.46 / 740.25 ms │ 447.16 / 466.28 ±16.09 / 495.10 ms │ +1.57x faster │
│ QQuery 10 │ 95.74 / 99.79 ±5.29 / 109.92 ms │ 94.71 / 96.84 ±1.60 / 99.56 ms │ no change │
│ QQuery 11 │ 109.95 / 110.32 ±0.39 / 111.06 ms │ 107.09 / 108.57 ±0.79 / 109.17 ms │ no change │
│ QQuery 12 │ 367.40 / 378.80 ±6.93 / 387.67 ms │ 351.07 / 360.79 ±6.96 / 372.55 ms │ no change │
│ QQuery 13 │ 491.05 / 506.55 ±11.23 / 523.31 ms │ 547.77 / 573.26 ±19.96 / 607.57 ms │ 1.13x slower │
│ QQuery 14 │ 358.62 / 372.98 ±10.60 / 390.91 ms │ 353.53 / 371.35 ±13.21 / 392.39 ms │ no change │
│ QQuery 15 │ 379.62 / 398.22 ±19.67 / 434.18 ms │ 338.88 / 348.80 ±7.98 / 361.91 ms │ +1.14x faster │
│ QQuery 16 │ 760.04 / 780.92 ±19.72 / 816.84 ms │ 687.33 / 719.86 ±19.78 / 745.63 ms │ +1.08x faster │
│ QQuery 17 │ 751.21 / 784.81 ±33.26 / 842.18 ms │ 686.28 / 706.47 ±13.33 / 724.14 ms │ +1.11x faster │
│ QQuery 18 │ 1528.44 / 1555.21 ±31.14 / 1614.12 ms │ 1329.82 / 1359.00 ±22.68 / 1387.23 ms │ +1.14x faster │
│ QQuery 19 │ 37.18 / 38.83 ±1.47 / 40.98 ms │ 36.04 / 38.74 ±1.75 / 41.01 ms │ no change │
│ QQuery 20 │ 718.48 / 729.13 ±8.69 / 739.51 ms │ 717.18 / 723.12 ±4.56 / 727.33 ms │ no change │
│ QQuery 21 │ 760.09 / 766.59 ±4.42 / 771.03 ms │ 756.05 / 763.27 ±4.35 / 768.17 ms │ no change │
│ QQuery 22 │ 1143.55 / 1155.22 ±8.42 / 1169.01 ms │ 1139.93 / 1143.38 ±2.72 / 1146.87 ms │ no change │
│ QQuery 23 │ 3197.37 / 3227.87 ±23.24 / 3254.63 ms │ 3109.50 / 3152.19 ±35.59 / 3204.29 ms │ no change │
│ QQuery 24 │ 103.59 / 106.82 ±2.45 / 111.11 ms │ 101.08 / 108.04 ±3.75 / 111.68 ms │ no change │
│ QQuery 25 │ 145.24 / 147.25 ±1.41 / 149.39 ms │ 142.84 / 145.37 ±1.40 / 146.98 ms │ no change │
│ QQuery 26 │ 101.76 / 105.55 ±2.85 / 109.46 ms │ 102.81 / 104.99 ±1.29 / 106.58 ms │ no change │
│ QQuery 27 │ 855.58 / 869.10 ±15.01 / 897.44 ms │ 859.39 / 864.95 ±5.80 / 875.89 ms │ no change │
│ QQuery 28 │ 3348.10 / 3366.07 ±15.87 / 3394.88 ms │ 3329.35 / 3383.79 ±64.26 / 3493.44 ms │ no change │
│ QQuery 29 │ 51.92 / 55.40 ±2.07 / 57.70 ms │ 53.00 / 55.83 ±1.98 / 59.18 ms │ no change │
│ QQuery 30 │ 375.73 / 380.37 ±4.69 / 388.41 ms │ 380.86 / 393.51 ±9.66 / 407.93 ms │ no change │
│ QQuery 31 │ 375.44 / 391.35 ±16.68 / 415.98 ms │ 420.47 / 441.90 ±17.21 / 465.49 ms │ 1.13x slower │
│ QQuery 32 │ 1127.04 / 1143.46 ±21.01 / 1182.89 ms │ 1439.00 / 1475.16 ±29.89 / 1526.15 ms │ 1.29x slower │
│ QQuery 33 │ 1511.81 / 1541.48 ±33.60 / 1602.47 ms │ 1533.10 / 1558.45 ±28.67 / 1609.68 ms │ no change │
│ QQuery 34 │ 1519.45 / 1540.45 ±17.32 / 1562.45 ms │ 1523.97 / 1572.21 ±33.83 / 1628.82 ms │ no change │
│ QQuery 35 │ 401.72 / 411.94 ±8.56 / 424.74 ms │ 352.69 / 367.72 ±11.63 / 387.68 ms │ +1.12x faster │
│ QQuery 36 │ 113.01 / 121.08 ±4.23 / 124.57 ms │ 116.36 / 118.73 ±1.19 / 119.61 ms │ no change │
│ QQuery 37 │ 48.42 / 49.41 ±0.56 / 50.15 ms │ 50.10 / 52.18 ±1.67 / 54.59 ms │ 1.06x slower │
│ QQuery 38 │ 77.60 / 78.59 ±0.80 / 79.69 ms │ 77.45 / 79.54 ±1.32 / 81.06 ms │ no change │
│ QQuery 39 │ 208.38 / 221.84 ±8.11 / 230.21 ms │ 197.85 / 203.12 ±4.64 / 211.75 ms │ +1.09x faster │
│ QQuery 40 │ 23.75 / 27.95 ±2.81 / 30.88 ms │ 25.61 / 27.66 ±2.10 / 31.23 ms │ no change │
│ QQuery 41 │ 20.80 / 22.69 ±1.74 / 25.02 ms │ 21.92 / 23.76 ±1.58 / 26.04 ms │ no change │
│ QQuery 42 │ 20.64 / 20.90 ±0.20 / 21.20 ms │ 20.77 / 21.39 ±0.41 / 21.92 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 23498.07ms │
│ Total Time (early-group-emission) │ 23184.45ms │
│ Average Time (HEAD) │ 546.47ms │
│ Average Time (early-group-emission) │ 539.17ms │
│ Queries Faster │ 8 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 30 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 118.7s |
| Peak memory | 40.1 GiB |
| Avg memory | 32.3 GiB |
| CPU user | 1119.0s |
| CPU sys | 92.1s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 116.9s |
| Peak memory | 40.1 GiB |
| Avg memory | 30.8 GiB |
| CPU user | 1081.5s |
| CPU sys | 74.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241564141
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.21 / 4.41 ±6.30 / 17.01 ms │ 1.20 / 4.38 ±6.28 / 16.93 ms │ no change │
│ QQuery 1 │ 14.40 / 14.74 ±0.27 / 15.04 ms │ 14.04 / 14.34 ±0.26 / 14.69 ms │ no change │
│ QQuery 2 │ 44.26 / 44.69 ±0.26 / 44.99 ms │ 43.73 / 43.97 ±0.15 / 44.13 ms │ no change │
│ QQuery 3 │ 43.35 / 45.17 ±1.33 / 46.94 ms │ 42.07 / 47.37 ±2.99 / 49.68 ms │ no change │
│ QQuery 4 │ 282.12 / 291.21 ±7.12 / 303.58 ms │ 308.75 / 315.95 ±5.79 / 325.78 ms │ 1.08x slower │
│ QQuery 5 │ 339.99 / 347.28 ±5.38 / 354.08 ms │ 356.74 / 363.00 ±4.79 / 371.14 ms │ no change │
│ QQuery 6 │ 5.83 / 7.25 ±0.95 / 8.50 ms │ 5.62 / 6.33 ±0.40 / 6.84 ms │ +1.15x faster │
│ QQuery 7 │ 16.93 / 17.63 ±0.96 / 19.51 ms │ 16.91 / 17.80 ±0.76 / 18.76 ms │ no change │
│ QQuery 8 │ 403.34 / 424.64 ±16.64 / 452.22 ms │ 416.50 / 433.75 ±12.88 / 453.25 ms │ no change │
│ QQuery 9 │ 629.83 / 643.98 ±9.67 / 655.84 ms │ 440.63 / 447.85 ±8.05 / 462.77 ms │ +1.44x faster │
│ QQuery 10 │ 94.14 / 96.17 ±2.30 / 100.46 ms │ 92.01 / 94.20 ±2.22 / 97.58 ms │ no change │
│ QQuery 11 │ 106.64 / 107.28 ±0.63 / 108.43 ms │ 102.41 / 106.36 ±2.37 / 109.72 ms │ no change │
│ QQuery 12 │ 335.55 / 340.93 ±4.04 / 346.82 ms │ 339.07 / 347.98 ±9.40 / 361.86 ms │ no change │
│ QQuery 13 │ 448.50 / 456.64 ±7.46 / 467.49 ms │ 538.15 / 558.53 ±11.85 / 575.05 ms │ 1.22x slower │
│ QQuery 14 │ 336.29 / 345.40 ±5.40 / 351.57 ms │ 354.66 / 362.51 ±5.37 / 369.48 ms │ no change │
│ QQuery 15 │ 351.30 / 364.66 ±15.35 / 390.74 ms │ 335.49 / 348.72 ±10.77 / 361.84 ms │ no change │
│ QQuery 16 │ 726.59 / 744.70 ±22.05 / 786.31 ms │ 678.82 / 707.99 ±26.13 / 750.14 ms │ no change │
│ QQuery 17 │ 703.60 / 706.03 ±2.61 / 710.03 ms │ 708.77 / 728.01 ±18.09 / 753.04 ms │ no change │
│ QQuery 18 │ 1403.07 / 1475.83 ±42.91 / 1521.75 ms │ 1251.59 / 1289.47 ±57.09 / 1402.71 ms │ +1.14x faster │
│ QQuery 19 │ 35.79 / 40.63 ±8.40 / 57.41 ms │ 34.89 / 36.25 ±1.22 / 38.54 ms │ +1.12x faster │
│ QQuery 20 │ 722.77 / 738.50 ±17.20 / 764.80 ms │ 718.53 / 720.68 ±1.92 / 723.36 ms │ no change │
│ QQuery 21 │ 768.05 / 777.55 ±10.75 / 798.17 ms │ 763.01 / 779.35 ±14.40 / 797.06 ms │ no change │
│ QQuery 22 │ 1131.44 / 1145.97 ±16.17 / 1168.88 ms │ 1133.17 / 1139.77 ±8.62 / 1156.28 ms │ no change │
│ QQuery 23 │ 3096.34 / 3109.83 ±10.33 / 3121.56 ms │ 3052.61 / 3079.63 ±24.47 / 3112.28 ms │ no change │
│ QQuery 24 │ 98.38 / 102.48 ±3.73 / 109.03 ms │ 100.46 / 102.99 ±1.45 / 104.49 ms │ no change │
│ QQuery 25 │ 138.08 / 139.27 ±1.00 / 140.45 ms │ 138.82 / 140.09 ±1.36 / 141.87 ms │ no change │
│ QQuery 26 │ 101.29 / 102.81 ±0.91 / 104.06 ms │ 96.92 / 100.08 ±1.80 / 101.78 ms │ no change │
│ QQuery 27 │ 845.41 / 851.11 ±6.55 / 863.79 ms │ 850.50 / 853.98 ±2.91 / 859.12 ms │ no change │
│ QQuery 28 │ 3270.48 / 3294.10 ±14.93 / 3311.44 ms │ 3249.86 / 3335.10 ±90.10 / 3472.42 ms │ no change │
│ QQuery 29 │ 50.37 / 55.58 ±4.25 / 63.06 ms │ 49.71 / 53.74 ±3.07 / 59.20 ms │ no change │
│ QQuery 30 │ 360.85 / 365.03 ±3.67 / 371.84 ms │ 357.16 / 372.67 ±14.31 / 398.42 ms │ no change │
│ QQuery 31 │ 367.64 / 373.93 ±8.17 / 389.44 ms │ 415.42 / 427.51 ±15.16 / 456.65 ms │ 1.14x slower │
│ QQuery 32 │ 1025.40 / 1047.51 ±23.89 / 1088.55 ms │ 1551.92 / 1578.87 ±31.15 / 1635.42 ms │ 1.51x slower │
│ QQuery 33 │ 1443.98 / 1450.01 ±5.49 / 1459.33 ms │ 1483.43 / 1533.45 ±31.99 / 1577.05 ms │ 1.06x slower │
│ QQuery 34 │ 1430.31 / 1447.66 ±18.79 / 1475.39 ms │ 1448.01 / 1526.00 ±53.70 / 1601.98 ms │ 1.05x slower │
│ QQuery 35 │ 379.91 / 386.12 ±7.44 / 400.62 ms │ 326.93 / 336.41 ±8.34 / 346.17 ms │ +1.15x faster │
│ QQuery 36 │ 114.03 / 119.72 ±2.91 / 122.10 ms │ 112.71 / 115.88 ±1.63 / 117.18 ms │ no change │
│ QQuery 37 │ 46.61 / 48.73 ±1.62 / 50.49 ms │ 47.71 / 48.56 ±0.86 / 49.96 ms │ no change │
│ QQuery 38 │ 74.13 / 76.50 ±1.94 / 79.20 ms │ 73.92 / 76.98 ±2.71 / 81.44 ms │ no change │
│ QQuery 39 │ 209.09 / 215.32 ±5.39 / 222.42 ms │ 192.77 / 199.23 ±4.41 / 206.14 ms │ +1.08x faster │
│ QQuery 40 │ 22.29 / 24.98 ±1.99 / 28.45 ms │ 22.48 / 25.23 ±1.83 / 27.66 ms │ no change │
│ QQuery 41 │ 19.54 / 20.73 ±1.44 / 23.46 ms │ 20.29 / 20.83 ±0.61 / 21.73 ms │ no change │
│ QQuery 42 │ 18.90 / 19.30 ±0.33 / 19.90 ms │ 19.27 / 19.89 ±0.85 / 21.51 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22432.00ms │
│ Total Time (early-group-emission) │ 22861.66ms │
│ Average Time (HEAD) │ 521.67ms │
│ Average Time (early-group-emission) │ 531.67ms │
│ Queries Faster │ 6 │
│ Queries Slower │ 6 │
│ Queries with No Change │ 31 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.3s |
| Peak memory | 40.9 GiB |
| Avg memory | 29.7 GiB |
| CPU user | 1067.9s |
| CPU sys | 84.9s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 115.2s |
| Peak memory | 37.0 GiB |
| Avg memory | 27.6 GiB |
| CPU user | 1051.6s |
| CPU sys | 87.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238599116
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.21 / 4.53 ±6.47 / 17.47 ms │ 1.20 / 4.51 ±6.44 / 17.39 ms │ no change │
│ QQuery 1 │ 14.46 / 14.84 ±0.20 / 15.04 ms │ 14.54 / 14.85 ±0.21 / 15.16 ms │ no change │
│ QQuery 2 │ 44.82 / 45.26 ±0.35 / 45.73 ms │ 44.35 / 44.52 ±0.16 / 44.82 ms │ no change │
│ QQuery 3 │ 42.33 / 45.89 ±2.10 / 47.58 ms │ 43.72 / 47.65 ±2.94 / 50.70 ms │ no change │
│ QQuery 4 │ 293.90 / 300.83 ±5.17 / 305.96 ms │ 307.87 / 314.71 ±6.50 / 325.54 ms │ no change │
│ QQuery 5 │ 342.91 / 349.57 ±4.06 / 354.56 ms │ 358.89 / 370.97 ±11.34 / 391.08 ms │ 1.06x slower │
│ QQuery 6 │ 5.72 / 6.59 ±0.79 / 7.73 ms │ 5.80 / 7.15 ±1.41 / 9.76 ms │ 1.09x slower │
│ QQuery 7 │ 17.36 / 17.97 ±0.49 / 18.86 ms │ 16.89 / 17.26 ±0.25 / 17.63 ms │ no change │
│ QQuery 8 │ 423.68 / 428.88 ±6.73 / 442.08 ms │ 426.31 / 437.41 ±13.74 / 463.65 ms │ no change │
│ QQuery 9 │ 662.71 / 679.77 ±14.27 / 704.99 ms │ 453.56 / 457.20 ±2.93 / 461.00 ms │ +1.49x faster │
│ QQuery 10 │ 94.05 / 97.06 ±1.97 / 99.50 ms │ 90.93 / 91.98 ±0.72 / 93.01 ms │ +1.06x faster │
│ QQuery 11 │ 107.84 / 109.59 ±1.35 / 111.81 ms │ 102.66 / 103.75 ±1.21 / 106.10 ms │ +1.06x faster │
│ QQuery 12 │ 346.89 / 349.90 ±2.71 / 353.73 ms │ 340.83 / 353.78 ±15.53 / 383.91 ms │ no change │
│ QQuery 13 │ 469.95 / 488.08 ±18.89 / 518.75 ms │ 553.16 / 568.08 ±15.85 / 597.20 ms │ 1.16x slower │
│ QQuery 14 │ 352.85 / 357.31 ±5.22 / 367.09 ms │ 335.65 / 368.56 ±19.39 / 396.29 ms │ no change │
│ QQuery 15 │ 358.89 / 372.04 ±9.69 / 388.93 ms │ 326.63 / 337.93 ±11.28 / 356.85 ms │ +1.10x faster │
│ QQuery 16 │ 731.34 / 745.27 ±15.41 / 772.52 ms │ 715.20 / 732.73 ±16.85 / 757.24 ms │ no change │
│ QQuery 17 │ 728.35 / 732.44 ±2.58 / 735.23 ms │ 689.50 / 704.35 ±10.82 / 721.28 ms │ no change │
│ QQuery 18 │ 1479.04 / 1517.70 ±36.40 / 1576.47 ms │ 1309.59 / 1353.48 ±28.45 / 1398.69 ms │ +1.12x faster │
│ QQuery 19 │ 38.06 / 39.14 ±1.30 / 41.38 ms │ 36.11 / 37.82 ±1.75 / 40.99 ms │ no change │
│ QQuery 20 │ 715.32 / 735.43 ±20.83 / 761.85 ms │ 720.76 / 732.16 ±12.38 / 754.21 ms │ no change │
│ QQuery 21 │ 760.89 / 765.28 ±3.73 / 771.32 ms │ 763.98 / 766.29 ±2.65 / 771.47 ms │ no change │
│ QQuery 22 │ 1139.86 / 1148.25 ±6.09 / 1156.33 ms │ 1142.88 / 1145.96 ±3.05 / 1150.98 ms │ no change │
│ QQuery 23 │ 3157.54 / 3178.01 ±17.94 / 3207.41 ms │ 3128.45 / 3131.85 ±2.51 / 3135.15 ms │ no change │
│ QQuery 24 │ 104.10 / 105.39 ±1.60 / 108.35 ms │ 98.87 / 103.04 ±4.03 / 110.27 ms │ no change │
│ QQuery 25 │ 138.14 / 141.90 ±2.53 / 144.43 ms │ 138.49 / 143.37 ±2.90 / 147.00 ms │ no change │
│ QQuery 26 │ 98.91 / 104.33 ±2.83 / 106.63 ms │ 101.20 / 103.00 ±1.52 / 104.69 ms │ no change │
│ QQuery 27 │ 846.74 / 851.65 ±4.08 / 858.82 ms │ 851.21 / 856.53 ±6.04 / 865.02 ms │ no change │
│ QQuery 28 │ 3287.48 / 3327.20 ±23.80 / 3360.92 ms │ 3286.38 / 3362.40 ±78.25 / 3473.06 ms │ no change │
│ QQuery 29 │ 51.37 / 56.77 ±5.41 / 66.41 ms │ 51.53 / 56.79 ±4.78 / 64.55 ms │ no change │
│ QQuery 30 │ 359.95 / 367.15 ±8.41 / 383.24 ms │ 362.20 / 374.03 ±12.15 / 397.40 ms │ no change │
│ QQuery 31 │ 360.39 / 380.78 ±11.57 / 395.58 ms │ 424.40 / 435.99 ±10.44 / 455.43 ms │ 1.14x slower │
│ QQuery 32 │ 1058.50 / 1081.30 ±24.56 / 1125.16 ms │ 1621.46 / 1651.53 ±21.04 / 1678.75 ms │ 1.53x slower │
│ QQuery 33 │ 1488.80 / 1518.07 ±19.56 / 1549.45 ms │ 1522.64 / 1591.12 ±61.32 / 1672.15 ms │ no change │
│ QQuery 34 │ 1475.44 / 1496.05 ±12.16 / 1508.97 ms │ 1514.63 / 1547.03 ±27.76 / 1594.23 ms │ no change │
│ QQuery 35 │ 398.62 / 407.65 ±4.59 / 410.78 ms │ 322.91 / 344.54 ±13.33 / 360.11 ms │ +1.18x faster │
│ QQuery 36 │ 112.62 / 121.68 ±6.60 / 132.61 ms │ 111.73 / 115.13 ±2.19 / 117.45 ms │ +1.06x faster │
│ QQuery 37 │ 49.23 / 50.36 ±1.39 / 52.93 ms │ 47.74 / 49.08 ±1.10 / 51.10 ms │ no change │
│ QQuery 38 │ 77.04 / 78.72 ±1.60 / 81.38 ms │ 76.64 / 77.87 ±1.13 / 80.00 ms │ no change │
│ QQuery 39 │ 211.24 / 220.71 ±5.37 / 227.30 ms │ 198.46 / 206.60 ±5.89 / 216.85 ms │ +1.07x faster │
│ QQuery 40 │ 25.03 / 26.64 ±1.42 / 28.59 ms │ 23.94 / 26.19 ±1.28 / 27.66 ms │ no change │
│ QQuery 41 │ 21.11 / 22.49 ±0.94 / 23.73 ms │ 20.12 / 21.40 ±1.03 / 23.17 ms │ no change │
│ QQuery 42 │ 20.16 / 20.77 ±0.38 / 21.26 ms │ 19.60 / 20.31 ±0.60 / 21.29 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22909.24ms │
│ Total Time (early-group-emission) │ 23230.90ms │
│ Average Time (HEAD) │ 532.77ms │
│ Average Time (early-group-emission) │ 540.25ms │
│ Queries Faster │ 8 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 30 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 115.7s |
| Peak memory | 41.3 GiB |
| Avg memory | 30.9 GiB |
| CPU user | 1092.9s |
| CPU sys | 85.9s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 117.1s |
| Peak memory | 36.7 GiB |
| Avg memory | 28.0 GiB |
| CPU user | 1071.5s |
| CPU sys | 90.2s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238805686
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238803110-1176-7vg6s 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (01835ce8773a93097f91e5ace1e53db1b1c30fce) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..01835ce8773a93097f91e5ace1e53db1b1c30fce) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238801589
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.87 / 7.36 ±0.77 / 8.89 ms │ 7.10 / 7.49 ±0.74 / 8.97 ms │ no change │
│ QQuery 2 │ 147.08 / 147.45 ±0.47 / 148.32 ms │ 147.15 / 147.87 ±0.59 / 148.91 ms │ no change │
│ QQuery 3 │ 115.87 / 116.62 ±0.69 / 117.69 ms │ 116.87 / 117.22 ±0.20 / 117.48 ms │ no change │
│ QQuery 4 │ 1467.41 / 1507.16 ±26.23 / 1542.11 ms │ 1443.35 / 1470.46 ±19.44 / 1488.47 ms │ no change │
│ QQuery 5 │ 179.02 / 179.83 ±0.58 / 180.84 ms │ 177.34 / 178.02 ±0.84 / 179.64 ms │ no change │
│ QQuery 6 │ 885.55 / 908.92 ±17.88 / 931.85 ms │ 875.46 / 905.84 ±22.81 / 944.09 ms │ no change │
│ QQuery 7 │ 350.72 / 353.46 ±1.97 / 355.74 ms │ 351.91 / 353.80 ±1.23 / 355.35 ms │ no change │
│ QQuery 8 │ 119.01 / 120.44 ±1.07 / 122.00 ms │ 118.08 / 119.34 ±0.81 / 120.22 ms │ no change │
│ QQuery 9 │ 102.86 / 105.37 ±2.18 / 107.81 ms │ 103.15 / 105.54 ±2.87 / 109.88 ms │ no change │
│ QQuery 10 │ 110.23 / 111.81 ±1.00 / 112.88 ms │ 109.15 / 110.85 ±1.30 / 112.90 ms │ no change │
│ QQuery 11 │ 1043.06 / 1055.51 ±9.12 / 1070.24 ms │ 1047.78 / 1064.85 ±10.29 / 1076.79 ms │ no change │
│ QQuery 12 │ 46.54 / 48.41 ±1.70 / 51.31 ms │ 48.54 / 49.19 ±0.45 / 49.92 ms │ no change │
│ QQuery 13 │ 414.74 / 418.00 ±2.70 / 421.90 ms │ 410.82 / 415.34 ±2.67 / 418.15 ms │ no change │
│ QQuery 14 │ 1015.91 / 1029.70 ±10.97 / 1049.28 ms │ 1023.52 / 1033.82 ±6.89 / 1041.18 ms │ no change │
│ QQuery 15 │ 16.69 / 18.41 ±1.78 / 21.07 ms │ 16.13 / 16.96 ±0.87 / 18.62 ms │ +1.09x faster │
│ QQuery 16 │ 7.75 / 8.09 ±0.37 / 8.71 ms │ 7.70 / 8.16 ±0.63 / 9.41 ms │ no change │
│ QQuery 17 │ 238.25 / 240.23 ±1.39 / 242.07 ms │ 237.63 / 239.19 ±1.54 / 241.62 ms │ no change │
│ QQuery 18 │ 130.68 / 132.34 ±1.02 / 133.74 ms │ 130.67 / 132.18 ±1.06 / 134.01 ms │ no change │
│ QQuery 19 │ 162.68 / 164.39 ±0.96 / 165.63 ms │ 162.28 / 165.46 ±2.60 / 169.82 ms │ no change │
│ QQuery 20 │ 14.87 / 15.38 ±0.46 / 15.94 ms │ 14.11 / 15.06 ±0.65 / 15.81 ms │ no change │
│ QQuery 21 │ 20.39 / 20.79 ±0.36 / 21.23 ms │ 20.12 / 20.51 ±0.37 / 21.01 ms │ no change │
│ QQuery 22 │ 517.28 / 525.45 ±6.28 / 533.86 ms │ 1230.09 / 1240.36 ±9.38 / 1256.07 ms │ 2.36x slower │
│ QQuery 23 │ 938.98 / 947.88 ±8.88 / 963.43 ms │ 936.71 / 943.60 ±5.33 / 950.14 ms │ no change │
│ QQuery 24 │ 400.15 / 401.54 ±1.32 / 403.75 ms │ 398.82 / 402.79 ±2.12 / 405.18 ms │ no change │
│ QQuery 25 │ 351.54 / 355.02 ±2.75 / 359.16 ms │ 351.32 / 353.28 ±1.43 / 355.07 ms │ no change │
│ QQuery 26 │ 84.36 / 85.39 ±0.69 / 86.41 ms │ 84.16 / 85.39 ±1.03 / 86.58 ms │ no change │
│ QQuery 27 │ 7.38 / 7.89 ±0.74 / 9.36 ms │ 7.40 / 7.74 ±0.56 / 8.85 ms │ no change │
│ QQuery 28 │ 153.79 / 154.88 ±1.10 / 156.71 ms │ 153.24 / 153.99 ±0.63 / 154.99 ms │ no change │
│ QQuery 29 │ 293.22 / 295.22 ±1.37 / 297.06 ms │ 290.93 / 293.95 ±1.66 / 296.03 ms │ no change │
│ QQuery 30 │ 46.57 / 47.63 ±0.94 / 48.82 ms │ 44.07 / 46.71 ±1.54 / 48.02 ms │ no change │
│ QQuery 31 │ 174.59 / 178.20 ±2.06 / 180.08 ms │ 176.70 / 178.82 ±1.71 / 181.79 ms │ no change │
│ QQuery 32 │ 57.68 / 67.44 ±17.55 / 102.53 ms │ 58.94 / 59.70 ±0.80 / 61.25 ms │ +1.13x faster │
│ QQuery 33 │ 145.19 / 145.98 ±0.65 / 146.83 ms │ 143.65 / 146.03 ±1.67 / 148.83 ms │ no change │
│ QQuery 34 │ 7.87 / 7.96 ±0.12 / 8.20 ms │ 7.41 / 7.94 ±0.56 / 9.00 ms │ no change │
│ QQuery 35 │ 111.21 / 113.28 ±1.25 / 114.84 ms │ 111.74 / 112.79 ±0.69 / 113.55 ms │ no change │
│ QQuery 36 │ 6.86 / 7.07 ±0.14 / 7.29 ms │ 6.90 / 7.30 ±0.37 / 7.86 ms │ no change │
│ QQuery 37 │ 8.85 / 9.19 ±0.28 / 9.61 ms │ 8.86 / 9.46 ±0.46 / 10.27 ms │ no change │
│ QQuery 38 │ 88.71 / 92.66 ±4.76 / 101.44 ms │ 86.39 / 91.51 ±4.06 / 96.67 ms │ no change │
│ QQuery 39 │ 136.44 / 139.19 ±1.71 / 141.84 ms │ 132.77 / 135.68 ±2.27 / 139.51 ms │ no change │
│ QQuery 40 │ 114.34 / 119.30 ±5.51 / 129.75 ms │ 110.86 / 119.06 ±9.59 / 137.05 ms │ no change │
│ QQuery 41 │ 15.31 / 16.58 ±1.03 / 17.99 ms │ 14.83 / 15.89 ±1.06 / 17.16 ms │ no change │
│ QQuery 42 │ 108.99 / 110.38 ±0.86 / 111.49 ms │ 110.08 / 112.65 ±1.60 / 114.44 ms │ no change │
│ QQuery 43 │ 6.35 / 6.44 ±0.08 / 6.59 ms │ 6.29 / 6.41 ±0.10 / 6.55 ms │ no change │
│ QQuery 44 │ 13.08 / 13.65 ±0.47 / 14.23 ms │ 11.98 / 13.03 ±0.70 / 13.92 ms │ no change │
│ QQuery 45 │ 52.62 / 53.69 ±0.78 / 55.02 ms │ 52.45 / 52.81 ±0.26 / 53.18 ms │ no change │
│ QQuery 46 │ 8.95 / 9.24 ±0.25 / 9.69 ms │ 8.87 / 9.19 ±0.21 / 9.43 ms │ no change │
│ QQuery 47 │ 824.58 / 843.58 ±11.13 / 856.67 ms │ 760.02 / 766.00 ±5.98 / 775.89 ms │ +1.10x faster │
│ QQuery 48 │ 305.21 / 308.21 ±3.07 / 313.62 ms │ 293.18 / 300.52 ±3.99 / 303.85 ms │ no change │
│ QQuery 49 │ 253.07 / 256.24 ±2.60 / 260.47 ms │ 254.55 / 256.53 ±1.66 / 259.07 ms │ no change │
│ QQuery 50 │ 236.80 / 241.07 ±4.04 / 247.89 ms │ 242.11 / 243.26 ±0.91 / 244.37 ms │ no change │
│ QQuery 51 │ 186.44 / 188.94 ±2.48 / 193.53 ms │ 185.45 / 188.58 ±3.47 / 194.10 ms │ no change │
│ QQuery 52 │ 110.48 / 111.88 ±1.04 / 113.45 ms │ 110.75 / 112.49 ±2.22 / 116.76 ms │ no change │
│ QQuery 53 │ 105.88 / 106.75 ±0.83 / 108.13 ms │ 105.47 / 106.43 ±1.31 / 108.98 ms │ no change │
│ QQuery 54 │ 153.75 / 154.88 ±1.15 / 156.55 ms │ 153.24 / 154.03 ±0.47 / 154.63 ms │ no change │
│ QQuery 55 │ 109.63 / 111.26 ±0.94 / 112.36 ms │ 110.56 / 112.82 ±1.48 / 114.33 ms │ no change │
│ QQuery 56 │ 146.42 / 147.53 ±1.00 / 149.09 ms │ 146.98 / 147.85 ±1.16 / 150.09 ms │ no change │
│ QQuery 57 │ 181.86 / 184.57 ±2.24 / 187.71 ms │ 181.71 / 183.00 ±1.81 / 186.46 ms │ no change │
│ QQuery 58 │ 298.26 / 308.58 ±7.84 / 321.57 ms │ 296.08 / 310.77 ±7.67 / 316.75 ms │ no change │
│ QQuery 59 │ 206.55 / 209.89 ±2.74 / 214.13 ms │ 206.12 / 209.00 ±1.85 / 211.88 ms │ no change │
│ QQuery 60 │ 148.35 / 150.01 ±1.46 / 152.41 ms │ 147.22 / 148.98 ±1.84 / 152.03 ms │ no change │
│ QQuery 61 │ 13.67 / 13.97 ±0.24 / 14.34 ms │ 13.65 / 13.98 ±0.26 / 14.36 ms │ no change │
│ QQuery 62 │ 946.34 / 1017.54 ±74.86 / 1124.92 ms │ 930.85 / 1012.18 ±66.28 / 1090.47 ms │ no change │
│ QQuery 63 │ 107.54 / 110.05 ±2.14 / 113.32 ms │ 105.43 / 108.00 ±2.76 / 113.19 ms │ no change │
│ QQuery 64 │ 718.35 / 723.29 ±2.86 / 726.63 ms │ 717.62 / 721.52 ±3.23 / 727.18 ms │ no change │
│ QQuery 65 │ 275.42 / 277.76 ±1.96 / 280.73 ms │ 277.79 / 279.16 ±2.09 / 283.31 ms │ no change │
│ QQuery 66 │ 264.87 / 272.85 ±9.96 / 290.49 ms │ 248.93 / 259.50 ±7.79 / 270.93 ms │ no change │
│ QQuery 67 │ 328.71 / 336.65 ±6.27 / 343.58 ms │ 316.38 / 326.19 ±6.53 / 334.20 ms │ no change │
│ QQuery 68 │ 9.02 / 10.47 ±1.07 / 11.48 ms │ 8.97 / 10.05 ±0.77 / 10.93 ms │ no change │
│ QQuery 69 │ 105.22 / 107.39 ±2.27 / 111.60 ms │ 105.57 / 106.80 ±0.98 / 107.94 ms │ no change │
│ QQuery 70 │ 344.39 / 353.86 ±5.99 / 362.63 ms │ 356.14 / 368.98 ±10.50 / 383.29 ms │ no change │
│ QQuery 71 │ 139.33 / 142.20 ±3.47 / 148.84 ms │ 139.52 / 142.49 ±1.72 / 144.29 ms │ no change │
│ QQuery 72 │ 646.05 / 657.98 ±9.31 / 669.42 ms │ 631.94 / 652.23 ±10.59 / 662.73 ms │ no change │
│ QQuery 73 │ 7.52 / 9.18 ±1.86 / 12.20 ms │ 7.53 / 8.77 ±0.72 / 9.50 ms │ no change │
│ QQuery 74 │ 671.57 / 679.06 ±4.60 / 685.79 ms │ 668.16 / 677.40 ±6.13 / 684.99 ms │ no change │
│ QQuery 75 │ 282.12 / 285.63 ±2.59 / 289.68 ms │ 280.07 / 284.25 ±2.60 / 287.05 ms │ no change │
│ QQuery 76 │ 136.76 / 138.09 ±1.33 / 140.40 ms │ 136.80 / 138.85 ±1.32 / 140.81 ms │ no change │
│ QQuery 77 │ 194.37 / 195.03 ±0.58 / 196.10 ms │ 192.21 / 194.23 ±1.51 / 196.62 ms │ no change │
│ QQuery 78 │ 351.46 / 356.85 ±3.19 / 360.97 ms │ 356.05 / 358.86 ±2.76 / 363.72 ms │ no change │
│ QQuery 79 │ 255.74 / 259.14 ±3.09 / 264.40 ms │ 258.26 / 259.52 ±1.29 / 261.90 ms │ no change │
│ QQuery 80 │ 325.95 / 329.88 ±3.09 / 334.40 ms │ 324.20 / 327.60 ±2.30 / 331.03 ms │ no change │
│ QQuery 81 │ 27.51 / 29.23 ±1.58 / 31.46 ms │ 26.76 / 28.21 ±1.00 / 29.45 ms │ no change │
│ QQuery 82 │ 203.66 / 204.55 ±0.84 / 205.95 ms │ 206.60 / 208.28 ±1.55 / 211.06 ms │ no change │
│ QQuery 83 │ 39.00 / 40.87 ±1.30 / 42.99 ms │ 40.53 / 41.75 ±0.97 / 43.35 ms │ no change │
│ QQuery 84 │ 49.30 / 50.49 ±0.86 / 51.54 ms │ 49.35 / 50.49 ±1.08 / 52.54 ms │ no change │
│ QQuery 85 │ 153.17 / 155.31 ±1.68 / 157.38 ms │ 151.61 / 154.34 ±2.77 / 159.09 ms │ no change │
│ QQuery 86 │ 40.40 / 42.08 ±1.06 / 43.57 ms │ 40.63 / 42.67 ±1.58 / 44.84 ms │ no change │
│ QQuery 87 │ 89.48 / 92.16 ±3.08 / 97.99 ms │ 89.92 / 93.35 ±3.05 / 98.80 ms │ no change │
│ QQuery 88 │ 103.17 / 104.60 ±1.31 / 106.98 ms │ 104.07 / 106.06 ±1.04 / 107.02 ms │ no change │
│ QQuery 89 │ 123.01 / 124.15 ±0.75 / 125.09 ms │ 121.53 / 123.22 ±1.11 / 124.43 ms │ no change │
│ QQuery 90 │ 24.09 / 25.32 ±0.80 / 26.35 ms │ 25.20 / 25.51 ±0.31 / 26.06 ms │ no change │
│ QQuery 91 │ 66.68 / 67.55 ±0.59 / 68.32 ms │ 64.26 / 66.12 ±1.27 / 68.09 ms │ no change │
│ QQuery 92 │ 59.42 / 60.18 ±0.53 / 61.05 ms │ 59.74 / 60.85 ±0.89 / 62.29 ms │ no change │
│ QQuery 93 │ 193.92 / 196.43 ±1.51 / 198.18 ms │ 193.72 / 196.28 ±1.44 / 197.72 ms │ no change │
│ QQuery 94 │ 62.62 / 64.31 ±1.13 / 65.50 ms │ 62.67 / 64.10 ±0.94 / 65.49 ms │ no change │
│ QQuery 95 │ 130.58 / 132.08 ±1.69 / 135.29 ms │ 130.21 / 131.71 ±1.14 / 133.64 ms │ no change │
│ QQuery 96 │ 75.60 / 76.17 ±0.55 / 77.08 ms │ 76.23 / 77.30 ±0.96 / 78.74 ms │ no change │
│ QQuery 97 │ 131.57 / 133.12 ±1.45 / 135.29 ms │ 130.52 / 133.66 ±1.76 / 135.70 ms │ no change │
│ QQuery 98 │ 160.01 / 162.40 ±1.28 / 163.45 ms │ 161.44 / 163.90 ±1.52 / 165.84 ms │ no change │
│ QQuery 99 │ 10880.10 / 10940.21 ±43.92 / 10996.32 ms │ 10944.10 / 10981.76 ±26.56 / 11024.16 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 32922.32ms │
│ Total Time (early-group-emission) │ 33523.65ms │
│ Average Time (HEAD) │ 332.55ms │
│ Average Time (early-group-emission) │ 338.62ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 1 │
│ Queries with No Change │ 95 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 165.0s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.6 GiB |
| CPU user | 273.3s |
| CPU sys | 17.9s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 167.9s |
| Peak memory | 5.5 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 279.7s |
| CPU sys | 18.9s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4239111224
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.58 / 7.02 ±0.78 / 8.57 ms │ 6.58 / 7.04 ±0.79 / 8.61 ms │ no change │
│ QQuery 2 │ 144.42 / 145.00 ±0.46 / 145.50 ms │ 143.79 / 145.40 ±0.84 / 146.05 ms │ no change │
│ QQuery 3 │ 113.37 / 114.74 ±0.74 / 115.60 ms │ 113.15 / 114.40 ±0.98 / 115.85 ms │ no change │
│ QQuery 4 │ 1270.53 / 1310.57 ±25.78 / 1339.29 ms │ 1287.14 / 1307.68 ±17.91 / 1331.64 ms │ no change │
│ QQuery 5 │ 171.71 / 174.48 ±1.83 / 176.62 ms │ 172.11 / 174.29 ±1.46 / 176.17 ms │ no change │
│ QQuery 6 │ 828.20 / 837.00 ±11.37 / 859.11 ms │ 826.49 / 854.23 ±27.44 / 900.51 ms │ no change │
│ QQuery 7 │ 342.17 / 343.90 ±2.18 / 348.05 ms │ 336.67 / 340.18 ±1.88 / 341.86 ms │ no change │
│ QQuery 8 │ 115.22 / 116.24 ±1.06 / 117.61 ms │ 115.13 / 116.36 ±0.91 / 117.41 ms │ no change │
│ QQuery 9 │ 99.91 / 105.01 ±2.81 / 107.63 ms │ 100.85 / 103.20 ±2.61 / 107.24 ms │ no change │
│ QQuery 10 │ 106.37 / 107.17 ±1.23 / 109.56 ms │ 106.01 / 106.76 ±0.50 / 107.40 ms │ no change │
│ QQuery 11 │ 877.49 / 888.44 ±11.18 / 909.70 ms │ 886.55 / 902.98 ±12.70 / 923.44 ms │ no change │
│ QQuery 12 │ 44.75 / 45.58 ±1.02 / 47.57 ms │ 44.04 / 45.98 ±1.62 / 48.47 ms │ no change │
│ QQuery 13 │ 391.53 / 397.63 ±3.36 / 401.21 ms │ 394.73 / 398.00 ±1.98 / 399.88 ms │ no change │
│ QQuery 14 │ 992.57 / 1001.42 ±4.64 / 1006.31 ms │ 992.48 / 1000.51 ±5.57 / 1007.69 ms │ no change │
│ QQuery 15 │ 15.54 / 15.97 ±0.47 / 16.73 ms │ 14.65 / 15.41 ±0.70 / 16.69 ms │ no change │
│ QQuery 16 │ 7.08 / 7.47 ±0.42 / 8.22 ms │ 7.05 / 7.17 ±0.10 / 7.27 ms │ no change │
│ QQuery 17 │ 225.25 / 227.03 ±1.17 / 228.53 ms │ 225.09 / 227.02 ±1.77 / 229.69 ms │ no change │
│ QQuery 18 │ 125.87 / 126.77 ±0.68 / 127.64 ms │ 125.52 / 127.13 ±1.26 / 129.35 ms │ no change │
│ QQuery 19 │ 153.96 / 155.21 ±0.96 / 156.48 ms │ 153.42 / 155.54 ±1.30 / 156.82 ms │ no change │
│ QQuery 20 │ 13.45 / 13.99 ±0.29 / 14.28 ms │ 13.88 / 14.25 ±0.30 / 14.76 ms │ no change │
│ QQuery 21 │ 19.20 / 19.58 ±0.25 / 19.89 ms │ 19.34 / 19.70 ±0.33 / 20.12 ms │ no change │
│ QQuery 22 │ 479.09 / 480.13 ±1.32 / 482.68 ms │ 482.56 / 484.88 ±2.17 / 488.55 ms │ no change │
│ QQuery 23 │ 852.42 / 864.47 ±10.97 / 878.65 ms │ 847.03 / 858.13 ±9.27 / 869.69 ms │ no change │
│ QQuery 24 │ 377.52 / 380.01 ±2.59 / 384.75 ms │ 374.53 / 378.36 ±4.08 / 386.27 ms │ no change │
│ QQuery 25 │ 336.70 / 337.96 ±0.86 / 339.39 ms │ 335.98 / 337.72 ±1.58 / 340.49 ms │ no change │
│ QQuery 26 │ 80.00 / 82.47 ±2.17 / 86.47 ms │ 79.31 / 81.09 ±1.38 / 82.91 ms │ no change │
│ QQuery 27 │ 6.70 / 6.91 ±0.17 / 7.22 ms │ 6.82 / 7.11 ±0.37 / 7.78 ms │ no change │
│ QQuery 28 │ 147.94 / 149.99 ±1.70 / 152.67 ms │ 147.53 / 148.52 ±0.86 / 150.13 ms │ no change │
│ QQuery 29 │ 277.34 / 278.85 ±1.54 / 280.93 ms │ 276.96 / 277.89 ±0.95 / 279.59 ms │ no change │
│ QQuery 30 │ 43.12 / 44.85 ±2.06 / 48.83 ms │ 43.59 / 44.63 ±0.88 / 46.12 ms │ no change │
│ QQuery 31 │ 169.00 / 170.34 ±0.85 / 171.32 ms │ 170.14 / 172.31 ±2.24 / 176.13 ms │ no change │
│ QQuery 32 │ 55.84 / 58.18 ±1.71 / 60.37 ms │ 56.45 / 57.17 ±0.54 / 57.91 ms │ no change │
│ QQuery 33 │ 139.21 / 140.76 ±1.52 / 143.08 ms │ 140.35 / 142.43 ±1.25 / 144.03 ms │ no change │
│ QQuery 34 │ 6.83 / 7.40 ±0.74 / 8.82 ms │ 6.93 / 7.31 ±0.47 / 8.20 ms │ no change │
│ QQuery 35 │ 107.20 / 109.27 ±1.75 / 112.51 ms │ 105.43 / 107.98 ±2.62 / 112.49 ms │ no change │
│ QQuery 36 │ 6.72 / 6.97 ±0.18 / 7.27 ms │ 6.47 / 6.69 ±0.20 / 6.92 ms │ no change │
│ QQuery 37 │ 8.30 / 9.09 ±0.67 / 10.32 ms │ 8.12 / 9.11 ±0.70 / 10.08 ms │ no change │
│ QQuery 38 │ 86.20 / 88.34 ±2.43 / 92.88 ms │ 81.85 / 85.96 ±4.02 / 93.62 ms │ no change │
│ QQuery 39 │ 124.11 / 125.05 ±0.87 / 126.68 ms │ 123.80 / 125.22 ±1.56 / 128.24 ms │ no change │
│ QQuery 40 │ 110.53 / 115.19 ±5.90 / 126.70 ms │ 104.81 / 111.89 ±6.18 / 122.86 ms │ no change │
│ QQuery 41 │ 14.02 / 14.77 ±0.71 / 15.98 ms │ 14.11 / 15.10 ±0.82 / 16.01 ms │ no change │
│ QQuery 42 │ 108.06 / 110.03 ±2.65 / 114.84 ms │ 107.85 / 109.68 ±1.08 / 110.69 ms │ no change │
│ QQuery 43 │ 5.91 / 6.00 ±0.10 / 6.19 ms │ 5.87 / 6.06 ±0.19 / 6.40 ms │ no change │
│ QQuery 44 │ 11.47 / 11.86 ±0.54 / 12.94 ms │ 11.41 / 11.64 ±0.12 / 11.77 ms │ no change │
│ QQuery 45 │ 49.99 / 50.78 ±0.81 / 52.26 ms │ 49.27 / 50.31 ±1.09 / 52.12 ms │ no change │
│ QQuery 46 │ 8.40 / 8.59 ±0.14 / 8.74 ms │ 8.19 / 8.95 ±1.02 / 10.88 ms │ no change │
│ QQuery 47 │ 686.62 / 692.04 ±3.28 / 696.88 ms │ 624.22 / 629.84 ±4.02 / 634.83 ms │ +1.10x faster │
│ QQuery 48 │ 289.11 / 292.36 ±3.19 / 297.00 ms │ 281.54 / 289.13 ±5.57 / 298.52 ms │ no change │
│ QQuery 49 │ 250.34 / 253.48 ±2.17 / 256.41 ms │ 250.45 / 252.21 ±1.28 / 253.80 ms │ no change │
│ QQuery 50 │ 216.27 / 225.28 ±6.16 / 233.17 ms │ 218.15 / 221.26 ±3.22 / 226.69 ms │ no change │
│ QQuery 51 │ 177.07 / 180.28 ±2.32 / 183.55 ms │ 178.15 / 182.02 ±2.91 / 186.60 ms │ no change │
│ QQuery 52 │ 106.89 / 108.60 ±1.09 / 110.29 ms │ 107.47 / 108.00 ±0.41 / 108.67 ms │ no change │
│ QQuery 53 │ 101.28 / 101.95 ±0.44 / 102.51 ms │ 102.19 / 102.84 ±0.52 / 103.69 ms │ no change │
│ QQuery 54 │ 144.93 / 146.15 ±1.12 / 148.06 ms │ 142.42 / 146.22 ±2.05 / 148.21 ms │ no change │
│ QQuery 55 │ 106.02 / 107.49 ±1.18 / 109.00 ms │ 106.51 / 108.13 ±1.21 / 109.47 ms │ no change │
│ QQuery 56 │ 140.90 / 142.11 ±0.78 / 143.03 ms │ 140.51 / 141.75 ±1.37 / 144.18 ms │ no change │
│ QQuery 57 │ 170.23 / 172.54 ±1.55 / 174.40 ms │ 173.18 / 174.85 ±2.21 / 179.00 ms │ no change │
│ QQuery 58 │ 289.32 / 294.33 ±3.16 / 299.16 ms │ 285.79 / 296.54 ±6.49 / 302.28 ms │ no change │
│ QQuery 59 │ 200.10 / 202.47 ±2.66 / 207.65 ms │ 195.48 / 198.11 ±1.94 / 200.46 ms │ no change │
│ QQuery 60 │ 143.56 / 144.87 ±1.25 / 146.80 ms │ 141.29 / 144.07 ±1.73 / 146.21 ms │ no change │
│ QQuery 61 │ 13.01 / 13.22 ±0.23 / 13.65 ms │ 12.81 / 12.97 ±0.24 / 13.43 ms │ no change │
│ QQuery 62 │ 890.56 / 942.79 ±47.17 / 1001.22 ms │ 915.02 / 946.40 ±36.41 / 1006.36 ms │ no change │
│ QQuery 63 │ 104.87 / 105.48 ±0.38 / 105.89 ms │ 103.00 / 104.56 ±1.03 / 105.88 ms │ no change │
│ QQuery 64 │ 680.68 / 687.62 ±4.27 / 692.20 ms │ 671.02 / 676.39 ±4.47 / 683.50 ms │ no change │
│ QQuery 65 │ 246.90 / 249.25 ±1.33 / 250.68 ms │ 244.62 / 250.82 ±5.52 / 260.00 ms │ no change │
│ QQuery 66 │ 238.55 / 253.09 ±7.83 / 260.18 ms │ 238.43 / 257.94 ±15.75 / 284.86 ms │ no change │
│ QQuery 67 │ 305.36 / 309.83 ±4.00 / 317.19 ms │ 309.20 / 316.05 ±5.75 / 325.31 ms │ no change │
│ QQuery 68 │ 9.07 / 10.53 ±0.99 / 12.04 ms │ 8.77 / 11.35 ±1.72 / 13.81 ms │ 1.08x slower │
│ QQuery 69 │ 102.10 / 103.97 ±1.46 / 106.17 ms │ 103.75 / 105.20 ±1.10 / 107.17 ms │ no change │
│ QQuery 70 │ 333.33 / 346.68 ±13.24 / 371.65 ms │ 330.17 / 345.93 ±11.65 / 365.97 ms │ no change │
│ QQuery 71 │ 133.59 / 134.97 ±1.57 / 137.87 ms │ 134.79 / 137.35 ±1.63 / 139.60 ms │ no change │
│ QQuery 72 │ 609.16 / 615.00 ±4.45 / 619.36 ms │ 601.91 / 617.69 ±9.51 / 628.07 ms │ no change │
│ QQuery 73 │ 6.70 / 7.87 ±0.96 / 9.43 ms │ 7.64 / 9.04 ±1.82 / 12.50 ms │ 1.15x slower │
│ QQuery 74 │ 547.93 / 553.45 ±3.94 / 558.97 ms │ 549.16 / 551.64 ±2.80 / 555.64 ms │ no change │
│ QQuery 75 │ 273.45 / 276.46 ±2.16 / 279.75 ms │ 273.59 / 276.14 ±2.24 / 279.44 ms │ no change │
│ QQuery 76 │ 131.18 / 133.67 ±1.63 / 136.31 ms │ 129.46 / 130.41 ±0.76 / 131.69 ms │ no change │
│ QQuery 77 │ 187.03 / 188.45 ±1.33 / 190.41 ms │ 188.74 / 190.30 ±1.61 / 193.17 ms │ no change │
│ QQuery 78 │ 334.22 / 338.85 ±3.44 / 344.37 ms │ 337.80 / 339.62 ±1.19 / 340.96 ms │ no change │
│ QQuery 79 │ 229.45 / 231.91 ±1.80 / 234.88 ms │ 229.80 / 231.66 ±1.58 / 233.87 ms │ no change │
│ QQuery 80 │ 318.94 / 323.07 ±3.49 / 327.85 ms │ 319.71 / 321.80 ±1.71 / 324.63 ms │ no change │
│ QQuery 81 │ 25.56 / 26.81 ±1.20 / 28.56 ms │ 25.47 / 26.56 ±0.71 / 27.52 ms │ no change │
│ QQuery 82 │ 200.08 / 200.78 ±0.45 / 201.29 ms │ 198.62 / 201.63 ±2.05 / 204.37 ms │ no change │
│ QQuery 83 │ 38.14 / 39.08 ±0.89 / 40.67 ms │ 38.04 / 39.40 ±0.98 / 40.44 ms │ no change │
│ QQuery 84 │ 47.69 / 48.76 ±0.61 / 49.39 ms │ 48.01 / 49.15 ±1.03 / 50.87 ms │ no change │
│ QQuery 85 │ 146.78 / 148.09 ±1.26 / 150.05 ms │ 145.28 / 147.01 ±1.69 / 150.22 ms │ no change │
│ QQuery 86 │ 38.33 / 39.20 ±0.60 / 40.21 ms │ 38.42 / 39.71 ±0.98 / 40.98 ms │ no change │
│ QQuery 87 │ 86.24 / 89.51 ±2.73 / 93.18 ms │ 85.22 / 88.36 ±2.46 / 91.54 ms │ no change │
│ QQuery 88 │ 100.97 / 102.03 ±0.68 / 102.70 ms │ 99.47 / 100.53 ±1.37 / 103.18 ms │ no change │
│ QQuery 89 │ 119.24 / 120.12 ±0.70 / 120.99 ms │ 117.28 / 118.28 ±0.75 / 119.42 ms │ no change │
│ QQuery 90 │ 23.79 / 24.56 ±0.63 / 25.31 ms │ 23.18 / 23.81 ±0.62 / 24.99 ms │ no change │
│ QQuery 91 │ 63.50 / 64.66 ±1.12 / 66.80 ms │ 64.13 / 65.38 ±1.27 / 67.69 ms │ no change │
│ QQuery 92 │ 55.76 / 58.18 ±1.50 / 60.42 ms │ 57.24 / 58.06 ±0.76 / 59.32 ms │ no change │
│ QQuery 93 │ 183.08 / 185.92 ±1.63 / 187.69 ms │ 181.74 / 184.63 ±1.75 / 186.16 ms │ no change │
│ QQuery 94 │ 61.16 / 61.66 ±0.70 / 63.06 ms │ 59.55 / 60.95 ±1.05 / 62.16 ms │ no change │
│ QQuery 95 │ 127.01 / 128.28 ±1.18 / 129.70 ms │ 125.10 / 126.97 ±1.72 / 129.06 ms │ no change │
│ QQuery 96 │ 73.20 / 74.10 ±0.63 / 74.80 ms │ 71.12 / 73.89 ±1.55 / 75.31 ms │ no change │
│ QQuery 97 │ 123.09 / 125.24 ±1.68 / 127.80 ms │ 125.79 / 128.44 ±1.79 / 130.76 ms │ no change │
│ QQuery 98 │ 151.50 / 153.95 ±1.67 / 155.91 ms │ 153.39 / 154.93 ±0.85 / 155.84 ms │ no change │
│ QQuery 99 │ 10695.33 / 10744.26 ±25.98 / 10766.07 ms │ 10723.67 / 10753.20 ±29.57 / 10805.54 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31143.82ms │
│ Total Time (early-group-emission) │ 31098.46ms │
│ Average Time (HEAD) │ 314.58ms │
│ Average Time (early-group-emission) │ 314.13ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 96 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 156.0s |
| Peak memory | 5.2 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 257.5s |
| CPU sys | 16.6s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 155.8s |
| Peak memory | 5.3 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 257.0s |
| CPU sys | 16.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241506095
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241497116-1187-2pfcx 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241603046
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241595190-1194-fhrtb 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (0b08df010715efd91b6237e70649d0310b383f90) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..0b08df010715efd91b6237e70649d0310b383f90) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241602990
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241595190-1195-jh424 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (0b08df010715efd91b6237e70649d0310b383f90) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..0b08df010715efd91b6237e70649d0310b383f90) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238961077
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238944023-1180-7xztf 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (1390b5ece2b27816c913beb8b77aacf335207181) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..1390b5ece2b27816c913beb8b77aacf335207181) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238496433
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238481303-1171-z9mwd 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2006bca0b43a2509f533c2c056ac77aab89b86d3) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2006bca0b43a2509f533c2c056ac77aab89b86d3) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238897488
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.69 / 7.17 ±0.84 / 8.84 ms │ 6.75 / 7.18 ±0.79 / 8.76 ms │ no change │
│ QQuery 2 │ 145.10 / 145.69 ±0.44 / 146.27 ms │ 145.36 / 146.06 ±0.50 / 146.71 ms │ no change │
│ QQuery 3 │ 114.02 / 114.93 ±0.56 / 115.75 ms │ 113.23 / 113.62 ±0.28 / 114.06 ms │ no change │
│ QQuery 4 │ 1300.48 / 1325.56 ±16.73 / 1353.24 ms │ 1249.70 / 1286.58 ±18.76 / 1299.41 ms │ no change │
│ QQuery 5 │ 171.65 / 174.62 ±2.10 / 177.86 ms │ 173.07 / 174.75 ±1.22 / 176.88 ms │ no change │
│ QQuery 6 │ 821.76 / 848.86 ±17.70 / 871.87 ms │ 825.04 / 846.21 ±16.80 / 874.79 ms │ no change │
│ QQuery 7 │ 343.04 / 344.27 ±1.12 / 346.16 ms │ 339.23 / 343.68 ±2.86 / 347.67 ms │ no change │
│ QQuery 8 │ 116.36 / 117.66 ±0.76 / 118.59 ms │ 116.18 / 117.17 ±0.94 / 118.90 ms │ no change │
│ QQuery 9 │ 100.82 / 103.39 ±2.97 / 107.82 ms │ 101.47 / 104.25 ±2.68 / 107.67 ms │ no change │
│ QQuery 10 │ 106.62 / 108.48 ±1.02 / 109.27 ms │ 105.19 / 108.14 ±1.66 / 109.86 ms │ no change │
│ QQuery 11 │ 867.94 / 888.25 ±17.66 / 909.34 ms │ 889.06 / 894.00 ±4.41 / 899.75 ms │ no change │
│ QQuery 12 │ 44.89 / 46.60 ±1.26 / 48.70 ms │ 46.07 / 47.43 ±1.67 / 50.72 ms │ no change │
│ QQuery 13 │ 400.39 / 404.45 ±2.88 / 408.54 ms │ 399.04 / 401.97 ±2.58 / 405.61 ms │ no change │
│ QQuery 14 │ 1009.67 / 1015.73 ±5.03 / 1024.65 ms │ 1001.52 / 1010.57 ±7.69 / 1024.83 ms │ no change │
│ QQuery 15 │ 14.97 / 16.36 ±1.25 / 18.71 ms │ 15.28 / 16.67 ±0.99 / 17.99 ms │ no change │
│ QQuery 16 │ 7.15 / 7.37 ±0.26 / 7.83 ms │ 7.52 / 8.52 ±1.14 / 10.64 ms │ 1.16x slower │
│ QQuery 17 │ 230.11 / 231.59 ±1.32 / 233.84 ms │ 228.85 / 230.95 ±1.38 / 232.37 ms │ no change │
│ QQuery 18 │ 126.89 / 128.59 ±1.00 / 130.01 ms │ 128.09 / 128.94 ±0.69 / 130.06 ms │ no change │
│ QQuery 19 │ 153.87 / 155.31 ±0.99 / 156.77 ms │ 154.91 / 156.19 ±0.99 / 157.76 ms │ no change │
│ QQuery 20 │ 13.74 / 14.08 ±0.30 / 14.63 ms │ 13.57 / 14.20 ±0.41 / 14.73 ms │ no change │
│ QQuery 21 │ 18.69 / 19.33 ±0.59 / 20.38 ms │ 19.16 / 19.54 ±0.24 / 19.81 ms │ no change │
│ QQuery 22 │ 485.46 / 487.67 ±2.07 / 490.79 ms │ 487.41 / 491.67 ±4.04 / 498.14 ms │ no change │
│ QQuery 23 │ 857.56 / 866.23 ±9.78 / 884.48 ms │ 848.09 / 862.11 ±10.17 / 877.45 ms │ no change │
│ QQuery 24 │ 377.74 / 382.24 ±2.66 / 385.08 ms │ 379.96 / 383.31 ±2.49 / 386.58 ms │ no change │
│ QQuery 25 │ 340.92 / 341.57 ±0.83 / 343.19 ms │ 339.14 / 342.08 ±2.53 / 346.23 ms │ no change │
│ QQuery 26 │ 81.21 / 82.86 ±1.37 / 84.73 ms │ 80.67 / 81.48 ±0.62 / 82.08 ms │ no change │
│ QQuery 27 │ 6.82 / 7.64 ±0.67 / 8.69 ms │ 6.97 / 7.35 ±0.50 / 8.32 ms │ no change │
│ QQuery 28 │ 148.18 / 149.95 ±1.35 / 151.68 ms │ 149.80 / 150.38 ±0.60 / 151.42 ms │ no change │
│ QQuery 29 │ 280.66 / 282.79 ±1.30 / 284.22 ms │ 280.29 / 282.78 ±2.07 / 286.45 ms │ no change │
│ QQuery 30 │ 42.23 / 44.61 ±1.68 / 46.95 ms │ 43.78 / 45.52 ±1.12 / 46.80 ms │ no change │
│ QQuery 31 │ 169.47 / 170.90 ±1.21 / 172.57 ms │ 170.22 / 172.08 ±1.06 / 173.04 ms │ no change │
│ QQuery 32 │ 56.55 / 57.23 ±0.69 / 58.47 ms │ 56.76 / 57.79 ±0.61 / 58.53 ms │ no change │
│ QQuery 33 │ 139.39 / 140.91 ±0.84 / 141.93 ms │ 139.76 / 141.70 ±1.71 / 144.50 ms │ no change │
│ QQuery 34 │ 7.00 / 7.19 ±0.26 / 7.69 ms │ 7.04 / 7.28 ±0.18 / 7.53 ms │ no change │
│ QQuery 35 │ 107.56 / 110.65 ±1.81 / 112.99 ms │ 109.42 / 110.32 ±0.94 / 112.07 ms │ no change │
│ QQuery 36 │ 6.64 / 6.86 ±0.31 / 7.47 ms │ 6.64 / 6.96 ±0.37 / 7.67 ms │ no change │
│ QQuery 37 │ 8.29 / 8.91 ±0.32 / 9.16 ms │ 8.64 / 9.11 ±0.30 / 9.46 ms │ no change │
│ QQuery 38 │ 85.92 / 88.71 ±1.55 / 90.15 ms │ 83.00 / 87.33 ±4.65 / 96.16 ms │ no change │
│ QQuery 39 │ 124.26 / 128.74 ±3.98 / 134.80 ms │ 123.68 / 127.37 ±3.08 / 132.10 ms │ no change │
│ QQuery 40 │ 108.67 / 115.76 ±7.00 / 129.08 ms │ 107.85 / 114.35 ±6.94 / 127.68 ms │ no change │
│ QQuery 41 │ 15.81 / 16.13 ±0.26 / 16.42 ms │ 14.23 / 15.53 ±0.96 / 16.98 ms │ no change │
│ QQuery 42 │ 108.55 / 110.57 ±1.61 / 112.82 ms │ 106.36 / 108.60 ±2.05 / 111.85 ms │ no change │
│ QQuery 43 │ 6.11 / 6.21 ±0.11 / 6.41 ms │ 6.18 / 6.29 ±0.13 / 6.54 ms │ no change │
│ QQuery 44 │ 12.16 / 12.96 ±0.48 / 13.43 ms │ 11.83 / 12.61 ±0.77 / 14.00 ms │ no change │
│ QQuery 45 │ 51.85 / 52.55 ±0.61 / 53.40 ms │ 50.87 / 52.05 ±0.86 / 53.38 ms │ no change │
│ QQuery 46 │ 8.97 / 9.33 ±0.22 / 9.61 ms │ 8.61 / 8.96 ±0.31 / 9.41 ms │ no change │
│ QQuery 47 │ 701.36 / 750.76 ±37.45 / 805.93 ms │ 629.64 / 636.43 ±4.41 / 642.81 ms │ +1.18x faster │
│ QQuery 48 │ 294.24 / 298.16 ±2.16 / 300.33 ms │ 291.75 / 297.27 ±5.06 / 303.56 ms │ no change │
│ QQuery 49 │ 256.63 / 260.92 ±2.80 / 263.70 ms │ 249.96 / 251.54 ±1.17 / 252.83 ms │ no change │
│ QQuery 50 │ 221.07 / 230.22 ±8.73 / 244.64 ms │ 223.85 / 228.06 ±4.14 / 234.99 ms │ no change │
│ QQuery 51 │ 179.25 / 182.71 ±2.85 / 187.95 ms │ 181.61 / 183.71 ±2.25 / 187.74 ms │ no change │
│ QQuery 52 │ 107.07 / 107.50 ±0.29 / 107.91 ms │ 107.04 / 108.96 ±1.32 / 110.53 ms │ no change │
│ QQuery 53 │ 102.56 / 104.19 ±0.90 / 105.08 ms │ 102.23 / 103.26 ±0.75 / 104.49 ms │ no change │
│ QQuery 54 │ 144.43 / 145.95 ±1.26 / 147.23 ms │ 144.75 / 146.19 ±0.91 / 147.25 ms │ no change │
│ QQuery 55 │ 107.22 / 109.48 ±1.24 / 110.86 ms │ 106.65 / 107.28 ±0.92 / 109.09 ms │ no change │
│ QQuery 56 │ 139.29 / 141.18 ±2.18 / 145.10 ms │ 140.19 / 141.73 ±2.12 / 145.94 ms │ no change │
│ QQuery 57 │ 172.44 / 175.32 ±1.64 / 177.12 ms │ 170.83 / 176.75 ±3.04 / 179.34 ms │ no change │
│ QQuery 58 │ 288.28 / 293.27 ±5.43 / 301.19 ms │ 290.44 / 297.07 ±5.32 / 305.42 ms │ no change │
│ QQuery 59 │ 199.42 / 202.79 ±3.17 / 208.32 ms │ 198.61 / 202.31 ±2.02 / 204.49 ms │ no change │
│ QQuery 60 │ 143.16 / 145.23 ±2.03 / 149.11 ms │ 144.22 / 145.72 ±1.15 / 147.50 ms │ no change │
│ QQuery 61 │ 13.01 / 13.59 ±0.52 / 14.35 ms │ 13.19 / 13.29 ±0.15 / 13.59 ms │ no change │
│ QQuery 62 │ 926.63 / 937.21 ±12.95 / 961.99 ms │ 859.06 / 967.03 ±100.15 / 1135.87 ms │ no change │
│ QQuery 63 │ 106.46 / 107.98 ±1.18 / 109.37 ms │ 107.03 / 107.86 ±0.97 / 109.59 ms │ no change │
│ QQuery 64 │ 675.55 / 686.19 ±7.75 / 698.90 ms │ 681.64 / 686.56 ±4.30 / 692.38 ms │ no change │
│ QQuery 65 │ 251.77 / 253.82 ±2.08 / 257.12 ms │ 249.89 / 253.40 ±2.19 / 256.68 ms │ no change │
│ QQuery 66 │ 245.18 / 253.46 ±6.78 / 265.31 ms │ 242.59 / 261.11 ±11.06 / 273.05 ms │ no change │
│ QQuery 67 │ 303.61 / 311.77 ±5.79 / 320.35 ms │ 307.38 / 319.90 ±7.97 / 326.87 ms │ no change │
│ QQuery 68 │ 8.66 / 10.16 ±1.15 / 11.85 ms │ 10.08 / 11.07 ±0.90 / 12.71 ms │ 1.09x slower │
│ QQuery 69 │ 105.14 / 106.41 ±0.92 / 107.60 ms │ 102.72 / 105.79 ±1.70 / 107.35 ms │ no change │
│ QQuery 70 │ 321.91 / 345.10 ±15.55 / 368.19 ms │ 341.85 / 353.13 ±12.31 / 376.85 ms │ no change │
│ QQuery 71 │ 135.33 / 137.96 ±2.60 / 141.50 ms │ 138.48 / 140.40 ±2.69 / 145.69 ms │ no change │
│ QQuery 72 │ 604.33 / 624.46 ±10.95 / 634.42 ms │ 616.82 / 656.87 ±29.25 / 699.87 ms │ 1.05x slower │
│ QQuery 73 │ 6.56 / 7.50 ±1.04 / 9.46 ms │ 6.91 / 8.70 ±1.80 / 11.89 ms │ 1.16x slower │
│ QQuery 74 │ 551.22 / 552.67 ±1.30 / 554.72 ms │ 549.02 / 559.59 ±9.07 / 575.82 ms │ no change │
│ QQuery 75 │ 272.38 / 277.75 ±3.32 / 281.21 ms │ 280.25 / 283.00 ±2.08 / 286.18 ms │ no change │
│ QQuery 76 │ 131.82 / 133.19 ±0.90 / 134.65 ms │ 131.34 / 134.63 ±1.91 / 136.60 ms │ no change │
│ QQuery 77 │ 187.01 / 189.24 ±1.20 / 190.42 ms │ 189.07 / 191.26 ±2.06 / 194.92 ms │ no change │
│ QQuery 78 │ 334.70 / 341.62 ±5.15 / 349.22 ms │ 336.77 / 344.93 ±5.83 / 354.95 ms │ no change │
│ QQuery 79 │ 231.85 / 233.86 ±1.68 / 236.05 ms │ 235.22 / 237.78 ±1.73 / 240.07 ms │ no change │
│ QQuery 80 │ 320.66 / 323.32 ±2.43 / 327.04 ms │ 324.14 / 325.66 ±1.59 / 327.63 ms │ no change │
│ QQuery 81 │ 26.68 / 27.77 ±0.92 / 28.96 ms │ 26.76 / 27.94 ±1.34 / 30.50 ms │ no change │
│ QQuery 82 │ 199.30 / 200.02 ±0.71 / 201.27 ms │ 199.83 / 201.59 ±1.71 / 203.93 ms │ no change │
│ QQuery 83 │ 38.74 / 40.12 ±1.25 / 42.09 ms │ 39.01 / 40.11 ±1.02 / 41.87 ms │ no change │
│ QQuery 84 │ 49.21 / 49.77 ±0.38 / 50.40 ms │ 49.21 / 49.57 ±0.23 / 49.85 ms │ no change │
│ QQuery 85 │ 149.45 / 150.16 ±0.71 / 151.33 ms │ 147.56 / 150.84 ±2.21 / 154.32 ms │ no change │
│ QQuery 86 │ 38.17 / 39.78 ±1.16 / 41.72 ms │ 38.64 / 40.25 ±1.14 / 41.59 ms │ no change │
│ QQuery 87 │ 85.09 / 88.91 ±3.90 / 96.08 ms │ 85.41 / 90.53 ±4.28 / 96.15 ms │ no change │
│ QQuery 88 │ 99.22 / 100.72 ±1.29 / 102.90 ms │ 99.88 / 100.96 ±0.77 / 101.80 ms │ no change │
│ QQuery 89 │ 117.80 / 119.76 ±1.27 / 121.81 ms │ 118.75 / 121.44 ±2.30 / 125.37 ms │ no change │
│ QQuery 90 │ 23.29 / 24.31 ±0.62 / 24.96 ms │ 23.38 / 23.87 ±0.36 / 24.42 ms │ no change │
│ QQuery 91 │ 63.05 / 64.66 ±1.27 / 66.42 ms │ 62.69 / 64.92 ±1.32 / 66.76 ms │ no change │
│ QQuery 92 │ 56.51 / 57.76 ±1.09 / 59.54 ms │ 56.80 / 58.34 ±0.84 / 59.34 ms │ no change │
│ QQuery 93 │ 184.06 / 187.29 ±1.80 / 189.22 ms │ 184.29 / 187.15 ±2.47 / 190.74 ms │ no change │
│ QQuery 94 │ 62.47 / 62.89 ±0.38 / 63.37 ms │ 60.72 / 62.42 ±1.51 / 65.21 ms │ no change │
│ QQuery 95 │ 127.10 / 128.68 ±1.26 / 130.00 ms │ 127.55 / 128.17 ±0.34 / 128.61 ms │ no change │
│ QQuery 96 │ 71.45 / 73.13 ±1.04 / 74.18 ms │ 73.75 / 75.94 ±1.18 / 76.98 ms │ no change │
│ QQuery 97 │ 124.39 / 128.51 ±2.29 / 131.26 ms │ 126.50 / 129.15 ±2.10 / 131.88 ms │ no change │
│ QQuery 98 │ 154.60 / 156.52 ±1.73 / 158.66 ms │ 155.05 / 156.72 ±0.93 / 157.60 ms │ no change │
│ QQuery 99 │ 10741.04 / 10755.74 ±17.90 / 10786.66 ms │ 10776.79 / 10849.69 ±51.16 / 10921.29 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31358.91ms │
│ Total Time (early-group-emission) │ 31409.43ms │
│ Average Time (HEAD) │ 316.76ms │
│ Average Time (early-group-emission) │ 317.27ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 94 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 157.2s |
| Peak memory | 5.3 GiB |
| Avg memory | 4.3 GiB |
| CPU user | 258.9s |
| CPU sys | 17.3s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 157.3s |
| Peak memory | 4.6 GiB |
| Avg memory | 4.0 GiB |
| CPU user | 260.7s |
| CPU sys | 17.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238689918
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238673913-1172-m8tjl 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (9619b00c04d25bf10ef24bd090359784ec828b00) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..9619b00c04d25bf10ef24bd090359784ec828b00) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238999229
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238986171-1183-szj4s 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241695295
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241683717)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241683717-1196-ksph2 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (0b08df010715efd91b6237e70649d0310b383f90) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..0b08df010715efd91b6237e70649d0310b383f90) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4700439432
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241666153
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.17 / 4.37 ±6.31 / 16.99 ms │ 1.19 / 4.40 ±6.32 / 17.03 ms │ no change │
│ QQuery 1 │ 14.23 / 14.48 ±0.14 / 14.66 ms │ 14.16 / 14.46 ±0.18 / 14.68 ms │ no change │
│ QQuery 2 │ 42.81 / 43.45 ±0.39 / 43.84 ms │ 44.02 / 44.23 ±0.13 / 44.41 ms │ no change │
│ QQuery 3 │ 41.50 / 43.34 ±1.88 / 46.62 ms │ 42.46 / 44.82 ±1.85 / 47.09 ms │ no change │
│ QQuery 4 │ 280.98 / 286.91 ±3.76 / 291.92 ms │ 276.22 / 296.21 ±15.05 / 315.38 ms │ no change │
│ QQuery 5 │ 335.17 / 340.41 ±4.34 / 346.36 ms │ 340.08 / 366.34 ±15.16 / 385.59 ms │ 1.08x slower │
│ QQuery 6 │ 4.90 / 5.22 ±0.26 / 5.62 ms │ 4.97 / 6.30 ±0.94 / 7.62 ms │ 1.21x slower │
│ QQuery 7 │ 16.29 / 16.65 ±0.29 / 17.17 ms │ 16.18 / 16.62 ±0.38 / 17.28 ms │ no change │
│ QQuery 8 │ 416.97 / 429.88 ±11.81 / 449.85 ms │ 407.26 / 423.95 ±9.46 / 434.39 ms │ no change │
│ QQuery 9 │ 625.65 / 635.85 ±7.56 / 644.78 ms │ 437.52 / 444.96 ±4.87 / 451.39 ms │ +1.43x faster │
│ QQuery 10 │ 91.09 / 94.06 ±4.21 / 102.38 ms │ 91.50 / 93.36 ±1.92 / 96.83 ms │ no change │
│ QQuery 11 │ 103.93 / 105.42 ±1.42 / 107.93 ms │ 104.23 / 106.74 ±1.87 / 109.25 ms │ no change │
│ QQuery 12 │ 334.38 / 338.07 ±3.19 / 343.63 ms │ 332.43 / 338.84 ±6.09 / 349.29 ms │ no change │
│ QQuery 13 │ 459.36 / 466.35 ±6.44 / 477.71 ms │ 465.79 / 484.84 ±11.12 / 500.42 ms │ no change │
│ QQuery 14 │ 339.28 / 342.30 ±2.76 / 347.11 ms │ 342.13 / 351.41 ±7.54 / 360.35 ms │ no change │
│ QQuery 15 │ 351.96 / 369.69 ±16.90 / 390.37 ms │ 324.80 / 335.43 ±5.45 / 339.96 ms │ +1.10x faster │
│ QQuery 16 │ 697.84 / 711.14 ±9.81 / 724.68 ms │ 661.09 / 706.03 ±36.70 / 747.27 ms │ no change │
│ QQuery 17 │ 690.80 / 700.68 ±6.16 / 708.80 ms │ 654.08 / 677.95 ±16.28 / 699.32 ms │ no change │
│ QQuery 18 │ 1339.70 / 1366.19 ±16.23 / 1388.52 ms │ 1295.63 / 1328.55 ±22.65 / 1359.99 ms │ no change │
│ QQuery 19 │ 34.77 / 35.70 ±0.79 / 36.85 ms │ 36.02 / 38.58 ±1.58 / 40.56 ms │ 1.08x slower │
│ QQuery 20 │ 707.92 / 718.78 ±13.55 / 743.79 ms │ 711.50 / 725.84 ±15.09 / 752.82 ms │ no change │
│ QQuery 21 │ 754.69 / 758.34 ±2.30 / 761.10 ms │ 756.80 / 763.60 ±4.47 / 770.54 ms │ no change │
│ QQuery 22 │ 1122.46 / 1130.25 ±10.87 / 1151.66 ms │ 1129.11 / 1132.76 ±3.24 / 1138.26 ms │ no change │
│ QQuery 23 │ 3068.27 / 3087.20 ±22.26 / 3128.43 ms │ 3026.72 / 3051.54 ±17.52 / 3072.17 ms │ no change │
│ QQuery 24 │ 99.60 / 103.37 ±3.16 / 107.61 ms │ 98.56 / 101.63 ±4.56 / 110.71 ms │ no change │
│ QQuery 25 │ 138.68 / 140.54 ±1.54 / 143.24 ms │ 136.88 / 138.79 ±1.86 / 142.23 ms │ no change │
│ QQuery 26 │ 101.75 / 103.05 ±1.12 / 104.50 ms │ 97.67 / 100.89 ±1.78 / 103.02 ms │ no change │
│ QQuery 27 │ 844.20 / 852.16 ±8.54 / 868.32 ms │ 845.17 / 851.18 ±4.25 / 857.71 ms │ no change │
│ QQuery 28 │ 3266.85 / 3289.03 ±13.98 / 3309.63 ms │ 3246.53 / 3272.99 ±19.20 / 3299.25 ms │ no change │
│ QQuery 29 │ 49.29 / 52.68 ±3.09 / 57.75 ms │ 49.43 / 54.29 ±4.69 / 63.02 ms │ no change │
│ QQuery 30 │ 351.23 / 357.94 ±4.16 / 364.24 ms │ 340.81 / 359.69 ±16.05 / 379.11 ms │ no change │
│ QQuery 31 │ 358.00 / 370.15 ±10.94 / 389.07 ms │ 353.37 / 372.21 ±12.22 / 387.82 ms │ no change │
│ QQuery 32 │ 1007.11 / 1020.76 ±9.20 / 1034.80 ms │ 1063.93 / 1214.41 ±116.39 / 1417.93 ms │ 1.19x slower │
│ QQuery 33 │ 1419.81 / 1427.58 ±8.07 / 1442.16 ms │ 1468.65 / 1526.34 ±32.55 / 1558.99 ms │ 1.07x slower │
│ QQuery 34 │ 1426.62 / 1452.16 ±23.15 / 1486.97 ms │ 1486.88 / 1510.79 ±24.66 / 1557.29 ms │ no change │
│ QQuery 35 │ 376.15 / 380.61 ±3.95 / 387.27 ms │ 310.99 / 335.61 ±14.64 / 353.51 ms │ +1.13x faster │
│ QQuery 36 │ 108.62 / 117.13 ±5.42 / 122.98 ms │ 111.79 / 115.18 ±2.36 / 118.63 ms │ no change │
│ QQuery 37 │ 46.21 / 49.36 ±1.68 / 51.11 ms │ 46.99 / 50.10 ±1.81 / 51.75 ms │ no change │
│ QQuery 38 │ 73.49 / 75.34 ±1.20 / 76.82 ms │ 75.47 / 77.66 ±1.12 / 78.50 ms │ no change │
│ QQuery 39 │ 199.43 / 212.68 ±6.84 / 217.58 ms │ 197.48 / 201.68 ±3.69 / 207.17 ms │ +1.05x faster │
│ QQuery 40 │ 21.62 / 24.89 ±2.31 / 28.44 ms │ 23.82 / 25.12 ±0.86 / 26.51 ms │ no change │
│ QQuery 41 │ 19.61 / 21.23 ±1.75 / 23.40 ms │ 19.08 / 21.26 ±1.49 / 23.38 ms │ no change │
│ QQuery 42 │ 18.26 / 19.67 ±1.02 / 21.41 ms │ 18.84 / 19.25 ±0.45 / 19.87 ms │ no change │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22115.05ms │
│ Total Time (early-group-emission) │ 22146.82ms │
│ Average Time (HEAD) │ 514.30ms │
│ Average Time (early-group-emission) │ 515.04ms │
│ Queries Faster │ 4 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 34 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 111.6s |
| Peak memory | 37.9 GiB |
| Avg memory | 32.0 GiB |
| CPU user | 1060.8s |
| CPU sys | 77.3s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 111.6s |
| Peak memory | 35.9 GiB |
| Avg memory | 30.0 GiB |
| CPU user | 1022.5s |
| CPU sys | 79.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4239003495
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238986171-1182-6bdkj 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241564976
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.54 / 7.02 ±0.79 / 8.57 ms │ 6.62 / 7.04 ±0.76 / 8.56 ms │ no change │
│ QQuery 2 │ 144.88 / 146.53 ±1.42 / 148.81 ms │ 144.40 / 145.87 ±0.75 / 146.46 ms │ no change │
│ QQuery 3 │ 113.07 / 113.76 ±0.47 / 114.51 ms │ 112.71 / 113.20 ±0.44 / 113.84 ms │ no change │
│ QQuery 4 │ 1333.19 / 1360.60 ±16.98 / 1386.39 ms │ 1322.81 / 1337.75 ±11.11 / 1352.53 ms │ no change │
│ QQuery 5 │ 172.22 / 174.08 ±1.69 / 176.23 ms │ 171.88 / 173.08 ±0.83 / 174.21 ms │ no change │
│ QQuery 6 │ 844.70 / 881.66 ±26.21 / 909.99 ms │ 813.90 / 830.45 ±10.33 / 840.46 ms │ +1.06x faster │
│ QQuery 7 │ 341.37 / 347.40 ±6.51 / 359.95 ms │ 338.15 / 341.30 ±1.73 / 342.85 ms │ no change │
│ QQuery 8 │ 114.34 / 116.38 ±1.44 / 117.66 ms │ 117.09 / 117.68 ±0.43 / 118.23 ms │ no change │
│ QQuery 9 │ 101.92 / 107.41 ±3.89 / 113.91 ms │ 105.97 / 110.05 ±4.18 / 116.01 ms │ no change │
│ QQuery 10 │ 107.75 / 108.13 ±0.49 / 109.02 ms │ 105.42 / 106.41 ±0.87 / 107.78 ms │ no change │
│ QQuery 11 │ 929.54 / 943.25 ±10.64 / 958.60 ms │ 924.24 / 942.77 ±19.56 / 971.69 ms │ no change │
│ QQuery 12 │ 44.39 / 46.63 ±1.85 / 49.81 ms │ 45.25 / 46.47 ±0.79 / 47.25 ms │ no change │
│ QQuery 13 │ 398.16 / 402.72 ±3.69 / 408.52 ms │ 394.63 / 399.69 ±3.85 / 406.48 ms │ no change │
│ QQuery 14 │ 1006.37 / 1013.45 ±5.08 / 1020.29 ms │ 988.78 / 998.33 ±8.49 / 1014.10 ms │ no change │
│ QQuery 15 │ 15.36 / 16.21 ±0.56 / 17.12 ms │ 15.46 / 16.52 ±1.13 / 18.65 ms │ no change │
│ QQuery 16 │ 7.07 / 7.63 ±0.54 / 8.57 ms │ 7.13 / 7.97 ±0.82 / 9.27 ms │ no change │
│ QQuery 17 │ 224.38 / 226.91 ±1.89 / 229.59 ms │ 226.31 / 228.31 ±1.51 / 230.31 ms │ no change │
│ QQuery 18 │ 126.71 / 128.27 ±1.08 / 129.56 ms │ 127.19 / 127.93 ±0.48 / 128.65 ms │ no change │
│ QQuery 19 │ 154.67 / 156.83 ±1.79 / 159.18 ms │ 155.39 / 157.36 ±2.25 / 161.62 ms │ no change │
│ QQuery 20 │ 14.27 / 14.72 ±0.40 / 15.34 ms │ 13.98 / 14.35 ±0.37 / 14.88 ms │ no change │
│ QQuery 21 │ 19.75 / 20.08 ±0.18 / 20.27 ms │ 19.35 / 19.84 ±0.38 / 20.32 ms │ no change │
│ QQuery 22 │ 481.49 / 484.45 ±3.57 / 491.02 ms │ 1149.50 / 1155.26 ±6.50 / 1165.43 ms │ 2.38x slower │
│ QQuery 23 │ 863.16 / 880.97 ±13.84 / 900.47 ms │ 945.98 / 961.52 ±8.19 / 968.39 ms │ 1.09x slower │
│ QQuery 24 │ 377.66 / 379.55 ±1.71 / 382.22 ms │ 377.87 / 381.45 ±2.86 / 385.93 ms │ no change │
│ QQuery 25 │ 338.11 / 338.79 ±0.59 / 339.77 ms │ 335.22 / 339.91 ±3.19 / 345.08 ms │ no change │
│ QQuery 26 │ 81.31 / 82.51 ±1.25 / 84.71 ms │ 80.19 / 82.28 ±1.46 / 83.91 ms │ no change │
│ QQuery 27 │ 6.68 / 6.87 ±0.17 / 7.15 ms │ 6.78 / 7.00 ±0.18 / 7.21 ms │ no change │
│ QQuery 28 │ 147.69 / 149.46 ±1.59 / 152.29 ms │ 147.80 / 149.16 ±1.69 / 152.16 ms │ no change │
│ QQuery 29 │ 278.28 / 279.12 ±0.79 / 280.47 ms │ 278.51 / 282.53 ±2.91 / 286.58 ms │ no change │
│ QQuery 30 │ 42.09 / 44.78 ±2.43 / 48.21 ms │ 44.46 / 46.24 ±1.69 / 49.41 ms │ no change │
│ QQuery 31 │ 168.39 / 170.87 ±1.62 / 173.31 ms │ 170.95 / 171.94 ±0.90 / 173.33 ms │ no change │
│ QQuery 32 │ 56.98 / 57.70 ±0.70 / 58.96 ms │ 57.48 / 58.30 ±0.92 / 59.63 ms │ no change │
│ QQuery 33 │ 138.86 / 140.67 ±1.54 / 142.64 ms │ 140.28 / 141.65 ±1.07 / 143.57 ms │ no change │
│ QQuery 34 │ 6.81 / 7.23 ±0.51 / 8.09 ms │ 6.74 / 7.05 ±0.26 / 7.53 ms │ no change │
│ QQuery 35 │ 108.21 / 109.92 ±1.52 / 112.32 ms │ 105.18 / 107.67 ±1.88 / 110.25 ms │ no change │
│ QQuery 36 │ 6.48 / 7.20 ±0.68 / 8.46 ms │ 6.38 / 6.66 ±0.25 / 7.14 ms │ +1.08x faster │
│ QQuery 37 │ 8.20 / 8.69 ±0.27 / 8.99 ms │ 8.68 / 9.20 ±0.51 / 9.84 ms │ 1.06x slower │
│ QQuery 38 │ 82.79 / 86.00 ±3.55 / 92.76 ms │ 84.98 / 87.29 ±1.65 / 89.57 ms │ no change │
│ QQuery 39 │ 124.23 / 125.61 ±1.36 / 128.17 ms │ 127.81 / 129.49 ±1.48 / 131.73 ms │ no change │
│ QQuery 40 │ 108.99 / 113.83 ±6.14 / 125.95 ms │ 113.37 / 116.32 ±2.56 / 120.70 ms │ no change │
│ QQuery 41 │ 14.39 / 15.16 ±0.76 / 16.53 ms │ 14.26 / 14.69 ±0.50 / 15.65 ms │ no change │
│ QQuery 42 │ 108.74 / 109.66 ±0.81 / 110.90 ms │ 108.90 / 109.59 ±0.58 / 110.57 ms │ no change │
│ QQuery 43 │ 5.91 / 6.05 ±0.12 / 6.28 ms │ 6.04 / 6.27 ±0.28 / 6.71 ms │ no change │
│ QQuery 44 │ 11.40 / 11.58 ±0.16 / 11.85 ms │ 11.37 / 11.99 ±0.82 / 13.59 ms │ no change │
│ QQuery 45 │ 51.38 / 52.04 ±0.39 / 52.53 ms │ 49.76 / 51.47 ±1.03 / 52.93 ms │ no change │
│ QQuery 46 │ 8.33 / 8.69 ±0.30 / 9.04 ms │ 8.45 / 8.58 ±0.18 / 8.93 ms │ no change │
│ QQuery 47 │ 712.10 / 713.32 ±1.01 / 715.09 ms │ 703.49 / 730.37 ±42.44 / 814.98 ms │ no change │
│ QQuery 48 │ 285.62 / 290.33 ±2.99 / 294.36 ms │ 283.42 / 290.60 ±3.84 / 294.95 ms │ no change │
│ QQuery 49 │ 248.96 / 251.86 ±2.21 / 254.94 ms │ 249.32 / 252.77 ±2.40 / 255.93 ms │ no change │
│ QQuery 50 │ 215.38 / 222.13 ±3.57 / 225.49 ms │ 220.02 / 223.66 ±2.82 / 228.04 ms │ no change │
│ QQuery 51 │ 180.98 / 181.95 ±0.87 / 183.38 ms │ 199.16 / 200.77 ±1.22 / 202.38 ms │ 1.10x slower │
│ QQuery 52 │ 106.77 / 107.50 ±0.56 / 108.08 ms │ 106.79 / 108.36 ±1.01 / 109.72 ms │ no change │
│ QQuery 53 │ 101.51 / 102.09 ±0.41 / 102.63 ms │ 102.03 / 103.16 ±0.94 / 104.59 ms │ no change │
│ QQuery 54 │ 144.96 / 147.62 ±1.65 / 149.36 ms │ 144.86 / 146.51 ±1.00 / 147.70 ms │ no change │
│ QQuery 55 │ 107.64 / 108.46 ±0.79 / 109.53 ms │ 106.42 / 107.54 ±1.14 / 109.56 ms │ no change │
│ QQuery 56 │ 142.14 / 144.17 ±1.91 / 146.67 ms │ 139.55 / 140.41 ±0.68 / 141.38 ms │ no change │
│ QQuery 57 │ 173.45 / 175.24 ±2.34 / 179.76 ms │ 173.06 / 175.01 ±1.95 / 178.66 ms │ no change │
│ QQuery 58 │ 300.66 / 309.39 ±6.96 / 318.25 ms │ 283.01 / 292.50 ±5.03 / 297.63 ms │ +1.06x faster │
│ QQuery 59 │ 197.75 / 201.20 ±5.66 / 212.44 ms │ 199.59 / 201.30 ±1.07 / 202.92 ms │ no change │
│ QQuery 60 │ 143.26 / 144.28 ±0.57 / 144.87 ms │ 141.72 / 142.94 ±0.88 / 144.34 ms │ no change │
│ QQuery 61 │ 13.05 / 13.56 ±0.48 / 14.41 ms │ 12.89 / 13.49 ±0.84 / 15.11 ms │ no change │
│ QQuery 62 │ 899.51 / 937.38 ±22.29 / 961.79 ms │ 931.14 / 969.93 ±38.50 / 1031.35 ms │ no change │
│ QQuery 63 │ 101.94 / 105.29 ±3.50 / 111.68 ms │ 103.24 / 106.25 ±2.51 / 109.74 ms │ no change │
│ QQuery 64 │ 680.10 / 683.96 ±2.94 / 688.18 ms │ 680.19 / 686.90 ±7.86 / 702.09 ms │ no change │
│ QQuery 65 │ 249.29 / 252.80 ±3.54 / 258.80 ms │ 249.95 / 254.08 ±3.24 / 259.71 ms │ no change │
│ QQuery 66 │ 246.51 / 262.54 ±15.96 / 289.42 ms │ 239.45 / 256.46 ±15.59 / 278.29 ms │ no change │
│ QQuery 67 │ 303.50 / 311.19 ±4.35 / 315.92 ms │ 299.44 / 305.43 ±4.08 / 311.35 ms │ no change │
│ QQuery 68 │ 9.55 / 10.25 ±0.48 / 10.81 ms │ 8.40 / 9.63 ±0.92 / 11.26 ms │ +1.06x faster │
│ QQuery 69 │ 102.89 / 103.57 ±0.52 / 104.25 ms │ 101.57 / 103.49 ±1.32 / 105.72 ms │ no change │
│ QQuery 70 │ 346.39 / 365.11 ±11.89 / 377.74 ms │ 328.31 / 343.14 ±9.65 / 353.93 ms │ +1.06x faster │
│ QQuery 71 │ 133.87 / 138.46 ±7.07 / 152.53 ms │ 133.52 / 137.93 ±4.80 / 147.01 ms │ no change │
│ QQuery 72 │ 598.50 / 615.60 ±11.63 / 631.09 ms │ 609.08 / 621.55 ±7.92 / 631.48 ms │ no change │
│ QQuery 73 │ 7.35 / 8.42 ±0.70 / 9.47 ms │ 6.78 / 7.24 ±0.36 / 7.81 ms │ +1.16x faster │
│ QQuery 74 │ 579.35 / 586.06 ±5.67 / 596.28 ms │ 578.38 / 585.08 ±5.53 / 591.09 ms │ no change │
│ QQuery 75 │ 276.29 / 277.77 ±1.59 / 280.13 ms │ 277.46 / 281.68 ±2.89 / 285.49 ms │ no change │
│ QQuery 76 │ 130.69 / 133.28 ±1.46 / 134.70 ms │ 132.17 / 134.98 ±1.99 / 137.93 ms │ no change │
│ QQuery 77 │ 185.47 / 188.61 ±1.70 / 190.28 ms │ 189.29 / 191.10 ±1.20 / 193.03 ms │ no change │
│ QQuery 78 │ 337.29 / 340.64 ±2.00 / 342.72 ms │ 341.48 / 345.10 ±2.65 / 349.44 ms │ no change │
│ QQuery 79 │ 230.24 / 233.19 ±3.35 / 239.49 ms │ 230.66 / 234.71 ±4.31 / 240.73 ms │ no change │
│ QQuery 80 │ 318.96 / 321.34 ±1.95 / 324.92 ms │ 319.57 / 322.11 ±1.84 / 325.14 ms │ no change │
│ QQuery 81 │ 26.25 / 26.97 ±0.74 / 28.00 ms │ 26.43 / 27.57 ±1.22 / 29.53 ms │ no change │
│ QQuery 82 │ 195.74 / 197.99 ±2.06 / 201.74 ms │ 198.71 / 201.79 ±2.69 / 206.59 ms │ no change │
│ QQuery 83 │ 38.44 / 39.11 ±0.74 / 40.43 ms │ 38.89 / 39.39 ±0.47 / 40.21 ms │ no change │
│ QQuery 84 │ 48.36 / 48.98 ±0.78 / 50.52 ms │ 48.61 / 50.36 ±1.87 / 53.81 ms │ no change │
│ QQuery 85 │ 147.49 / 149.13 ±1.24 / 150.64 ms │ 147.07 / 148.36 ±0.92 / 149.88 ms │ no change │
│ QQuery 86 │ 38.88 / 39.47 ±0.44 / 40.16 ms │ 38.99 / 40.57 ±1.34 / 42.17 ms │ no change │
│ QQuery 87 │ 84.53 / 88.46 ±3.52 / 94.74 ms │ 83.49 / 86.46 ±3.17 / 92.22 ms │ no change │
│ QQuery 88 │ 99.54 / 100.97 ±1.43 / 103.24 ms │ 100.32 / 101.11 ±0.75 / 102.43 ms │ no change │
│ QQuery 89 │ 117.73 / 118.98 ±1.06 / 120.21 ms │ 119.02 / 120.65 ±1.08 / 122.28 ms │ no change │
│ QQuery 90 │ 23.16 / 23.58 ±0.41 / 24.10 ms │ 23.73 / 24.22 ±0.31 / 24.70 ms │ no change │
│ QQuery 91 │ 61.94 / 63.64 ±1.25 / 65.26 ms │ 63.79 / 64.34 ±0.57 / 65.43 ms │ no change │
│ QQuery 92 │ 57.22 / 58.08 ±0.46 / 58.43 ms │ 57.25 / 57.61 ±0.43 / 58.44 ms │ no change │
│ QQuery 93 │ 184.78 / 187.14 ±1.31 / 188.66 ms │ 184.79 / 186.23 ±0.78 / 187.19 ms │ no change │
│ QQuery 94 │ 61.27 / 62.12 ±0.52 / 62.63 ms │ 60.60 / 62.09 ±0.86 / 62.92 ms │ no change │
│ QQuery 95 │ 126.67 / 128.13 ±0.84 / 128.95 ms │ 128.28 / 129.60 ±0.93 / 131.09 ms │ no change │
│ QQuery 96 │ 69.80 / 73.21 ±1.74 / 74.45 ms │ 72.54 / 74.07 ±0.82 / 74.97 ms │ no change │
│ QQuery 97 │ 123.13 / 124.79 ±1.33 / 126.86 ms │ 125.27 / 127.42 ±1.43 / 129.35 ms │ no change │
│ QQuery 98 │ 154.24 / 155.91 ±0.88 / 156.77 ms │ 152.99 / 155.99 ±2.12 / 159.10 ms │ no change │
│ QQuery 99 │ 10750.27 / 10792.13 ±28.88 / 10830.37 ms │ 10849.30 / 10993.68 ±136.01 / 11225.25 ms │ no change │
└───────────┴──────────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31484.36ms │
│ Total Time (early-group-emission) │ 32403.55ms │
│ Average Time (HEAD) │ 318.02ms │
│ Average Time (early-group-emission) │ 327.31ms │
│ Queries Faster │ 6 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 89 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 157.8s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 259.4s |
| CPU sys | 17.5s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 162.3s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 269.1s |
| CPU sys | 17.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238496079
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238481303-1170-g4fgx 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2006bca0b43a2509f533c2c056ac77aab89b86d3) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2006bca0b43a2509f533c2c056ac77aab89b86d3) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238481303
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238690651
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238673913-1174-2l7wl 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (9619b00c04d25bf10ef24bd090359784ec828b00) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..9619b00c04d25bf10ef24bd090359784ec828b00) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238999361
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238986171)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238986171-1181-9f5m7 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..ecaea3d0ece6a9c14e6ed5b3cabb5fe31873dba4) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241565908
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241504219)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.75 / 7.28 ±0.79 / 8.85 ms │ 6.74 / 7.17 ±0.78 / 8.72 ms │ no change │
│ QQuery 2 │ 143.82 / 145.22 ±0.93 / 146.04 ms │ 145.41 / 146.43 ±0.76 / 147.53 ms │ no change │
│ QQuery 3 │ 113.66 / 114.10 ±0.56 / 115.19 ms │ 114.10 / 114.46 ±0.42 / 115.28 ms │ no change │
│ QQuery 4 │ 1380.81 / 1404.92 ±17.93 / 1423.41 ms │ 1366.23 / 1389.59 ±18.52 / 1415.07 ms │ no change │
│ QQuery 5 │ 171.74 / 173.98 ±1.78 / 177.21 ms │ 173.08 / 174.80 ±1.21 / 175.96 ms │ no change │
│ QQuery 6 │ 854.85 / 889.49 ±24.56 / 913.67 ms │ 830.60 / 858.02 ±29.39 / 914.98 ms │ no change │
│ QQuery 7 │ 341.82 / 346.41 ±3.15 / 351.60 ms │ 341.37 / 343.87 ±1.82 / 346.06 ms │ no change │
│ QQuery 8 │ 116.89 / 118.48 ±1.07 / 120.20 ms │ 115.87 / 116.48 ±0.51 / 117.26 ms │ no change │
│ QQuery 9 │ 101.37 / 105.46 ±2.71 / 108.39 ms │ 104.05 / 107.13 ±1.69 / 108.88 ms │ no change │
│ QQuery 10 │ 108.60 / 110.48 ±0.98 / 111.29 ms │ 105.37 / 107.42 ±1.13 / 108.79 ms │ no change │
│ QQuery 11 │ 1034.26 / 1045.57 ±8.38 / 1056.55 ms │ 967.26 / 993.10 ±16.49 / 1013.30 ms │ +1.05x faster │
│ QQuery 12 │ 46.79 / 47.88 ±0.81 / 48.84 ms │ 45.38 / 46.43 ±1.01 / 47.80 ms │ no change │
│ QQuery 13 │ 409.41 / 412.23 ±1.89 / 414.88 ms │ 401.87 / 405.04 ±2.76 / 408.55 ms │ no change │
│ QQuery 14 │ 1005.97 / 1012.89 ±4.36 / 1019.51 ms │ 1000.30 / 1013.65 ±8.50 / 1023.04 ms │ no change │
│ QQuery 15 │ 15.75 / 17.75 ±2.05 / 21.28 ms │ 15.89 / 16.97 ±0.92 / 18.55 ms │ no change │
│ QQuery 16 │ 7.32 / 8.05 ±0.92 / 9.85 ms │ 7.29 / 8.24 ±0.82 / 9.46 ms │ no change │
│ QQuery 17 │ 229.01 / 232.02 ±1.89 / 234.26 ms │ 230.39 / 231.96 ±1.10 / 233.68 ms │ no change │
│ QQuery 18 │ 128.92 / 130.32 ±1.14 / 131.93 ms │ 127.59 / 128.43 ±0.55 / 129.29 ms │ no change │
│ QQuery 19 │ 155.11 / 156.80 ±1.15 / 158.31 ms │ 156.45 / 157.42 ±0.74 / 158.37 ms │ no change │
│ QQuery 20 │ 14.02 / 14.44 ±0.31 / 14.79 ms │ 14.35 / 14.70 ±0.30 / 15.15 ms │ no change │
│ QQuery 21 │ 19.62 / 20.43 ±0.50 / 20.95 ms │ 19.91 / 20.43 ±0.36 / 20.88 ms │ no change │
│ QQuery 22 │ 488.97 / 496.81 ±4.71 / 503.40 ms │ 1161.73 / 1169.67 ±4.35 / 1174.13 ms │ 2.35x slower │
│ QQuery 23 │ 889.05 / 900.51 ±7.51 / 908.17 ms │ 982.91 / 989.43 ±4.40 / 995.99 ms │ 1.10x slower │
│ QQuery 24 │ 382.93 / 386.14 ±3.79 / 391.75 ms │ 381.18 / 386.03 ±3.40 / 390.31 ms │ no change │
│ QQuery 25 │ 340.74 / 342.62 ±1.55 / 345.20 ms │ 339.85 / 344.72 ±3.64 / 349.30 ms │ no change │
│ QQuery 26 │ 83.19 / 84.21 ±0.85 / 85.69 ms │ 80.59 / 82.47 ±2.18 / 86.68 ms │ no change │
│ QQuery 27 │ 7.34 / 7.54 ±0.17 / 7.82 ms │ 6.83 / 7.67 ±1.02 / 9.53 ms │ no change │
│ QQuery 28 │ 148.79 / 149.59 ±0.91 / 151.33 ms │ 148.42 / 151.01 ±2.15 / 154.90 ms │ no change │
│ QQuery 29 │ 280.12 / 282.42 ±1.73 / 284.40 ms │ 283.43 / 285.39 ±1.34 / 287.34 ms │ no change │
│ QQuery 30 │ 44.70 / 46.00 ±1.22 / 47.56 ms │ 43.03 / 45.14 ±1.13 / 46.17 ms │ no change │
│ QQuery 31 │ 171.21 / 172.21 ±0.84 / 173.71 ms │ 168.83 / 171.09 ±1.65 / 173.97 ms │ no change │
│ QQuery 32 │ 56.21 / 65.38 ±16.32 / 98.00 ms │ 56.86 / 66.51 ±16.94 / 100.36 ms │ no change │
│ QQuery 33 │ 139.78 / 141.49 ±1.53 / 143.84 ms │ 140.23 / 141.78 ±1.26 / 143.53 ms │ no change │
│ QQuery 34 │ 7.19 / 7.33 ±0.14 / 7.54 ms │ 7.02 / 7.46 ±0.31 / 7.84 ms │ no change │
│ QQuery 35 │ 109.93 / 111.43 ±1.08 / 112.57 ms │ 106.19 / 109.09 ±1.56 / 110.78 ms │ no change │
│ QQuery 36 │ 6.80 / 7.13 ±0.24 / 7.45 ms │ 6.75 / 7.03 ±0.16 / 7.22 ms │ no change │
│ QQuery 37 │ 8.47 / 9.21 ±0.74 / 10.58 ms │ 8.44 / 8.81 ±0.33 / 9.38 ms │ no change │
│ QQuery 38 │ 85.82 / 87.97 ±2.43 / 92.60 ms │ 85.41 / 89.02 ±4.40 / 97.25 ms │ no change │
│ QQuery 39 │ 129.13 / 132.66 ±1.99 / 135.35 ms │ 126.70 / 130.86 ±2.66 / 134.28 ms │ no change │
│ QQuery 40 │ 109.92 / 116.89 ±6.55 / 128.51 ms │ 111.27 / 118.06 ±7.50 / 132.51 ms │ no change │
│ QQuery 41 │ 14.74 / 16.37 ±1.38 / 18.06 ms │ 14.66 / 15.40 ±0.45 / 15.89 ms │ +1.06x faster │
│ QQuery 42 │ 107.01 / 108.93 ±1.52 / 111.03 ms │ 107.33 / 109.32 ±1.09 / 110.36 ms │ no change │
│ QQuery 43 │ 6.08 / 6.22 ±0.12 / 6.41 ms │ 5.98 / 6.09 ±0.11 / 6.31 ms │ no change │
│ QQuery 44 │ 12.02 / 12.37 ±0.40 / 13.15 ms │ 12.02 / 12.21 ±0.18 / 12.55 ms │ no change │
│ QQuery 45 │ 51.54 / 52.04 ±0.58 / 53.07 ms │ 50.48 / 51.44 ±0.77 / 52.36 ms │ no change │
│ QQuery 46 │ 8.99 / 9.17 ±0.10 / 9.29 ms │ 8.54 / 9.21 ±0.85 / 10.86 ms │ no change │
│ QQuery 47 │ 747.02 / 758.29 ±6.32 / 763.92 ms │ 713.89 / 717.12 ±2.85 / 721.37 ms │ +1.06x faster │
│ QQuery 48 │ 291.75 / 296.57 ±3.31 / 301.09 ms │ 289.77 / 298.33 ±4.41 / 301.75 ms │ no change │
│ QQuery 49 │ 250.33 / 254.16 ±2.38 / 257.14 ms │ 249.55 / 252.93 ±2.03 / 254.95 ms │ no change │
│ QQuery 50 │ 222.81 / 226.68 ±3.97 / 233.87 ms │ 226.49 / 231.44 ±3.30 / 236.61 ms │ no change │
│ QQuery 51 │ 178.76 / 181.87 ±1.92 / 184.69 ms │ 198.04 / 199.44 ±1.98 / 203.31 ms │ 1.10x slower │
│ QQuery 52 │ 106.85 / 108.89 ±1.55 / 110.72 ms │ 107.03 / 109.19 ±1.48 / 110.93 ms │ no change │
│ QQuery 53 │ 101.93 / 103.11 ±1.18 / 105.35 ms │ 103.80 / 105.60 ±1.36 / 107.23 ms │ no change │
│ QQuery 54 │ 145.73 / 148.45 ±2.28 / 152.23 ms │ 147.07 / 148.46 ±1.30 / 150.65 ms │ no change │
│ QQuery 55 │ 107.18 / 108.54 ±1.68 / 111.56 ms │ 107.25 / 109.35 ±2.09 / 113.25 ms │ no change │
│ QQuery 56 │ 141.14 / 142.10 ±0.69 / 143.14 ms │ 142.02 / 142.85 ±0.68 / 143.84 ms │ no change │
│ QQuery 57 │ 173.44 / 176.42 ±2.08 / 179.02 ms │ 173.42 / 177.05 ±1.88 / 178.80 ms │ no change │
│ QQuery 58 │ 286.51 / 297.58 ±9.12 / 308.33 ms │ 291.40 / 302.72 ±9.03 / 315.48 ms │ no change │
│ QQuery 59 │ 201.21 / 202.18 ±0.75 / 203.35 ms │ 200.10 / 202.08 ±2.42 / 206.82 ms │ no change │
│ QQuery 60 │ 143.92 / 145.12 ±1.20 / 147.05 ms │ 143.76 / 144.61 ±1.03 / 146.61 ms │ no change │
│ QQuery 61 │ 13.39 / 13.74 ±0.21 / 14.04 ms │ 13.17 / 13.51 ±0.18 / 13.71 ms │ no change │
│ QQuery 62 │ 913.19 / 958.55 ±41.58 / 1033.63 ms │ 910.26 / 928.37 ±15.83 / 957.90 ms │ no change │
│ QQuery 63 │ 106.47 / 107.31 ±0.69 / 108.20 ms │ 105.34 / 108.83 ±1.90 / 111.09 ms │ no change │
│ QQuery 64 │ 691.25 / 693.74 ±2.71 / 698.51 ms │ 694.81 / 699.89 ±3.30 / 704.90 ms │ no change │
│ QQuery 65 │ 253.69 / 257.41 ±3.75 / 264.46 ms │ 255.03 / 260.54 ±3.43 / 265.50 ms │ no change │
│ QQuery 66 │ 241.19 / 262.00 ±14.55 / 283.14 ms │ 254.39 / 260.55 ±4.18 / 267.48 ms │ no change │
│ QQuery 67 │ 314.14 / 323.42 ±7.50 / 334.52 ms │ 302.01 / 309.02 ±4.25 / 314.84 ms │ no change │
│ QQuery 68 │ 8.45 / 9.36 ±0.51 / 9.79 ms │ 9.46 / 10.93 ±0.87 / 12.04 ms │ 1.17x slower │
│ QQuery 69 │ 102.69 / 104.64 ±1.41 / 106.55 ms │ 101.19 / 103.02 ±1.24 / 104.44 ms │ no change │
│ QQuery 70 │ 329.53 / 352.10 ±15.26 / 376.28 ms │ 332.46 / 346.90 ±12.47 / 369.95 ms │ no change │
│ QQuery 71 │ 135.58 / 136.97 ±0.82 / 138.06 ms │ 135.31 / 135.78 ±0.43 / 136.44 ms │ no change │
│ QQuery 72 │ 608.53 / 619.84 ±10.67 / 638.98 ms │ 617.83 / 626.86 ±5.65 / 632.97 ms │ no change │
│ QQuery 73 │ 6.58 / 7.88 ±1.40 / 10.27 ms │ 7.47 / 8.99 ±1.14 / 10.86 ms │ 1.14x slower │
│ QQuery 74 │ 601.20 / 615.41 ±8.00 / 622.35 ms │ 609.76 / 621.97 ±7.95 / 630.80 ms │ no change │
│ QQuery 75 │ 275.80 / 279.23 ±2.21 / 281.48 ms │ 279.83 / 281.31 ±1.16 / 282.55 ms │ no change │
│ QQuery 76 │ 131.44 / 133.68 ±1.39 / 135.55 ms │ 130.85 / 132.35 ±0.89 / 133.47 ms │ no change │
│ QQuery 77 │ 188.27 / 189.76 ±0.87 / 190.61 ms │ 188.36 / 189.82 ±1.34 / 191.65 ms │ no change │
│ QQuery 78 │ 343.78 / 347.93 ±3.49 / 353.79 ms │ 340.05 / 343.57 ±4.56 / 352.57 ms │ no change │
│ QQuery 79 │ 232.89 / 236.90 ±2.85 / 240.90 ms │ 233.46 / 237.83 ±3.66 / 244.57 ms │ no change │
│ QQuery 80 │ 320.07 / 322.90 ±2.26 / 326.85 ms │ 319.60 / 324.72 ±3.35 / 328.27 ms │ no change │
│ QQuery 81 │ 26.42 / 27.24 ±0.60 / 28.24 ms │ 26.65 / 27.61 ±1.53 / 30.65 ms │ no change │
│ QQuery 82 │ 198.71 / 201.15 ±2.33 / 205.15 ms │ 203.17 / 206.17 ±1.86 / 208.76 ms │ no change │
│ QQuery 83 │ 39.77 / 41.07 ±1.44 / 43.65 ms │ 40.39 / 40.99 ±0.49 / 41.76 ms │ no change │
│ QQuery 84 │ 48.97 / 49.56 ±0.58 / 50.37 ms │ 48.40 / 49.45 ±0.66 / 50.32 ms │ no change │
│ QQuery 85 │ 150.75 / 151.99 ±1.42 / 154.00 ms │ 148.81 / 150.69 ±1.86 / 154.02 ms │ no change │
│ QQuery 86 │ 39.56 / 40.37 ±0.79 / 41.71 ms │ 38.99 / 40.61 ±1.34 / 43.07 ms │ no change │
│ QQuery 87 │ 86.95 / 90.13 ±2.96 / 94.66 ms │ 85.23 / 88.83 ±3.91 / 96.11 ms │ no change │
│ QQuery 88 │ 100.15 / 101.54 ±1.34 / 103.85 ms │ 101.02 / 102.45 ±1.47 / 104.98 ms │ no change │
│ QQuery 89 │ 116.21 / 119.34 ±1.86 / 121.62 ms │ 121.33 / 121.92 ±0.69 / 123.14 ms │ no change │
│ QQuery 90 │ 23.37 / 24.08 ±0.61 / 25.01 ms │ 23.72 / 25.25 ±0.92 / 26.60 ms │ no change │
│ QQuery 91 │ 63.32 / 64.61 ±1.29 / 67.08 ms │ 61.39 / 65.15 ±2.06 / 66.89 ms │ no change │
│ QQuery 92 │ 58.50 / 59.07 ±0.37 / 59.65 ms │ 57.51 / 58.58 ±0.86 / 59.64 ms │ no change │
│ QQuery 93 │ 185.43 / 186.54 ±1.04 / 188.45 ms │ 186.99 / 188.67 ±0.95 / 189.94 ms │ no change │
│ QQuery 94 │ 60.96 / 62.45 ±1.29 / 64.56 ms │ 61.56 / 62.64 ±0.69 / 63.65 ms │ no change │
│ QQuery 95 │ 126.13 / 128.15 ±1.43 / 130.61 ms │ 127.45 / 129.58 ±1.51 / 131.61 ms │ no change │
│ QQuery 96 │ 69.64 / 73.70 ±2.17 / 75.37 ms │ 73.22 / 74.85 ±1.11 / 76.08 ms │ no change │
│ QQuery 97 │ 124.52 / 126.49 ±1.92 / 129.23 ms │ 122.90 / 127.15 ±2.38 / 129.36 ms │ no change │
│ QQuery 98 │ 155.51 / 156.92 ±1.00 / 158.00 ms │ 153.15 / 156.24 ±2.83 / 160.75 ms │ no change │
│ QQuery 99 │ 10799.55 / 10828.15 ±18.42 / 10851.98 ms │ 10832.61 / 10886.12 ±30.52 / 10922.25 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31922.13ms │
│ Total Time (early-group-emission) │ 32605.48ms │
│ Average Time (HEAD) │ 322.45ms │
│ Average Time (early-group-emission) │ 329.35ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 91 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 160.0s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 263.9s |
| CPU sys | 17.2s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 163.3s |
| Peak memory | 5.8 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 272.2s |
| CPU sys | 17.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238799937
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.20 / 4.54 ±6.52 / 17.57 ms │ 1.20 / 4.36 ±6.25 / 16.86 ms │ no change │
│ QQuery 1 │ 14.41 / 14.77 ±0.19 / 14.92 ms │ 14.59 / 14.83 ±0.38 / 15.58 ms │ no change │
│ QQuery 2 │ 44.85 / 45.06 ±0.16 / 45.34 ms │ 43.55 / 44.01 ±0.32 / 44.44 ms │ no change │
│ QQuery 3 │ 45.87 / 46.27 ±0.38 / 46.73 ms │ 41.39 / 44.22 ±2.25 / 46.46 ms │ no change │
│ QQuery 4 │ 282.03 / 288.23 ±4.29 / 294.83 ms │ 285.56 / 292.92 ±8.29 / 306.52 ms │ no change │
│ QQuery 5 │ 338.76 / 347.09 ±6.94 / 358.48 ms │ 338.59 / 350.59 ±12.66 / 374.58 ms │ no change │
│ QQuery 6 │ 4.97 / 6.22 ±1.23 / 8.55 ms │ 5.94 / 6.53 ±0.83 / 8.11 ms │ no change │
│ QQuery 7 │ 17.07 / 19.65 ±3.27 / 25.96 ms │ 16.58 / 18.75 ±1.39 / 20.68 ms │ no change │
│ QQuery 8 │ 409.62 / 422.78 ±13.09 / 442.52 ms │ 395.14 / 405.20 ±7.81 / 417.65 ms │ no change │
│ QQuery 9 │ 627.46 / 664.84 ±28.40 / 695.32 ms │ 446.46 / 458.79 ±12.83 / 480.37 ms │ +1.45x faster │
│ QQuery 10 │ 91.55 / 94.26 ±2.39 / 98.69 ms │ 89.84 / 92.47 ±2.33 / 96.09 ms │ no change │
│ QQuery 11 │ 106.48 / 108.57 ±1.62 / 110.94 ms │ 102.20 / 103.85 ±1.30 / 105.65 ms │ no change │
│ QQuery 12 │ 339.00 / 350.51 ±12.03 / 372.15 ms │ 335.02 / 347.83 ±11.88 / 367.05 ms │ no change │
│ QQuery 13 │ 450.51 / 464.73 ±8.91 / 478.47 ms │ 477.96 / 486.74 ±8.35 / 500.20 ms │ no change │
│ QQuery 14 │ 337.78 / 343.62 ±4.45 / 350.66 ms │ 326.24 / 344.42 ±17.18 / 377.02 ms │ no change │
│ QQuery 15 │ 341.65 / 357.69 ±12.21 / 377.87 ms │ 321.03 / 335.03 ±7.25 / 341.42 ms │ +1.07x faster │
│ QQuery 16 │ 709.57 / 738.56 ±19.63 / 760.88 ms │ 699.77 / 712.96 ±11.39 / 729.29 ms │ no change │
│ QQuery 17 │ 699.85 / 712.63 ±10.50 / 729.04 ms │ 681.86 / 703.13 ±19.91 / 731.55 ms │ no change │
│ QQuery 18 │ 1376.18 / 1453.30 ±52.13 / 1536.63 ms │ 1275.42 / 1320.16 ±41.68 / 1390.43 ms │ +1.10x faster │
│ QQuery 19 │ 34.77 / 52.30 ±31.98 / 116.12 ms │ 35.03 / 35.74 ±0.81 / 37.27 ms │ +1.46x faster │
│ QQuery 20 │ 712.97 / 738.56 ±26.36 / 782.78 ms │ 715.32 / 720.87 ±6.81 / 734.27 ms │ no change │
│ QQuery 21 │ 760.98 / 764.97 ±4.44 / 773.07 ms │ 758.39 / 763.48 ±5.40 / 770.58 ms │ no change │
│ QQuery 22 │ 1127.35 / 1131.36 ±3.93 / 1137.50 ms │ 1127.66 / 1129.59 ±2.06 / 1133.48 ms │ no change │
│ QQuery 23 │ 3069.85 / 3095.84 ±26.76 / 3146.26 ms │ 3048.98 / 3059.49 ±13.02 / 3084.99 ms │ no change │
│ QQuery 24 │ 100.07 / 103.47 ±3.55 / 110.12 ms │ 97.44 / 98.58 ±1.05 / 100.04 ms │ no change │
│ QQuery 25 │ 137.12 / 139.46 ±1.67 / 141.83 ms │ 138.89 / 141.31 ±2.37 / 145.09 ms │ no change │
│ QQuery 26 │ 96.95 / 99.51 ±2.47 / 103.82 ms │ 96.08 / 99.22 ±3.00 / 104.40 ms │ no change │
│ QQuery 27 │ 846.30 / 850.38 ±6.24 / 862.78 ms │ 849.33 / 855.05 ±3.97 / 861.37 ms │ no change │
│ QQuery 28 │ 3271.38 / 3297.44 ±17.31 / 3320.03 ms │ 3266.28 / 3308.97 ±62.77 / 3432.82 ms │ no change │
│ QQuery 29 │ 55.61 / 107.06 ±52.19 / 190.59 ms │ 49.57 / 54.27 ±3.69 / 59.99 ms │ +1.97x faster │
│ QQuery 30 │ 361.05 / 373.00 ±7.10 / 383.03 ms │ 351.61 / 369.32 ±14.94 / 392.06 ms │ no change │
│ QQuery 31 │ 364.51 / 400.45 ±22.64 / 433.03 ms │ 358.96 / 390.00 ±18.82 / 415.74 ms │ no change │
│ QQuery 32 │ 1049.37 / 1182.67 ±96.48 / 1311.13 ms │ 1017.76 / 1028.30 ±9.13 / 1041.56 ms │ +1.15x faster │
│ QQuery 33 │ 1425.38 / 1446.59 ±18.63 / 1478.05 ms │ 1486.44 / 1525.78 ±25.43 / 1566.48 ms │ 1.05x slower │
│ QQuery 34 │ 1451.73 / 1472.12 ±17.43 / 1502.35 ms │ 1489.02 / 1532.46 ±31.48 / 1576.61 ms │ no change │
│ QQuery 35 │ 376.67 / 378.24 ±1.52 / 380.63 ms │ 325.20 / 338.92 ±9.72 / 355.27 ms │ +1.12x faster │
│ QQuery 36 │ 111.21 / 119.70 ±4.80 / 124.84 ms │ 111.35 / 114.46 ±1.69 / 116.44 ms │ no change │
│ QQuery 37 │ 47.66 / 49.39 ±1.61 / 52.28 ms │ 47.94 / 48.56 ±0.72 / 49.79 ms │ no change │
│ QQuery 38 │ 75.49 / 77.79 ±1.95 / 81.19 ms │ 75.23 / 75.97 ±1.06 / 78.07 ms │ no change │
│ QQuery 39 │ 209.32 / 215.15 ±5.05 / 222.67 ms │ 202.73 / 206.62 ±3.28 / 211.62 ms │ no change │
│ QQuery 40 │ 21.84 / 25.45 ±2.64 / 29.91 ms │ 22.37 / 25.38 ±1.53 / 26.44 ms │ no change │
│ QQuery 41 │ 19.49 / 21.26 ±1.05 / 22.40 ms │ 20.61 / 20.74 ±0.13 / 20.97 ms │ no change │
│ QQuery 42 │ 19.45 / 19.72 ±0.42 / 20.56 ms │ 19.65 / 20.21 ±0.44 / 20.94 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22645.18ms │
│ Total Time (early-group-emission) │ 22050.10ms │
│ Average Time (HEAD) │ 526.63ms │
│ Average Time (early-group-emission) │ 512.79ms │
│ Queries Faster │ 7 │
│ Queries Slower │ 1 │
│ Queries with No Change │ 35 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 114.3s |
| Peak memory | 40.2 GiB |
| Avg memory | 29.0 GiB |
| CPU user | 1071.3s |
| CPU sys | 91.4s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 111.1s |
| Peak memory | 37.9 GiB |
| Avg memory | 28.1 GiB |
| CPU user | 1029.2s |
| CPU sys | 70.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #21595: feat: early group emission with emit+reset for partial aggregation
URL: https://github.com/apache/datafusion/pull/21595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: two-generation early emission for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238689969
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238673913)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238673913-1173-pf7n8 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (9619b00c04d25bf10ef24bd090359784ec828b00) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..9619b00c04d25bf10ef24bd090359784ec828b00) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238805366
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238803110-1175-2289q 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (01835ce8773a93097f91e5ace1e53db1b1c30fce) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..01835ce8773a93097f91e5ace1e53db1b1c30fce) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4239065012
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238944023)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.53 / 6.99 ±0.81 / 8.60 ms │ 6.65 / 7.06 ±0.77 / 8.61 ms │ no change │
│ QQuery 2 │ 145.36 / 146.12 ±0.54 / 147.00 ms │ 144.91 / 145.29 ±0.57 / 146.43 ms │ no change │
│ QQuery 3 │ 112.45 / 113.52 ±0.87 / 115.05 ms │ 112.62 / 113.54 ±0.77 / 114.60 ms │ no change │
│ QQuery 4 │ 1293.08 / 1321.40 ±19.10 / 1343.61 ms │ 1277.19 / 1324.56 ±29.65 / 1352.60 ms │ no change │
│ QQuery 5 │ 172.88 / 173.61 ±0.71 / 174.95 ms │ 170.38 / 172.08 ±1.23 / 173.79 ms │ no change │
│ QQuery 6 │ 839.26 / 853.88 ±7.99 / 863.68 ms │ 844.23 / 851.11 ±4.85 / 856.96 ms │ no change │
│ QQuery 7 │ 338.44 / 340.80 ±1.71 / 342.86 ms │ 335.54 / 342.27 ±4.30 / 347.60 ms │ no change │
│ QQuery 8 │ 114.96 / 116.65 ±0.97 / 117.79 ms │ 114.19 / 115.18 ±1.21 / 117.51 ms │ no change │
│ QQuery 9 │ 100.45 / 105.39 ±2.50 / 107.07 ms │ 101.36 / 105.49 ±4.67 / 114.10 ms │ no change │
│ QQuery 10 │ 105.69 / 107.58 ±1.24 / 109.07 ms │ 104.08 / 106.00 ±1.10 / 107.04 ms │ no change │
│ QQuery 11 │ 887.02 / 908.10 ±15.05 / 927.99 ms │ 886.54 / 903.43 ±9.68 / 915.28 ms │ no change │
│ QQuery 12 │ 43.68 / 45.81 ±1.28 / 47.33 ms │ 45.02 / 45.85 ±0.60 / 46.76 ms │ no change │
│ QQuery 13 │ 398.69 / 400.06 ±0.88 / 401.23 ms │ 394.97 / 401.34 ±3.83 / 406.26 ms │ no change │
│ QQuery 14 │ 994.18 / 1005.82 ±5.89 / 1009.53 ms │ 993.97 / 1007.68 ±10.41 / 1024.31 ms │ no change │
│ QQuery 15 │ 15.25 / 15.94 ±0.67 / 17.18 ms │ 15.35 / 16.80 ±1.54 / 19.60 ms │ 1.05x slower │
│ QQuery 16 │ 7.00 / 7.46 ±0.49 / 8.35 ms │ 7.15 / 7.74 ±0.74 / 9.17 ms │ no change │
│ QQuery 17 │ 227.17 / 228.45 ±0.94 / 229.61 ms │ 228.07 / 229.00 ±0.75 / 229.95 ms │ no change │
│ QQuery 18 │ 127.13 / 128.00 ±0.56 / 128.85 ms │ 126.38 / 127.93 ±1.16 / 129.48 ms │ no change │
│ QQuery 19 │ 155.72 / 157.17 ±0.93 / 158.51 ms │ 154.71 / 155.84 ±0.83 / 156.86 ms │ no change │
│ QQuery 20 │ 13.97 / 14.48 ±0.26 / 14.72 ms │ 13.57 / 13.73 ±0.11 / 13.86 ms │ +1.05x faster │
│ QQuery 21 │ 19.29 / 20.06 ±0.83 / 21.57 ms │ 18.60 / 19.36 ±0.51 / 19.90 ms │ no change │
│ QQuery 22 │ 482.58 / 484.21 ±1.88 / 487.36 ms │ 478.65 / 481.66 ±2.33 / 485.76 ms │ no change │
│ QQuery 23 │ 854.55 / 862.23 ±5.34 / 870.34 ms │ 863.16 / 870.08 ±5.68 / 876.37 ms │ no change │
│ QQuery 24 │ 377.21 / 379.08 ±1.64 / 381.36 ms │ 374.87 / 377.20 ±3.14 / 383.35 ms │ no change │
│ QQuery 25 │ 338.17 / 339.71 ±1.04 / 341.42 ms │ 335.43 / 339.34 ±2.70 / 342.40 ms │ no change │
│ QQuery 26 │ 79.47 / 82.38 ±1.59 / 84.21 ms │ 80.65 / 81.78 ±0.70 / 82.46 ms │ no change │
│ QQuery 27 │ 6.71 / 7.43 ±1.03 / 9.46 ms │ 6.82 / 7.17 ±0.38 / 7.85 ms │ no change │
│ QQuery 28 │ 148.23 / 149.24 ±0.84 / 150.32 ms │ 148.25 / 149.75 ±1.31 / 151.47 ms │ no change │
│ QQuery 29 │ 278.79 / 281.01 ±1.59 / 283.35 ms │ 276.61 / 279.03 ±1.69 / 281.05 ms │ no change │
│ QQuery 30 │ 44.00 / 45.15 ±0.80 / 46.37 ms │ 43.12 / 43.91 ±0.87 / 45.40 ms │ no change │
│ QQuery 31 │ 168.85 / 169.87 ±1.02 / 171.69 ms │ 168.95 / 171.17 ±1.67 / 173.12 ms │ no change │
│ QQuery 32 │ 56.19 / 56.90 ±0.59 / 57.99 ms │ 56.83 / 58.05 ±1.95 / 61.93 ms │ no change │
│ QQuery 33 │ 138.97 / 140.65 ±1.66 / 143.75 ms │ 140.15 / 140.89 ±0.60 / 141.57 ms │ no change │
│ QQuery 34 │ 6.92 / 7.11 ±0.22 / 7.51 ms │ 6.77 / 7.56 ±1.42 / 10.40 ms │ 1.06x slower │
│ QQuery 35 │ 108.08 / 108.83 ±0.59 / 109.85 ms │ 107.96 / 109.83 ±2.20 / 114.04 ms │ no change │
│ QQuery 36 │ 6.50 / 6.86 ±0.35 / 7.42 ms │ 6.46 / 7.24 ±1.00 / 9.18 ms │ 1.06x slower │
│ QQuery 37 │ 8.27 / 8.85 ±0.32 / 9.15 ms │ 8.74 / 8.99 ±0.28 / 9.51 ms │ no change │
│ QQuery 38 │ 84.45 / 88.48 ±3.08 / 92.21 ms │ 80.68 / 86.28 ±5.32 / 95.82 ms │ no change │
│ QQuery 39 │ 122.48 / 125.34 ±2.51 / 129.41 ms │ 123.22 / 125.88 ±1.85 / 128.66 ms │ no change │
│ QQuery 40 │ 108.95 / 113.76 ±5.66 / 124.04 ms │ 104.43 / 113.40 ±7.18 / 125.29 ms │ no change │
│ QQuery 41 │ 14.74 / 15.72 ±0.88 / 17.07 ms │ 14.08 / 15.00 ±0.90 / 16.23 ms │ no change │
│ QQuery 42 │ 106.54 / 108.67 ±2.11 / 112.39 ms │ 105.81 / 107.67 ±1.66 / 110.76 ms │ no change │
│ QQuery 43 │ 5.81 / 5.98 ±0.13 / 6.22 ms │ 5.88 / 6.12 ±0.20 / 6.46 ms │ no change │
│ QQuery 44 │ 11.18 / 12.01 ±0.66 / 13.21 ms │ 11.82 / 12.46 ±0.99 / 14.43 ms │ no change │
│ QQuery 45 │ 49.43 / 50.72 ±0.79 / 51.46 ms │ 50.58 / 51.04 ±0.28 / 51.33 ms │ no change │
│ QQuery 46 │ 8.41 / 8.70 ±0.21 / 8.99 ms │ 8.37 / 8.59 ±0.15 / 8.78 ms │ no change │
│ QQuery 47 │ 686.00 / 692.23 ±3.83 / 697.94 ms │ 620.71 / 623.78 ±2.35 / 627.96 ms │ +1.11x faster │
│ QQuery 48 │ 278.34 / 288.73 ±5.96 / 294.91 ms │ 291.01 / 292.96 ±1.85 / 295.55 ms │ no change │
│ QQuery 49 │ 250.74 / 252.74 ±1.29 / 254.45 ms │ 248.36 / 250.53 ±2.43 / 255.26 ms │ no change │
│ QQuery 50 │ 216.30 / 222.63 ±3.57 / 226.28 ms │ 213.89 / 222.22 ±4.70 / 228.39 ms │ no change │
│ QQuery 51 │ 182.31 / 183.99 ±1.08 / 185.09 ms │ 177.69 / 180.22 ±3.27 / 186.52 ms │ no change │
│ QQuery 52 │ 106.84 / 107.92 ±0.88 / 108.85 ms │ 107.02 / 108.90 ±1.60 / 111.44 ms │ no change │
│ QQuery 53 │ 101.93 / 102.60 ±0.63 / 103.76 ms │ 101.60 / 102.71 ±0.92 / 104.28 ms │ no change │
│ QQuery 54 │ 144.26 / 145.85 ±1.51 / 147.94 ms │ 144.36 / 146.26 ±1.54 / 148.62 ms │ no change │
│ QQuery 55 │ 107.36 / 109.15 ±1.38 / 110.76 ms │ 106.42 / 107.61 ±1.26 / 109.89 ms │ no change │
│ QQuery 56 │ 143.63 / 144.48 ±0.64 / 145.27 ms │ 139.59 / 142.34 ±1.76 / 144.96 ms │ no change │
│ QQuery 57 │ 174.62 / 176.07 ±1.70 / 178.95 ms │ 169.35 / 173.16 ±2.59 / 176.71 ms │ no change │
│ QQuery 58 │ 299.96 / 310.82 ±7.21 / 318.61 ms │ 288.21 / 307.02 ±13.84 / 330.47 ms │ no change │
│ QQuery 59 │ 199.79 / 202.12 ±1.81 / 205.18 ms │ 197.77 / 199.52 ±0.96 / 200.39 ms │ no change │
│ QQuery 60 │ 143.18 / 145.39 ±1.44 / 147.16 ms │ 141.49 / 143.41 ±1.12 / 144.36 ms │ no change │
│ QQuery 61 │ 13.02 / 13.32 ±0.28 / 13.69 ms │ 12.72 / 13.01 ±0.47 / 13.93 ms │ no change │
│ QQuery 62 │ 899.40 / 926.16 ±16.27 / 945.41 ms │ 890.12 / 934.39 ±29.57 / 975.01 ms │ no change │
│ QQuery 63 │ 104.24 / 107.09 ±2.47 / 111.27 ms │ 103.34 / 105.72 ±2.05 / 109.02 ms │ no change │
│ QQuery 64 │ 668.40 / 676.32 ±6.57 / 685.07 ms │ 679.74 / 682.97 ±3.44 / 689.47 ms │ no change │
│ QQuery 65 │ 247.33 / 253.45 ±4.92 / 260.86 ms │ 247.10 / 250.57 ±3.28 / 255.73 ms │ no change │
│ QQuery 66 │ 252.58 / 260.70 ±5.05 / 268.16 ms │ 234.04 / 256.64 ±15.98 / 282.19 ms │ no change │
│ QQuery 67 │ 298.27 / 307.96 ±5.74 / 314.59 ms │ 304.65 / 313.86 ±7.12 / 323.83 ms │ no change │
│ QQuery 68 │ 8.52 / 10.42 ±1.38 / 12.25 ms │ 8.26 / 9.60 ±0.75 / 10.42 ms │ +1.09x faster │
│ QQuery 69 │ 101.17 / 103.38 ±1.85 / 106.57 ms │ 100.10 / 102.46 ±1.81 / 104.92 ms │ no change │
│ QQuery 70 │ 332.93 / 356.78 ±15.40 / 379.19 ms │ 328.76 / 344.39 ±14.74 / 369.44 ms │ no change │
│ QQuery 71 │ 133.94 / 136.56 ±1.91 / 139.78 ms │ 133.47 / 137.45 ±2.35 / 139.68 ms │ no change │
│ QQuery 72 │ 601.26 / 613.17 ±11.01 / 629.74 ms │ 590.94 / 604.77 ±8.89 / 618.72 ms │ no change │
│ QQuery 73 │ 6.45 / 7.13 ±0.66 / 8.22 ms │ 6.38 / 7.28 ±0.95 / 9.12 ms │ no change │
│ QQuery 74 │ 557.05 / 565.25 ±5.27 / 572.16 ms │ 548.00 / 552.89 ±3.21 / 557.45 ms │ no change │
│ QQuery 75 │ 278.28 / 280.10 ±1.53 / 282.56 ms │ 272.38 / 274.72 ±1.78 / 277.45 ms │ no change │
│ QQuery 76 │ 130.43 / 131.35 ±1.01 / 133.25 ms │ 130.18 / 132.35 ±1.37 / 133.73 ms │ no change │
│ QQuery 77 │ 186.55 / 188.95 ±1.85 / 191.81 ms │ 186.57 / 189.96 ±2.61 / 194.46 ms │ no change │
│ QQuery 78 │ 339.82 / 344.50 ±3.58 / 347.71 ms │ 333.45 / 337.82 ±4.75 / 346.49 ms │ no change │
│ QQuery 79 │ 231.18 / 232.58 ±1.92 / 236.36 ms │ 229.00 / 232.11 ±2.71 / 235.89 ms │ no change │
│ QQuery 80 │ 319.03 / 320.82 ±1.25 / 322.78 ms │ 316.89 / 319.15 ±1.88 / 322.25 ms │ no change │
│ QQuery 81 │ 25.97 / 26.92 ±0.81 / 28.34 ms │ 25.60 / 26.47 ±0.82 / 28.02 ms │ no change │
│ QQuery 82 │ 198.73 / 201.00 ±1.94 / 203.33 ms │ 198.78 / 201.63 ±2.33 / 205.31 ms │ no change │
│ QQuery 83 │ 37.83 / 39.14 ±0.79 / 39.95 ms │ 38.12 / 38.59 ±0.43 / 39.16 ms │ no change │
│ QQuery 84 │ 48.05 / 48.90 ±0.59 / 49.90 ms │ 48.50 / 49.69 ±1.21 / 52.00 ms │ no change │
│ QQuery 85 │ 148.74 / 150.12 ±1.19 / 151.61 ms │ 146.35 / 148.64 ±1.50 / 150.85 ms │ no change │
│ QQuery 86 │ 39.99 / 40.99 ±1.00 / 42.66 ms │ 38.13 / 39.55 ±1.08 / 41.38 ms │ no change │
│ QQuery 87 │ 84.23 / 87.94 ±3.90 / 95.30 ms │ 84.26 / 87.73 ±2.91 / 92.96 ms │ no change │
│ QQuery 88 │ 99.61 / 100.67 ±0.92 / 102.20 ms │ 98.67 / 99.40 ±0.62 / 100.47 ms │ no change │
│ QQuery 89 │ 117.91 / 119.54 ±1.31 / 121.51 ms │ 118.13 / 118.87 ±0.56 / 119.77 ms │ no change │
│ QQuery 90 │ 22.57 / 23.24 ±0.53 / 23.99 ms │ 23.09 / 24.13 ±0.71 / 25.26 ms │ no change │
│ QQuery 91 │ 64.04 / 65.23 ±1.39 / 67.75 ms │ 59.55 / 63.37 ±2.39 / 65.71 ms │ no change │
│ QQuery 92 │ 57.25 / 58.03 ±0.43 / 58.47 ms │ 56.88 / 58.16 ±1.12 / 59.81 ms │ no change │
│ QQuery 93 │ 185.55 / 186.86 ±1.09 / 188.46 ms │ 183.73 / 184.84 ±0.58 / 185.36 ms │ no change │
│ QQuery 94 │ 61.75 / 62.98 ±0.81 / 63.84 ms │ 60.37 / 61.71 ±0.74 / 62.63 ms │ no change │
│ QQuery 95 │ 127.09 / 128.41 ±1.16 / 130.10 ms │ 126.07 / 126.84 ±0.62 / 127.66 ms │ no change │
│ QQuery 96 │ 73.75 / 75.22 ±1.14 / 76.62 ms │ 72.80 / 74.21 ±1.24 / 76.53 ms │ no change │
│ QQuery 97 │ 125.58 / 126.43 ±0.69 / 127.37 ms │ 125.39 / 126.91 ±0.90 / 127.93 ms │ no change │
│ QQuery 98 │ 153.24 / 155.86 ±1.77 / 158.71 ms │ 148.38 / 152.57 ±2.79 / 155.26 ms │ no change │
│ QQuery 99 │ 10727.29 / 10781.70 ±28.43 / 10806.15 ms │ 10776.69 / 10800.71 ±21.01 / 10830.22 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31276.06ms │
│ Total Time (early-group-emission) │ 31158.15ms │
│ Average Time (HEAD) │ 315.92ms │
│ Average Time (early-group-emission) │ 314.73ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 93 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 156.7s |
| Peak memory | 5.3 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 258.2s |
| CPU sys | 16.8s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 156.1s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.6 GiB |
| CPU user | 257.0s |
| CPU sys | 17.4s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: bounded hash table with overflow passthrough for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4238815821
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4238803110)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4238803110-1177-4hzvd 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (01835ce8773a93097f91e5ace1e53db1b1c30fce) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..01835ce8773a93097f91e5ace1e53db1b1c30fce) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241506167
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241497116)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241497116-1188-2fkrs 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (2ca217f76978a1144ce9047f92445ed9389c6f45) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..2ca217f76978a1144ce9047f92445ed9389c6f45) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241603006
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4241595190-1193-rw9n8 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing early-group-emission (0b08df010715efd91b6237e70649d0310b383f90) to 0143dfe (merge-base) [diff](https://github.com/apache/datafusion/compare/0143dfe1692a7a56a72c736041d93797ef651642..0b08df010715efd91b6237e70649d0310b383f90) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: early group emission with emit+reset for partial aggregation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21595:
URL: https://github.com/apache/datafusion/pull/21595#issuecomment-4241664940
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21595#issuecomment-4241595190)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and early-group-emission
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ early-group-emission ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.65 / 7.16 ±0.85 / 8.86 ms │ 6.59 / 7.05 ±0.78 / 8.61 ms │ no change │
│ QQuery 2 │ 143.75 / 144.92 ±0.70 / 145.66 ms │ 144.37 / 145.66 ±0.79 / 146.69 ms │ no change │
│ QQuery 3 │ 113.66 / 114.03 ±0.35 / 114.57 ms │ 113.65 / 114.27 ±0.83 / 115.87 ms │ no change │
│ QQuery 4 │ 1354.03 / 1377.66 ±21.89 / 1417.92 ms │ 1338.16 / 1364.17 ±17.08 / 1386.24 ms │ no change │
│ QQuery 5 │ 171.77 / 173.55 ±1.11 / 174.79 ms │ 172.59 / 174.23 ±1.99 / 177.97 ms │ no change │
│ QQuery 6 │ 840.95 / 861.49 ±26.45 / 912.43 ms │ 821.65 / 845.50 ±14.59 / 859.82 ms │ no change │
│ QQuery 7 │ 343.16 / 346.37 ±2.43 / 349.20 ms │ 337.35 / 345.91 ±4.65 / 350.48 ms │ no change │
│ QQuery 8 │ 116.40 / 117.30 ±1.27 / 119.73 ms │ 115.64 / 117.23 ±1.37 / 119.75 ms │ no change │
│ QQuery 9 │ 106.55 / 112.95 ±8.48 / 128.43 ms │ 100.82 / 108.94 ±10.23 / 128.59 ms │ no change │
│ QQuery 10 │ 107.94 / 109.42 ±0.89 / 110.58 ms │ 106.60 / 107.51 ±0.85 / 108.82 ms │ no change │
│ QQuery 11 │ 976.88 / 987.28 ±8.12 / 998.75 ms │ 934.99 / 948.66 ±18.91 / 985.37 ms │ no change │
│ QQuery 12 │ 46.89 / 48.49 ±1.29 / 50.54 ms │ 46.54 / 47.88 ±1.26 / 50.18 ms │ no change │
│ QQuery 13 │ 399.47 / 405.54 ±3.61 / 409.97 ms │ 405.35 / 406.70 ±1.46 / 409.49 ms │ no change │
│ QQuery 14 │ 1005.42 / 1012.28 ±4.29 / 1017.59 ms │ 1007.60 / 1074.31 ±117.86 / 1309.62 ms │ 1.06x slower │
│ QQuery 15 │ 16.23 / 17.58 ±0.98 / 18.77 ms │ 15.52 / 17.01 ±0.99 / 18.36 ms │ no change │
│ QQuery 16 │ 7.92 / 8.26 ±0.51 / 9.27 ms │ 7.31 / 7.99 ±0.88 / 9.73 ms │ no change │
│ QQuery 17 │ 232.53 / 233.15 ±0.45 / 233.84 ms │ 231.92 / 232.78 ±0.94 / 234.59 ms │ no change │
│ QQuery 18 │ 130.21 / 131.55 ±1.10 / 132.90 ms │ 127.50 / 128.96 ±1.09 / 130.76 ms │ no change │
│ QQuery 19 │ 157.13 / 158.78 ±1.21 / 160.45 ms │ 155.04 / 157.24 ±1.15 / 158.41 ms │ no change │
│ QQuery 20 │ 14.23 / 14.75 ±0.48 / 15.63 ms │ 13.95 / 14.23 ±0.22 / 14.48 ms │ no change │
│ QQuery 21 │ 20.46 / 21.33 ±0.58 / 22.18 ms │ 19.14 / 19.96 ±0.48 / 20.43 ms │ +1.07x faster │
│ QQuery 22 │ 497.65 / 499.67 ±1.41 / 501.27 ms │ 1155.72 / 1168.18 ±7.06 / 1175.48 ms │ 2.34x slower │
│ QQuery 23 │ 893.28 / 909.24 ±9.25 / 919.82 ms │ 881.27 / 894.06 ±8.21 / 906.92 ms │ no change │
│ QQuery 24 │ 389.64 / 392.67 ±3.31 / 399.05 ms │ 386.27 / 389.54 ±3.11 / 394.01 ms │ no change │
│ QQuery 25 │ 344.26 / 347.01 ±2.50 / 351.29 ms │ 344.77 / 347.28 ±2.57 / 352.03 ms │ no change │
│ QQuery 26 │ 83.41 / 84.53 ±1.10 / 86.23 ms │ 79.61 / 81.45 ±1.26 / 83.34 ms │ no change │
│ QQuery 27 │ 7.20 / 7.86 ±0.64 / 8.92 ms │ 6.74 / 7.17 ±0.52 / 8.19 ms │ +1.10x faster │
│ QQuery 28 │ 150.85 / 152.35 ±1.16 / 154.00 ms │ 148.77 / 150.75 ±1.84 / 153.82 ms │ no change │
│ QQuery 29 │ 283.89 / 286.16 ±1.34 / 287.80 ms │ 281.17 / 282.27 ±0.85 / 283.65 ms │ no change │
│ QQuery 30 │ 45.73 / 47.08 ±1.25 / 49.14 ms │ 43.78 / 46.17 ±1.34 / 47.52 ms │ no change │
│ QQuery 31 │ 172.60 / 172.94 ±0.26 / 173.26 ms │ 169.78 / 170.81 ±0.59 / 171.29 ms │ no change │
│ QQuery 32 │ 57.30 / 59.00 ±1.43 / 61.00 ms │ 56.55 / 57.94 ±1.09 / 59.34 ms │ no change │
│ QQuery 33 │ 140.94 / 143.65 ±1.83 / 145.89 ms │ 142.28 / 143.93 ±1.25 / 145.91 ms │ no change │
│ QQuery 34 │ 7.21 / 7.99 ±0.94 / 9.83 ms │ 7.03 / 7.57 ±0.61 / 8.60 ms │ +1.06x faster │
│ QQuery 35 │ 108.19 / 110.34 ±1.31 / 112.26 ms │ 105.46 / 107.45 ±1.40 / 109.29 ms │ no change │
│ QQuery 36 │ 6.79 / 7.03 ±0.16 / 7.25 ms │ 6.54 / 6.77 ±0.24 / 7.21 ms │ no change │
│ QQuery 37 │ 8.44 / 8.97 ±0.39 / 9.51 ms │ 8.59 / 9.28 ±0.75 / 10.71 ms │ no change │
│ QQuery 38 │ 86.10 / 88.96 ±3.51 / 95.67 ms │ 80.19 / 86.43 ±5.10 / 95.33 ms │ no change │
│ QQuery 39 │ 125.49 / 130.08 ±3.72 / 135.46 ms │ 128.02 / 131.13 ±2.81 / 134.86 ms │ no change │
│ QQuery 40 │ 113.39 / 117.18 ±6.66 / 130.48 ms │ 108.84 / 117.48 ±8.00 / 132.10 ms │ no change │
│ QQuery 41 │ 15.14 / 16.13 ±1.23 / 18.46 ms │ 15.13 / 16.52 ±1.21 / 18.38 ms │ no change │
│ QQuery 42 │ 108.15 / 110.26 ±1.75 / 113.49 ms │ 108.88 / 111.23 ±1.65 / 113.85 ms │ no change │
│ QQuery 43 │ 6.41 / 6.54 ±0.07 / 6.61 ms │ 6.07 / 6.24 ±0.21 / 6.65 ms │ no change │
│ QQuery 44 │ 12.18 / 12.87 ±0.68 / 14.16 ms │ 11.71 / 11.88 ±0.13 / 12.04 ms │ +1.08x faster │
│ QQuery 45 │ 51.72 / 52.57 ±0.58 / 53.32 ms │ 49.62 / 51.45 ±1.08 / 52.61 ms │ no change │
│ QQuery 46 │ 8.80 / 9.10 ±0.22 / 9.48 ms │ 8.41 / 8.66 ±0.19 / 8.92 ms │ no change │
│ QQuery 47 │ 760.43 / 765.56 ±5.97 / 776.87 ms │ 697.87 / 706.38 ±7.41 / 719.75 ms │ +1.08x faster │
│ QQuery 48 │ 294.64 / 298.97 ±3.39 / 305.01 ms │ 288.27 / 291.31 ±3.42 / 297.32 ms │ no change │
│ QQuery 49 │ 253.53 / 255.27 ±1.97 / 258.67 ms │ 251.86 / 253.24 ±1.47 / 255.93 ms │ no change │
│ QQuery 50 │ 232.17 / 233.02 ±0.65 / 234.15 ms │ 229.20 / 232.55 ±2.67 / 235.78 ms │ no change │
│ QQuery 51 │ 181.65 / 184.61 ±2.58 / 189.24 ms │ 180.93 / 184.28 ±3.05 / 189.80 ms │ no change │
│ QQuery 52 │ 107.30 / 108.58 ±0.98 / 109.83 ms │ 107.98 / 110.20 ±2.37 / 113.59 ms │ no change │
│ QQuery 53 │ 102.65 / 102.99 ±0.27 / 103.46 ms │ 102.40 / 103.02 ±0.71 / 104.38 ms │ no change │
│ QQuery 54 │ 147.04 / 149.55 ±1.40 / 151.34 ms │ 145.34 / 146.76 ±1.29 / 149.03 ms │ no change │
│ QQuery 55 │ 106.92 / 108.86 ±1.36 / 110.61 ms │ 107.34 / 109.44 ±1.52 / 111.78 ms │ no change │
│ QQuery 56 │ 141.56 / 142.92 ±0.95 / 144.43 ms │ 142.06 / 143.39 ±1.32 / 145.91 ms │ no change │
│ QQuery 57 │ 174.81 / 175.99 ±1.05 / 177.30 ms │ 176.10 / 178.09 ±1.09 / 179.14 ms │ no change │
│ QQuery 58 │ 290.15 / 295.26 ±4.91 / 303.57 ms │ 304.98 / 312.47 ±7.83 / 326.13 ms │ 1.06x slower │
│ QQuery 59 │ 200.42 / 202.59 ±1.55 / 205.10 ms │ 199.99 / 205.39 ±5.64 / 216.21 ms │ no change │
│ QQuery 60 │ 143.27 / 146.58 ±1.82 / 148.32 ms │ 145.71 / 147.30 ±1.18 / 148.79 ms │ no change │
│ QQuery 61 │ 13.55 / 13.80 ±0.14 / 13.96 ms │ 13.44 / 14.03 ±0.53 / 15.01 ms │ no change │
│ QQuery 62 │ 919.80 / 957.57 ±51.05 / 1057.09 ms │ 963.56 / 1075.80 ±89.25 / 1204.93 ms │ 1.12x slower │
│ QQuery 63 │ 105.82 / 107.60 ±1.51 / 110.12 ms │ 103.47 / 108.31 ±3.34 / 112.86 ms │ no change │
│ QQuery 64 │ 684.87 / 695.31 ±7.77 / 705.93 ms │ 694.68 / 707.65 ±7.79 / 716.47 ms │ no change │
│ QQuery 65 │ 256.55 / 260.10 ±2.94 / 265.46 ms │ 259.77 / 264.24 ±4.66 / 271.76 ms │ no change │
│ QQuery 66 │ 244.08 / 262.93 ±12.12 / 279.80 ms │ 263.06 / 269.88 ±5.27 / 277.14 ms │ no change │
│ QQuery 67 │ 316.32 / 321.94 ±3.73 / 326.11 ms │ 303.33 / 318.33 ±10.44 / 331.73 ms │ no change │
│ QQuery 68 │ 8.75 / 9.89 ±0.81 / 10.74 ms │ 8.97 / 10.23 ±1.01 / 11.72 ms │ no change │
│ QQuery 69 │ 102.73 / 104.72 ±2.08 / 108.53 ms │ 105.43 / 107.24 ±1.88 / 110.64 ms │ no change │
│ QQuery 70 │ 332.65 / 341.07 ±5.58 / 349.86 ms │ 351.24 / 374.71 ±15.96 / 388.99 ms │ 1.10x slower │
│ QQuery 71 │ 134.33 / 136.48 ±2.09 / 139.35 ms │ 136.97 / 139.81 ±2.87 / 145.15 ms │ no change │
│ QQuery 72 │ 613.14 / 629.12 ±13.18 / 649.87 ms │ 636.77 / 648.20 ±8.87 / 658.75 ms │ no change │
│ QQuery 73 │ 6.96 / 8.95 ±1.08 / 9.94 ms │ 7.47 / 8.27 ±0.44 / 8.67 ms │ +1.08x faster │
│ QQuery 74 │ 600.54 / 620.22 ±20.40 / 657.91 ms │ 615.99 / 633.95 ±10.31 / 646.28 ms │ no change │
│ QQuery 75 │ 278.70 / 280.78 ±1.33 / 282.80 ms │ 275.47 / 279.57 ±2.50 / 282.51 ms │ no change │
│ QQuery 76 │ 131.79 / 135.33 ±3.04 / 140.23 ms │ 132.26 / 133.72 ±1.41 / 135.95 ms │ no change │
│ QQuery 77 │ 189.45 / 190.43 ±0.96 / 192.13 ms │ 186.89 / 190.69 ±3.24 / 196.69 ms │ no change │
│ QQuery 78 │ 340.64 / 343.34 ±1.91 / 345.34 ms │ 348.36 / 351.47 ±2.12 / 353.87 ms │ no change │
│ QQuery 79 │ 236.89 / 240.63 ±3.01 / 244.02 ms │ 242.06 / 244.94 ±1.56 / 246.50 ms │ no change │
│ QQuery 80 │ 320.76 / 326.43 ±3.45 / 331.45 ms │ 321.88 / 326.00 ±2.50 / 329.75 ms │ no change │
│ QQuery 81 │ 26.41 / 27.26 ±0.79 / 28.71 ms │ 27.40 / 28.06 ±0.93 / 29.85 ms │ no change │
│ QQuery 82 │ 198.84 / 199.70 ±0.66 / 200.70 ms │ 203.37 / 205.15 ±1.34 / 207.51 ms │ no change │
│ QQuery 83 │ 39.21 / 40.28 ±1.36 / 42.90 ms │ 39.43 / 40.84 ±2.11 / 45.01 ms │ no change │
│ QQuery 84 │ 48.83 / 49.72 ±1.04 / 51.71 ms │ 48.46 / 50.51 ±1.17 / 51.68 ms │ no change │
│ QQuery 85 │ 150.28 / 150.83 ±0.33 / 151.23 ms │ 149.62 / 150.42 ±0.90 / 151.54 ms │ no change │
│ QQuery 86 │ 38.76 / 40.20 ±0.86 / 41.13 ms │ 39.52 / 40.33 ±0.79 / 41.82 ms │ no change │
│ QQuery 87 │ 86.19 / 89.23 ±3.27 / 95.27 ms │ 85.08 / 87.93 ±3.42 / 94.60 ms │ no change │
│ QQuery 88 │ 102.14 / 102.71 ±1.07 / 104.85 ms │ 99.56 / 101.82 ±1.86 / 104.63 ms │ no change │
│ QQuery 89 │ 118.12 / 119.61 ±0.87 / 120.81 ms │ 119.93 / 121.32 ±1.09 / 122.96 ms │ no change │
│ QQuery 90 │ 23.87 / 24.36 ±0.40 / 24.95 ms │ 23.41 / 24.39 ±1.13 / 26.32 ms │ no change │
│ QQuery 91 │ 64.22 / 65.84 ±1.06 / 67.26 ms │ 63.16 / 64.82 ±1.26 / 66.69 ms │ no change │
│ QQuery 92 │ 57.96 / 59.93 ±1.37 / 62.18 ms │ 57.80 / 59.56 ±1.25 / 61.23 ms │ no change │
│ QQuery 93 │ 186.98 / 189.98 ±1.57 / 191.27 ms │ 187.85 / 189.32 ±1.04 / 190.96 ms │ no change │
│ QQuery 94 │ 61.95 / 62.73 ±0.72 / 63.81 ms │ 61.58 / 62.33 ±0.43 / 62.90 ms │ no change │
│ QQuery 95 │ 127.00 / 130.61 ±2.34 / 133.17 ms │ 127.38 / 128.76 ±0.92 / 130.15 ms │ no change │
│ QQuery 96 │ 73.92 / 74.87 ±0.62 / 75.61 ms │ 70.92 / 74.18 ±1.67 / 75.44 ms │ no change │
│ QQuery 97 │ 123.56 / 126.66 ±2.13 / 129.89 ms │ 126.85 / 128.15 ±0.77 / 129.05 ms │ no change │
│ QQuery 98 │ 154.68 / 156.67 ±1.01 / 157.47 ms │ 151.99 / 155.43 ±2.04 / 157.92 ms │ no change │
│ QQuery 99 │ 10770.19 / 10840.63 ±60.61 / 10917.22 ms │ 10839.47 / 10886.37 ±29.14 / 10930.29 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31893.24ms │
│ Total Time (early-group-emission) │ 32728.42ms │
│ Average Time (HEAD) │ 322.15ms │
│ Average Time (early-group-emission) │ 330.59ms │
│ Queries Faster │ 6 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 88 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 159.8s |
| Peak memory | 5.5 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 264.1s |
| CPU sys | 17.6s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 163.9s |
| Peak memory | 4.9 GiB |
| Avg memory | 4.3 GiB |
| CPU user | 271.3s |
| CPU sys | 19.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org