You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Rachelint (via GitHub)" <gi...@apache.org> on 2026/06/29 01:09:27 UTC
[PR] Poc: expose group hash feedback from GroupValues [datafusion]
Rachelint opened a new pull request, #23229:
URL: https://github.com/apache/datafusion/pull/23229
## Which issue does this PR close?
Part of hash aggregate repartition POC.
## Rationale for this change
This POC prepares `GroupValues` for partition-aware partial aggregate output. The partial aggregate output path needs to know which input rows created new groups, and the hash for those rows, so it can later route newly-created group ids to target partitions without relying on a separate repartition + coalesce pipeline.
## What changes are included in this PR?
- Extend `GroupValues::intern` to fill per-row hashes and optionally return the input rows that created new groups.
- Keep hash calculation inside `GroupValues` implementations.
- Wire reusable hash/new-group-row buffers through aggregate hash table and row-hash paths.
- Update existing call sites and benchmarks to the new `intern` signature.
## Are these changes tested?
Yes:
- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion-physical-plan group_values --lib`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4844880187
๐ค Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4844538345)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and fuse-aggr-repart-poc-2
--------------------
Benchmark clickbench_partitioned.json
--------------------
โโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโ
โ Query โ HEAD โ fuse-aggr-repart-poc-2 โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 1.23 / 3.96 ยฑ5.40 / 14.76 ms โ 1.25 / 4.06 ยฑ5.55 / 15.16 ms โ no change โ
โ QQuery 1 โ 12.58 / 12.88 ยฑ0.17 / 13.10 ms โ 13.31 / 13.83 ยฑ0.54 / 14.66 ms โ 1.07x slower โ
โ QQuery 2 โ 35.61 / 35.95 ยฑ0.24 / 36.31 ms โ 36.30 / 36.49 ยฑ0.16 / 36.69 ms โ no change โ
โ QQuery 3 โ 30.48 / 31.39 ยฑ0.94 / 33.07 ms โ 30.61 / 30.85 ยฑ0.21 / 31.19 ms โ no change โ
โ QQuery 4 โ 1649.38 / 1727.72 ยฑ54.68 / 1775.51 ms โ 1171.38 / 1192.61 ยฑ14.95 / 1214.30 ms โ +1.45x faster โ
โ QQuery 5 โ 1681.09 / 1749.40 ยฑ44.57 / 1807.88 ms โ 842.19 / 874.40 ยฑ30.95 / 928.91 ms โ +2.00x faster โ
โ QQuery 6 โ 1.30 / 1.48 ยฑ0.27 / 2.02 ms โ 1.29 / 1.45 ยฑ0.26 / 1.95 ms โ no change โ
โ QQuery 7 โ 13.85 / 14.08 ยฑ0.13 / 14.22 ms โ 14.94 / 15.70 ยฑ1.22 / 18.13 ms โ 1.12x slower โ
โ QQuery 8 โ 2050.58 / 2119.62 ยฑ62.00 / 2229.22 ms โ 345.95 / 351.80 ยฑ3.62 / 356.40 ms โ +6.03x faster โ
โ QQuery 9 โ 463.95 / 494.32 ยฑ21.45 / 529.75 ms โ 1061.05 / 1113.73 ยฑ28.10 / 1141.00 ms โ 2.25x slower โ
โ QQuery 10 โ 76.43 / 77.14 ยฑ0.67 / 78.29 ms โ 74.42 / 75.60 ยฑ1.29 / 78.11 ms โ no change โ
โ QQuery 11 โ 87.28 / 93.49 ยฑ9.97 / 113.36 ms โ 85.33 / 85.68 ยฑ0.24 / 86.06 ms โ +1.09x faster โ
โ QQuery 12 โ 1648.92 / 1803.18 ยฑ88.83 / 1914.26 ms โ 968.58 / 1001.73 ยฑ19.55 / 1023.72 ms โ +1.80x faster โ
โ QQuery 13 โ 582.93 / 710.85 ยฑ116.35 / 863.04 ms โ 425.52 / 455.07 ยฑ33.13 / 518.38 ms โ +1.56x faster โ
โ QQuery 14 โ 541.54 / 555.41 ยฑ8.46 / 566.38 ms โ 290.45 / 302.97 ยฑ17.52 / 336.95 ms โ +1.83x faster โ
โ QQuery 15 โ 1907.57 / 1966.29 ยฑ46.84 / 2041.00 ms โ 1479.19 / 1555.23 ยฑ41.25 / 1596.65 ms โ +1.26x faster โ
โ QQuery 16 โ 4364.79 / 4460.91 ยฑ77.97 / 4576.34 ms โ 629.73 / 659.41 ยฑ23.69 / 687.98 ms โ +6.77x faster โ
โ QQuery 17 โ 4269.36 / 4391.60 ยฑ120.37 / 4549.93 ms โ 634.90 / 648.42 ยฑ10.94 / 662.18 ms โ +6.77x faster โ
โ QQuery 18 โ 17761.38 / 18597.37 ยฑ489.11 / 19253.41 ms โ 1251.26 / 1293.08 ยฑ31.85 / 1348.34 ms โ +14.38x faster โ
โ QQuery 19 โ 28.54 / 31.94 ยฑ5.89 / 43.68 ms โ 28.10 / 31.45 ยฑ5.55 / 42.48 ms โ no change โ
โ QQuery 20 โ 515.30 / 523.53 ยฑ9.02 / 535.77 ms โ 514.94 / 530.28 ยฑ15.71 / 559.54 ms โ no change โ
โ QQuery 21 โ 515.14 / 518.61 ยฑ3.57 / 525.37 ms โ 517.45 / 522.79 ยฑ5.60 / 533.16 ms โ no change โ
โ QQuery 22 โ 996.15 / 1002.36 ยฑ6.68 / 1011.86 ms โ 990.05 / 1003.37 ยฑ9.44 / 1016.11 ms โ no change โ
โ QQuery 23 โ 3066.49 / 3084.19 ยฑ15.47 / 3105.13 ms โ 3107.51 / 3148.50 ยฑ29.36 / 3191.77 ms โ no change โ
โ QQuery 24 โ 40.74 / 41.26 ยฑ0.52 / 42.22 ms โ 41.74 / 43.90 ยฑ2.70 / 48.94 ms โ 1.06x slower โ
โ QQuery 25 โ 110.31 / 112.48 ยฑ2.65 / 117.32 ms โ 112.96 / 114.11 ยฑ0.92 / 115.20 ms โ no change โ
โ QQuery 26 โ 41.66 / 42.64 ยฑ0.73 / 43.92 ms โ 42.01 / 46.51 ยฑ3.55 / 52.28 ms โ 1.09x slower โ
โ QQuery 27 โ 667.13 / 679.88 ยฑ12.13 / 702.44 ms โ 677.27 / 681.03 ยฑ5.53 / 691.95 ms โ no change โ
โ QQuery 28 โ 3443.23 / 3649.18 ยฑ149.01 / 3829.98 ms โ 3742.55 / 3837.57 ยฑ82.61 / 3950.28 ms โ 1.05x slower โ
โ QQuery 29 โ 40.51 / 52.89 ยฑ15.30 / 76.77 ms โ 40.44 / 40.69 ยฑ0.22 / 41.03 ms โ +1.30x faster โ
โ QQuery 30 โ 554.77 / 569.66 ยฑ13.29 / 589.31 ms โ 305.36 / 316.78 ยฑ7.84 / 329.53 ms โ +1.80x faster โ
โ QQuery 31 โ 289.92 / 303.12 ยฑ18.96 / 340.69 ms โ 283.19 / 292.80 ยฑ7.03 / 302.07 ms โ no change โ
โ QQuery 32 โ 949.81 / 1007.50 ยฑ33.07 / 1048.45 ms โ 1061.06 / 1074.22 ยฑ7.63 / 1083.88 ms โ 1.07x slower โ
โ QQuery 33 โ 25867.68 / 27037.94 ยฑ995.91 / 28314.21 ms โ 4548.33 / 4793.03 ยฑ167.98 / 5042.23 ms โ +5.64x faster โ
โ QQuery 34 โ 27679.77 / 28987.55 ยฑ876.01 / 30135.41 ms โ 4685.79 / 5028.50 ยฑ225.99 / 5309.30 ms โ +5.76x faster โ
โ QQuery 35 โ 1088.53 / 1161.10 ยฑ88.40 / 1330.31 ms โ 1335.53 / 1399.27 ยฑ64.02 / 1479.84 ms โ 1.21x slower โ
โ QQuery 36 โ 162.86 / 172.00 ยฑ7.97 / 183.72 ms โ 81.10 / 86.86 ยฑ4.80 / 93.05 ms โ +1.98x faster โ
โ QQuery 37 โ 38.33 / 49.38 ยฑ17.99 / 85.15 ms โ 37.14 / 38.33 ยฑ1.18 / 40.51 ms โ +1.29x faster โ
โ QQuery 38 โ 44.75 / 45.63 ยฑ0.71 / 46.87 ms โ 40.76 / 51.23 ยฑ16.00 / 82.88 ms โ 1.12x slower โ
โ QQuery 39 โ 185.96 / 194.54 ยฑ6.27 / 204.49 ms โ 135.12 / 149.71 ยฑ14.88 / 176.01 ms โ +1.30x faster โ
โ QQuery 40 โ 14.28 / 14.71 ยฑ0.26 / 15.10 ms โ 14.32 / 16.75 ยฑ4.13 / 24.99 ms โ 1.14x slower โ
โ QQuery 41 โ 13.67 / 14.18 ยฑ0.43 / 14.90 ms โ 13.89 / 15.00 ยฑ1.74 / 18.47 ms โ 1.06x slower โ
โ QQuery 42 โ 12.90 / 14.30 ยฑ2.45 / 19.19 ms โ 13.17 / 13.46 ยฑ0.28 / 14.00 ms โ +1.06x faster โ
โโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (HEAD) โ 108157.60ms โ
โ Total Time (fuse-aggr-repart-poc-2) โ 32988.25ms โ
โ Average Time (HEAD) โ 2515.29ms โ
โ Average Time (fuse-aggr-repart-poc-2) โ 767.17ms โ
โ Queries Faster โ 19 โ
โ Queries Slower โ 11 โ
โ Queries with No Change โ 13 โ
โ Queries with Failure โ 0 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned โ base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 545.1s |
| Peak memory | 12.3 GiB |
| Avg memory | 6.4 GiB |
| CPU user | 4842.8s |
| CPU sys | 322.8s |
| Peak spill | 0 B |
**clickbench_partitioned โ branch**
| Metric | Value |
|--------|-------|
| Wall time | 170.0s |
| Peak memory | 12.5 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 1024.4s |
| CPU sys | 86.4s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4843647256
run benchmarks clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4843915295
๐ค Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4843647256)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and fuse-aggr-repart-poc-2
--------------------
Benchmark clickbench_partitioned.json
--------------------
โโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโ
โ Query โ HEAD โ fuse-aggr-repart-poc-2 โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 1.23 / 3.95 ยฑ5.41 / 14.78 ms โ 1.24 / 4.05 ยฑ5.48 / 15.02 ms โ no change โ
โ QQuery 1 โ 12.84 / 13.05 ยฑ0.20 / 13.41 ms โ 12.74 / 12.83 ยฑ0.06 / 12.92 ms โ no change โ
โ QQuery 2 โ 36.80 / 37.13 ยฑ0.25 / 37.55 ms โ 35.57 / 35.79 ยฑ0.19 / 36.09 ms โ no change โ
โ QQuery 3 โ 30.75 / 31.41 ยฑ0.76 / 32.79 ms โ 30.34 / 30.85 ยฑ0.42 / 31.50 ms โ no change โ
โ QQuery 4 โ 1641.91 / 1729.24 ยฑ64.24 / 1835.36 ms โ 230.71 / 233.36 ยฑ2.60 / 237.88 ms โ +7.41x faster โ
โ QQuery 5 โ 1573.18 / 1762.93 ยฑ185.86 / 2092.25 ms โ 270.25 / 276.34 ยฑ4.55 / 282.17 ms โ +6.38x faster โ
โ QQuery 6 โ 1.26 / 1.41 ยฑ0.23 / 1.86 ms โ 1.27 / 1.44 ยฑ0.24 / 1.91 ms โ no change โ
โ QQuery 7 โ 13.70 / 13.92 ยฑ0.16 / 14.17 ms โ 13.92 / 14.02 ยฑ0.10 / 14.21 ms โ no change โ
โ QQuery 8 โ 2033.24 / 2055.11 ยฑ18.34 / 2083.82 ms โ 340.73 / 344.57 ยฑ2.80 / 348.74 ms โ +5.96x faster โ
โ QQuery 9 โ 479.21 / 485.59 ยฑ5.34 / 492.40 ms โ 453.60 / 470.04 ยฑ13.29 / 493.25 ms โ no change โ
โ QQuery 10 โ 75.79 / 86.69 ยฑ17.00 / 120.57 ms โ 69.50 / 73.22 ยฑ5.41 / 83.94 ms โ +1.18x faster โ
โ QQuery 11 โ 90.62 / 91.84 ยฑ1.10 / 93.50 ms โ 80.40 / 81.62 ยฑ1.47 / 84.39 ms โ +1.13x faster โ
โ QQuery 12 โ 1634.55 / 1701.81 ยฑ77.06 / 1850.32 ms โ 269.88 / 272.97 ยฑ3.64 / 279.33 ms โ +6.23x faster โ
โ QQuery 13 โ 565.80 / 798.94 ยฑ147.53 / 958.94 ms โ 360.25 / 373.10 ยฑ9.75 / 384.03 ms โ +2.14x faster โ
โ QQuery 14 โ 559.23 / 568.91 ยฑ8.28 / 581.13 ms โ 283.82 / 290.73 ยฑ4.16 / 295.32 ms โ +1.96x faster โ
โ QQuery 15 โ 1948.90 / 2028.62 ยฑ63.16 / 2130.34 ms โ 276.97 / 281.58 ยฑ3.57 / 286.66 ms โ +7.20x faster โ
โ QQuery 16 โ 4162.27 / 4335.65 ยฑ90.20 / 4419.39 ms โ 630.48 / 639.52 ยฑ7.84 / 652.10 ms โ +6.78x faster โ
โ QQuery 17 โ 4153.91 / 4306.26 ยฑ118.49 / 4441.64 ms โ 628.67 / 650.03 ยฑ22.64 / 692.45 ms โ +6.62x faster โ
โ QQuery 18 โ 17861.22 / 18714.80 ยฑ535.28 / 19436.77 ms โ 1271.81 / 1295.27 ยฑ17.46 / 1320.70 ms โ +14.45x faster โ
โ QQuery 19 โ 28.17 / 30.17 ยฑ2.96 / 36.05 ms โ 27.85 / 28.08 ยฑ0.12 / 28.17 ms โ +1.07x faster โ
โ QQuery 20 โ 514.46 / 520.09 ยฑ5.05 / 529.05 ms โ 512.82 / 523.76 ยฑ10.82 / 544.27 ms โ no change โ
โ QQuery 21 โ 511.95 / 517.24 ยฑ5.77 / 526.93 ms โ 510.28 / 525.06 ยฑ10.41 / 541.68 ms โ no change โ
โ QQuery 22 โ 979.12 / 987.22 ยฑ9.88 / 1006.20 ms โ 989.27 / 1012.37 ยฑ20.85 / 1043.08 ms โ no change โ
โ QQuery 23 โ 3010.41 / 3042.56 ยฑ24.52 / 3064.73 ms โ 3084.83 / 3124.51 ยฑ33.99 / 3169.72 ms โ no change โ
โ QQuery 24 โ 41.02 / 41.59 ยฑ0.76 / 43.02 ms โ 41.71 / 43.44 ยฑ1.82 / 46.93 ms โ no change โ
โ QQuery 25 โ 110.73 / 113.76 ยฑ2.50 / 116.27 ms โ 111.19 / 114.68 ยฑ5.41 / 125.43 ms โ no change โ
โ QQuery 26 โ 41.66 / 42.86 ยฑ0.74 / 43.79 ms โ 41.83 / 45.82 ยฑ3.63 / 50.12 ms โ 1.07x slower โ
โ QQuery 27 โ 665.88 / 676.57 ยฑ11.10 / 696.07 ms โ 671.79 / 677.52 ยฑ3.91 / 682.32 ms โ no change โ
โ QQuery 28 โ 3493.90 / 3798.99 ยฑ158.00 / 3948.21 ms โ 3072.35 / 3077.99 ยฑ4.90 / 3085.57 ms โ +1.23x faster โ
โ QQuery 29 โ 40.46 / 40.95 ยฑ0.39 / 41.54 ms โ 40.84 / 57.80 ยฑ15.82 / 76.97 ms โ 1.41x slower โ
โ QQuery 30 โ 549.65 / 570.50 ยฑ11.31 / 583.08 ms โ 307.59 / 312.38 ยฑ4.06 / 318.83 ms โ +1.83x faster โ
โ QQuery 31 โ 299.87 / 305.00 ยฑ3.00 / 309.13 ms โ 284.44 / 298.14 ยฑ9.00 / 309.72 ms โ no change โ
โ QQuery 32 โ 939.71 / 966.12 ยฑ37.59 / 1040.32 ms โ 1054.98 / 1064.72 ยฑ11.10 / 1080.27 ms โ 1.10x slower โ
โ QQuery 33 โ 25445.84 / 27426.47 ยฑ2060.28 / 31321.90 ms โ 1469.87 / 1505.13 ยฑ20.49 / 1530.42 ms โ +18.22x faster โ
โ QQuery 34 โ 28373.67 / 30066.24 ยฑ1095.85 / 31809.92 ms โ 1511.09 / 1538.69 ยฑ26.70 / 1572.55 ms โ +19.54x faster โ
โ QQuery 35 โ 1053.64 / 1122.79 ยฑ102.45 / 1326.72 ms โ 280.69 / 328.90 ยฑ66.63 / 460.86 ms โ +3.41x faster โ
โ QQuery 36 โ 158.22 / 177.87 ยฑ18.60 / 209.54 ms โ 70.90 / 75.27 ยฑ3.66 / 79.72 ms โ +2.36x faster โ
โ QQuery 37 โ 38.90 / 49.04 ยฑ18.40 / 85.76 ms โ 36.14 / 42.06 ยฑ11.17 / 64.39 ms โ +1.17x faster โ
โ QQuery 38 โ 42.36 / 43.71 ยฑ0.90 / 44.79 ms โ 43.45 / 46.90 ยฑ3.62 / 53.03 ms โ 1.07x slower โ
โ QQuery 39 โ 169.06 / 193.35 ยฑ15.99 / 218.83 ms โ 135.89 / 150.06 ยฑ8.90 / 160.42 ms โ +1.29x faster โ
โ QQuery 40 โ 14.98 / 15.48 ยฑ0.54 / 16.48 ms โ 14.54 / 15.07 ยฑ0.69 / 16.41 ms โ no change โ
โ QQuery 41 โ 13.83 / 14.03 ยฑ0.14 / 14.23 ms โ 13.95 / 14.22 ยฑ0.32 / 14.65 ms โ no change โ
โ QQuery 42 โ 13.55 / 13.78 ยฑ0.14 / 13.98 ms โ 13.33 / 18.08 ยฑ5.71 / 27.79 ms โ 1.31x slower โ
โโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (HEAD) โ 109543.66ms โ
โ Total Time (fuse-aggr-repart-poc-2) โ 20021.94ms โ
โ Average Time (HEAD) โ 2547.53ms โ
โ Average Time (fuse-aggr-repart-poc-2) โ 465.63ms โ
โ Queries Faster โ 21 โ
โ Queries Slower โ 5 โ
โ Queries with No Change โ 17 โ
โ Queries with Failure โ 0 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned โ base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 550.1s |
| Peak memory | 11.9 GiB |
| Avg memory | 6.5 GiB |
| CPU user | 4861.6s |
| CPU sys | 312.6s |
| Peak spill | 0 B |
**clickbench_partitioned โ branch**
| Metric | Value |
|--------|-------|
| Wall time | 105.0s |
| Peak memory | 13.2 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 1031.6s |
| CPU sys | 71.2s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4828155135
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-physical-plan v54.0.0 (current)
Built [ 38.945s] (current)
Parsing datafusion-physical-plan v54.0.0 (current)
Parsed [ 0.140s] (current)
Building datafusion-physical-plan v54.0.0 (baseline)
Built [ 36.807s] (baseline)
Parsing datafusion-physical-plan v54.0.0 (baseline)
Parsed [ 0.141s] (baseline)
Checking datafusion-physical-plan v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.982s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip
--- failure trait_method_parameter_count_changed: pub trait method parameter count changed ---
Description:
A trait method now takes a different number of parameters.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-item-signature
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/trait_method_parameter_count_changed.ron
Failed in:
GroupValues::intern now takes 4 instead of 2 parameters, in file /home/runner/work/datafusion/datafusion/datafusion/physical-plan/src/aggregates/group_values/mod.rs:102
Summary semver requires new major version: 1 major and 0 minor checks failed
Finished [ 79.793s] datafusion-physical-plan
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4843642027
run benckmarks clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4848044462
run benchmarks clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4844538345
run benchmarks clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4848194613
๐ค Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4848044462)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and fuse-aggr-repart-poc-2
--------------------
Benchmark clickbench_partitioned.json
--------------------
โโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโ
โ Query โ HEAD โ fuse-aggr-repart-poc-2 โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 1.20 / 4.00 ยฑ5.48 / 14.95 ms โ 1.20 / 3.93 ยฑ5.37 / 14.66 ms โ no change โ
โ QQuery 1 โ 12.72 / 13.19 ยฑ0.25 / 13.47 ms โ 12.45 / 12.79 ยฑ0.18 / 12.93 ms โ no change โ
โ QQuery 2 โ 35.73 / 36.11 ยฑ0.33 / 36.57 ms โ 35.22 / 35.69 ยฑ0.43 / 36.35 ms โ no change โ
โ QQuery 3 โ 30.56 / 31.17 ยฑ0.53 / 32.15 ms โ 30.15 / 30.27 ยฑ0.15 / 30.56 ms โ no change โ
โ QQuery 4 โ 1648.82 / 1715.26 ยฑ53.36 / 1804.02 ms โ 1149.16 / 1162.16 ยฑ10.40 / 1178.98 ms โ +1.48x faster โ
โ QQuery 5 โ 1569.86 / 1705.57 ยฑ92.33 / 1831.38 ms โ 809.19 / 857.90 ยฑ33.56 / 899.06 ms โ +1.99x faster โ
โ QQuery 6 โ 1.25 / 1.41 ยฑ0.25 / 1.91 ms โ 1.27 / 1.42 ยฑ0.22 / 1.85 ms โ no change โ
โ QQuery 7 โ 14.67 / 15.38 ยฑ1.00 / 17.36 ms โ 13.61 / 13.73 ยฑ0.10 / 13.88 ms โ +1.12x faster โ
โ QQuery 8 โ 2021.69 / 2076.59 ยฑ52.26 / 2172.37 ms โ 341.71 / 351.87 ยฑ8.41 / 362.41 ms โ +5.90x faster โ
โ QQuery 9 โ 468.17 / 487.60 ยฑ17.17 / 507.97 ms โ 1013.40 / 1081.86 ยฑ39.45 / 1136.48 ms โ 2.22x slower โ
โ QQuery 10 โ 77.43 / 78.40 ยฑ0.65 / 79.45 ms โ 71.42 / 72.79 ยฑ1.47 / 75.55 ms โ +1.08x faster โ
โ QQuery 11 โ 90.68 / 104.53 ยฑ26.62 / 157.75 ms โ 81.29 / 81.98 ยฑ0.49 / 82.76 ms โ +1.28x faster โ
โ QQuery 12 โ 1709.10 / 1746.42 ยฑ29.98 / 1783.56 ms โ 918.41 / 938.34 ยฑ12.38 / 954.47 ms โ +1.86x faster โ
โ QQuery 13 โ 638.84 / 763.50 ยฑ92.31 / 865.94 ms โ 414.99 / 426.25 ยฑ5.69 / 430.61 ms โ +1.79x faster โ
โ QQuery 14 โ 555.63 / 570.96 ยฑ15.29 / 596.16 ms โ 282.08 / 289.58 ยฑ9.77 / 307.93 ms โ +1.97x faster โ
โ QQuery 15 โ 1908.80 / 1951.87 ยฑ38.71 / 2018.78 ms โ 1431.65 / 1475.47 ยฑ51.00 / 1560.77 ms โ +1.32x faster โ
โ QQuery 16 โ 4160.52 / 4251.61 ยฑ65.35 / 4344.58 ms โ 625.96 / 637.79 ยฑ8.66 / 647.64 ms โ +6.67x faster โ
โ QQuery 17 โ 4131.35 / 4290.53 ยฑ151.21 / 4569.71 ms โ 621.63 / 640.82 ยฑ13.79 / 664.45 ms โ +6.70x faster โ
โ QQuery 18 โ 16991.42 / 18181.00 ยฑ888.47 / 19670.10 ms โ 1252.25 / 1274.23 ยฑ14.74 / 1294.58 ms โ +14.27x faster โ
โ QQuery 19 โ 27.86 / 28.40 ยฑ0.48 / 29.15 ms โ 27.43 / 35.50 ยฑ11.23 / 56.42 ms โ 1.25x slower โ
โ QQuery 20 โ 519.16 / 527.32 ยฑ4.44 / 531.35 ms โ 511.00 / 520.24 ยฑ6.13 / 528.25 ms โ no change โ
โ QQuery 21 โ 508.53 / 517.49 ยฑ7.56 / 528.10 ms โ 513.07 / 521.27 ยฑ5.69 / 530.19 ms โ no change โ
โ QQuery 22 โ 973.17 / 986.78 ยฑ12.60 / 1010.67 ms โ 981.15 / 989.81 ยฑ5.14 / 996.65 ms โ no change โ
โ QQuery 23 โ 2987.69 / 3051.27 ยฑ42.10 / 3098.30 ms โ 3073.43 / 3088.63 ยฑ13.59 / 3113.66 ms โ no change โ
โ QQuery 24 โ 41.56 / 46.82 ยฑ7.54 / 61.82 ms โ 42.16 / 48.51 ยฑ7.54 / 60.79 ms โ no change โ
โ QQuery 25 โ 110.21 / 113.93 ยฑ3.70 / 118.63 ms โ 111.87 / 113.12 ยฑ1.82 / 116.71 ms โ no change โ
โ QQuery 26 โ 42.08 / 42.70 ยฑ0.45 / 43.35 ms โ 41.80 / 42.40 ยฑ0.75 / 43.86 ms โ no change โ
โ QQuery 27 โ 663.82 / 668.57 ยฑ4.86 / 674.65 ms โ 666.51 / 674.05 ยฑ6.24 / 681.79 ms โ no change โ
โ QQuery 28 โ 3448.63 / 3595.39 ยฑ128.05 / 3815.67 ms โ 3722.39 / 3816.87 ยฑ62.10 / 3885.29 ms โ 1.06x slower โ
โ QQuery 29 โ 40.37 / 45.22 ยฑ5.61 / 53.06 ms โ 39.76 / 50.89 ยฑ19.49 / 89.66 ms โ 1.13x slower โ
โ QQuery 30 โ 543.71 / 565.90 ยฑ12.62 / 578.60 ms โ 301.88 / 316.08 ยฑ15.47 / 334.97 ms โ +1.79x faster โ
โ QQuery 31 โ 290.26 / 301.63 ยฑ7.52 / 312.66 ms โ 276.30 / 290.98 ยฑ9.06 / 304.06 ms โ no change โ
โ QQuery 32 โ 962.53 / 994.81 ยฑ33.40 / 1055.99 ms โ 1026.99 / 1037.10 ยฑ8.93 / 1052.52 ms โ no change โ
โ QQuery 33 โ 25263.60 / 26965.50 ยฑ1302.69 / 28423.44 ms โ 4706.08 / 4774.00 ยฑ91.03 / 4943.92 ms โ +5.65x faster โ
โ QQuery 34 โ 25598.05 / 27951.54 ยฑ1494.97 / 29958.43 ms โ 4597.75 / 4894.04 ยฑ196.28 / 5164.98 ms โ +5.71x faster โ
โ QQuery 35 โ 976.69 / 1158.11 ยฑ160.97 / 1456.82 ms โ 1325.76 / 1384.92 ยฑ55.88 / 1490.93 ms โ 1.20x slower โ
โ QQuery 36 โ 154.71 / 166.68 ยฑ8.32 / 176.30 ms โ 78.07 / 86.10 ยฑ7.66 / 95.54 ms โ +1.94x faster โ
โ QQuery 37 โ 37.37 / 54.02 ยฑ28.24 / 110.14 ms โ 35.70 / 36.46 ยฑ0.50 / 37.16 ms โ +1.48x faster โ
โ QQuery 38 โ 42.73 / 44.38 ยฑ1.17 / 46.05 ms โ 42.66 / 51.23 ยฑ11.57 / 73.61 ms โ 1.15x slower โ
โ QQuery 39 โ 185.14 / 198.35 ยฑ15.41 / 228.59 ms โ 140.79 / 156.12 ยฑ15.58 / 184.86 ms โ +1.27x faster โ
โ QQuery 40 โ 14.20 / 15.09 ยฑ0.48 / 15.57 ms โ 13.72 / 15.01 ยฑ2.35 / 19.72 ms โ no change โ
โ QQuery 41 โ 13.60 / 13.77 ยฑ0.14 / 13.94 ms โ 13.20 / 14.61 ยฑ2.09 / 18.75 ms โ 1.06x slower โ
โ QQuery 42 โ 12.80 / 13.00 ยฑ0.13 / 13.19 ms โ 13.09 / 14.92 ยฑ3.47 / 21.86 ms โ 1.15x slower โ
โโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (HEAD) โ 106091.76ms โ
โ Total Time (fuse-aggr-repart-poc-2) โ 32361.74ms โ
โ Average Time (HEAD) โ 2467.25ms โ
โ Average Time (fuse-aggr-repart-poc-2) โ 752.60ms โ
โ Queries Faster โ 19 โ
โ Queries Slower โ 8 โ
โ Queries with No Change โ 16 โ
โ Queries with Failure โ 0 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned โ base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 535.1s |
| Peak memory | 12.2 GiB |
| Avg memory | 6.5 GiB |
| CPU user | 4783.2s |
| CPU sys | 315.1s |
| Peak spill | 0 B |
**clickbench_partitioned โ branch**
| Metric | Value |
|--------|-------|
| Wall time | 165.0s |
| Peak memory | 11.8 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 1013.0s |
| CPU sys | 84.0s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4848062138
๐ค Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4848044462)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4848044462-762-p8jwn 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing fuse-aggr-repart-poc-2 (bc70b9de3d9ff5695b50fc6fff3b02e883b6fe0b) to 32d3d3a (merge-base) [diff](https://github.com/apache/datafusion/compare/32d3d3a0e3ec69cdc1795240d7a0ba0f2b9c38b0..bc70b9de3d9ff5695b50fc6fff3b02e883b6fe0b) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4844537025
Huh what's HEAD doing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4844617530
> Huh what's HEAD doing?
Can see
https://github.com/apache/datafusion/pull/23250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4843671932
๐ค Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4843647256)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4843647256-752-r4jz6 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing fuse-aggr-repart-poc-2 (d82f05bdbf6939cee9bde932eb427ff1d08f4899) to 32d3d3a (merge-base) [diff](https://github.com/apache/datafusion/compare/32d3d3a0e3ec69cdc1795240d7a0ba0f2b9c38b0..d82f05bdbf6939cee9bde932eb427ff1d08f4899) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #23229:
URL: https://github.com/apache/datafusion/pull/23229#issuecomment-4844557751
๐ค Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/23229#issuecomment-4844538345)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4844538345-753-x75z6 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing fuse-aggr-repart-poc-2 (bc70b9de3d9ff5695b50fc6fff3b02e883b6fe0b) to 32d3d3a (merge-base) [diff](https://github.com/apache/datafusion/compare/32d3d3a0e3ec69cdc1795240d7a0ba0f2b9c38b0..bc70b9de3d9ff5695b50fc6fff3b02e883b6fe0b) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Poc: expose group hash feedback from GroupValues [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint closed pull request #23229: Poc: expose group hash feedback from GroupValues
URL: https://github.com/apache/datafusion/pull/23229
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org