You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Dandandan (via GitHub)" <gi...@apache.org> on 2026/04/18 13:09:11 UTC
[PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Dandandan opened a new pull request, #21718:
URL: https://github.com/apache/datafusion/pull/21718
## Summary
Adds an experimental workspace crate, `datafusion-morsel-scheduler`, that executes an `ExecutionPlan` on a pool of pinned per-core Tokio `current_thread` runtimes instead of the default multi-threaded runtime. The ClickBench benchmark runner defaults to the new scheduler; existing paths remain available via `--no-morsel-scheduler`.
This is a **draft / prototype** to explore the morsel-driven + thread-per-core direction on top of today's `ExecutionPlan` abstraction. Not aimed for merge in its current shape — opening for discussion of the architecture.
## Architecture
Three layers in the new crate:
### 1. `runtime::WorkerPool`
N OS threads, each owning a dedicated `current_thread` runtime plus a `LocalSet`. Work is shipped as a `Send` closure that constructs the real (possibly `!Send`) future on-worker and `spawn_local`-s it, so the future never migrates across threads. `spawn_on(worker_id, ..)` for partition-affinity, `spawn_any(..)` for round-robin.
### 2. Inboxes + pipeline splitting
A **Pipeline** is a breaker-free `ExecutionPlan` subgraph. The planner walks the plan and cuts at v1 breakers — `CoalescePartitionsExec` and `SortPreservingMergeExec` — replacing each breaker's children with `InboxSourceExec` stubs and recording the detached subtrees as upstream pipelines. Morsels flow through bounded mpsc inboxes between pipelines (natural backpressure via the bounded capacity).
\`\`\`
Pipeline A (final): Pipeline B (upstream):
Root OriginalChild
└─ SPM (N partitions)
└─ InboxSourceExec ◄──── one sender per partition
\`\`\`
Breakers explicitly **not** cut in v1: `RepartitionExec` (would require reimplementing partitioning), `SortExec`, `HashJoinExec` build-side.
### 3. `WorkerDispatchExec` + executor
Before scheduling, each pipeline's leaves are wrapped with `WorkerDispatchExec` so that `RepartitionExec`'s lazy-spawned input fetchers spread across the pool instead of piling onto whichever worker wins the first-poll race. The top-level `execute()` splits the plan into pipelines, schedules each partition on `worker = partition % N`, and bridges the final pipeline's output back to the caller as a normal `SendableRecordBatchStream`.
### ClickBench wiring
The benchmark runner now builds a `WorkerPool` by default and routes query execution through `morsel_scheduler::execute`. New flags:
- \`--no-morsel-scheduler\` — fall back to the ambient multi-threaded runtime.
- \`--morsel-workers N\` — override worker count (default: \`std::thread::available_parallelism()\`).
## What's intentionally out of scope for v1
- **Cutting at RepartitionExec / SortExec / hash-join build** — hooks exist in the planner but these would need per-breaker partitioning logic.
- **NUMA / CPU pinning** — workers are OS threads, not core-pinned.
- **Explicit work-stealing across workers** — partitions are pinned to \`partition % N\`; a skewed pipeline will stall its worker.
- **Async I/O on workers** — no \`tokio-uring\` integration in this crate; the \`tokio-uring-benchmark-demo\` branch is a separate experiment that could eventually layer on top.
## Test plan
- [x] \`cargo fmt --all\`
- [x] \`cargo clippy -p datafusion-morsel-scheduler -p datafusion-benchmarks --all-targets -- -D warnings\`
- [x] \`cargo test -p datafusion-morsel-scheduler --lib\` — 7 unit tests covering \`WorkerPool\`, \`Inbox\`/\`InboxSourceExec\`, planner cuts, and an end-to-end plan run
- [ ] End-to-end ClickBench comparison (morsel scheduler vs. baseline)
- [ ] Sanity test on larger queries (joins, aggregations with RepartitionExec)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273806363
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273756125-1473-kqx9c 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (112637f26a070e610500c18c50e2633fc822588c) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..112637f26a070e610500c18c50e2633fc822588c) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273727398
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273723891-1465-lx5ms 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (580bb936213d6513f753d97bcb4728ffe91c8e21) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..580bb936213d6513f753d97bcb4728ffe91c8e21) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275379531
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275376685-1494-cg78n 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (de8c7f280fa58ce87b040e62103ff2d46db84706) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..de8c7f280fa58ce87b040e62103ff2d46db84706) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273831362
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273827001-1476-mlznv 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (99e872a19bade26aa57b201022f8a3f9b86692a8) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..99e872a19bade26aa57b201022f8a3f9b86692a8) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273865616
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.19 / 4.45 ±6.39 / 17.23 ms │ 1.24 / 4.50 ±6.39 / 17.28 ms │ no change │
│ QQuery 1 │ 14.35 / 14.63 ±0.17 / 14.86 ms │ 22.17 / 30.08 ±10.64 / 50.00 ms │ 2.06x slower │
│ QQuery 2 │ 43.71 / 44.18 ±0.31 / 44.62 ms │ 58.84 / 96.73 ±27.14 / 131.88 ms │ 2.19x slower │
│ QQuery 3 │ 40.23 / 41.05 ±1.05 / 43.00 ms │ 82.64 / 110.86 ±23.95 / 141.69 ms │ 2.70x slower │
│ QQuery 4 │ 281.04 / 286.67 ±4.06 / 293.62 ms │ 737.72 / 942.25 ±158.08 / 1197.65 ms │ 3.29x slower │
│ QQuery 5 │ 331.31 / 338.96 ±4.79 / 344.15 ms │ 708.22 / 823.73 ±89.06 / 981.55 ms │ 2.43x slower │
│ QQuery 6 │ 6.56 / 8.18 ±2.78 / 13.72 ms │ 3.78 / 5.81 ±1.21 / 7.44 ms │ +1.41x faster │
│ QQuery 7 │ 16.64 / 17.04 ±0.30 / 17.54 ms │ 24.25 / 30.42 ±4.01 / 35.59 ms │ 1.79x slower │
│ QQuery 8 │ 397.17 / 414.12 ±9.29 / 421.86 ms │ 904.47 / 1062.75 ±91.88 / 1155.98 ms │ 2.57x slower │
│ QQuery 9 │ 629.98 / 647.22 ±9.05 / 656.72 ms │ 1435.88 / 1561.95 ±101.17 / 1728.48 ms │ 2.41x slower │
│ QQuery 10 │ 90.14 / 93.28 ±2.80 / 98.52 ms │ 164.34 / 238.49 ±54.72 / 304.04 ms │ 2.56x slower │
│ QQuery 11 │ 103.83 / 104.98 ±1.22 / 107.33 ms │ 205.20 / 247.35 ±39.39 / 320.23 ms │ 2.36x slower │
│ QQuery 12 │ 335.47 / 342.08 ±3.39 / 344.37 ms │ 720.17 / 829.66 ±136.86 / 1096.63 ms │ 2.43x slower │
│ QQuery 13 │ 451.85 / 470.32 ±12.87 / 490.14 ms │ 985.15 / 1187.48 ±110.07 / 1316.62 ms │ 2.52x slower │
│ QQuery 14 │ 345.79 / 349.05 ±3.58 / 354.03 ms │ 758.53 / 837.51 ±87.00 / 979.93 ms │ 2.40x slower │
│ QQuery 15 │ 345.78 / 366.79 ±16.19 / 388.39 ms │ 985.66 / 1147.12 ±93.18 / 1246.85 ms │ 3.13x slower │
│ QQuery 16 │ 709.54 / 713.48 ±3.29 / 718.74 ms │ 1724.26 / 2068.07 ±414.64 / 2870.80 ms │ 2.90x slower │
│ QQuery 17 │ 706.27 / 710.69 ±2.78 / 714.89 ms │ 1530.51 / 1764.96 ±146.04 / 1973.03 ms │ 2.48x slower │
│ QQuery 18 │ 1417.68 / 1455.85 ±24.75 / 1480.76 ms │ 3140.72 / 3684.14 ±297.16 / 3992.08 ms │ 2.53x slower │
│ QQuery 19 │ 36.00 / 37.58 ±0.90 / 38.58 ms │ 82.08 / 99.08 ±16.20 / 128.20 ms │ 2.64x slower │
│ QQuery 20 │ 717.02 / 732.62 ±11.39 / 750.95 ms │ 873.12 / 1347.30 ±399.74 / 1994.71 ms │ 1.84x slower │
│ QQuery 21 │ 770.36 / 778.66 ±7.43 / 791.64 ms │ 1042.65 / 1273.75 ±183.87 / 1502.21 ms │ 1.64x slower │
│ QQuery 22 │ 1142.88 / 1151.84 ±7.62 / 1165.76 ms │ 2327.07 / 3128.20 ±444.15 / 3683.81 ms │ 2.72x slower │
│ QQuery 23 │ 3095.26 / 3115.14 ±15.24 / 3140.21 ms │ 4135.44 / 4691.17 ±359.30 / 5189.03 ms │ 1.51x slower │
│ QQuery 24 │ 102.33 / 105.50 ±2.14 / 108.24 ms │ 100.97 / 123.67 ±13.31 / 137.03 ms │ 1.17x slower │
│ QQuery 25 │ 137.93 / 140.21 ±1.90 / 142.38 ms │ 252.90 / 291.13 ±26.09 / 321.83 ms │ 2.08x slower │
│ QQuery 26 │ 101.27 / 102.28 ±0.59 / 103.04 ms │ 126.09 / 161.20 ±27.31 / 205.13 ms │ 1.58x slower │
│ QQuery 27 │ 848.97 / 851.97 ±2.07 / 854.11 ms │ 1185.09 / 1446.36 ±202.21 / 1749.88 ms │ 1.70x slower │
│ QQuery 28 │ 3244.70 / 3270.11 ±13.69 / 3284.28 ms │ 10498.43 / 12693.67 ±1230.14 / 14070.57 ms │ 3.88x slower │
│ QQuery 29 │ 51.78 / 55.21 ±4.43 / 63.47 ms │ 102.91 / 135.34 ±26.44 / 174.55 ms │ 2.45x slower │
│ QQuery 30 │ 359.73 / 365.09 ±3.48 / 370.45 ms │ 730.26 / 791.03 ±70.90 / 879.80 ms │ 2.17x slower │
│ QQuery 31 │ 357.55 / 372.05 ±9.15 / 386.36 ms │ 633.47 / 822.98 ±152.45 / 1096.63 ms │ 2.21x slower │
│ QQuery 32 │ 1240.65 / 1279.88 ±24.81 / 1308.71 ms │ 3281.84 / 3631.49 ±303.73 / 4111.77 ms │ 2.84x slower │
│ QQuery 33 │ 1498.18 / 1532.67 ±33.46 / 1586.67 ms │ 3006.28 / 3865.90 ±670.90 / 4579.40 ms │ 2.52x slower │
│ QQuery 34 │ 1472.35 / 1489.45 ±11.53 / 1505.68 ms │ 3163.68 / 4208.75 ±729.15 / 5403.07 ms │ 2.83x slower │
│ QQuery 35 │ 391.73 / 398.48 ±8.27 / 414.43 ms │ 741.95 / 924.81 ±155.06 / 1101.11 ms │ 2.32x slower │
│ QQuery 36 │ 111.20 / 115.93 ±5.24 / 124.05 ms │ 94.06 / 114.32 ±19.47 / 150.11 ms │ no change │
│ QQuery 37 │ 47.97 / 50.13 ±1.79 / 52.41 ms │ 41.30 / 45.72 ±6.32 / 58.09 ms │ +1.10x faster │
│ QQuery 38 │ 77.12 / 78.18 ±0.81 / 79.31 ms │ 70.90 / 73.53 ±1.37 / 74.74 ms │ +1.06x faster │
│ QQuery 39 │ 219.30 / 221.84 ±1.35 / 222.88 ms │ 182.04 / 198.57 ±11.02 / 215.78 ms │ +1.12x faster │
│ QQuery 40 │ 24.41 / 26.96 ±2.36 / 31.01 ms │ 20.06 / 24.04 ±3.29 / 28.37 ms │ +1.12x faster │
│ QQuery 41 │ 19.18 / 20.65 ±0.87 / 21.53 ms │ 17.56 / 20.14 ±2.83 / 24.32 ms │ no change │
│ QQuery 42 │ 19.15 / 19.70 ±0.35 / 20.20 ms │ 15.92 / 16.62 ±0.58 / 17.69 ms │ +1.19x faster │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22735.15ms │
│ Total Time (morsel-scheduler) │ 56802.65ms │
│ Average Time (HEAD) │ 528.72ms │
│ Average Time (morsel-scheduler) │ 1320.99ms │
│ Queries Faster │ 6 │
│ Queries Slower │ 34 │
│ Queries with No Change │ 3 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 114.7s |
| Peak memory | 33.7 GiB |
| Avg memory | 25.9 GiB |
| CPU user | 1068.6s |
| CPU sys | 97.4s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 285.5s |
| Peak memory | 40.6 GiB |
| Avg memory | 29.3 GiB |
| CPU user | 994.7s |
| CPU sys | 119.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273728372
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273723891-1467-lnt4n 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (580bb936213d6513f753d97bcb4728ffe91c8e21) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..580bb936213d6513f753d97bcb4728ffe91c8e21) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273767545
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273756125-1471-v8gff 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (112637f26a070e610500c18c50e2633fc822588c) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..112637f26a070e610500c18c50e2633fc822588c) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275398298
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.54 / 7.02 ±0.85 / 8.72 ms │ 6.63 / 7.09 ±0.79 / 8.67 ms │ no change │
│ QQuery 2 │ 146.69 / 147.04 ±0.24 / 147.39 ms │ 145.53 / 146.88 ±1.07 / 148.74 ms │ no change │
│ QQuery 3 │ 113.58 / 114.17 ±0.38 / 114.61 ms │ 112.13 / 112.73 ±0.39 / 113.12 ms │ no change │
│ QQuery 4 │ 1255.87 / 1288.77 ±22.77 / 1312.82 ms │ 1249.31 / 1305.62 ±34.90 / 1343.88 ms │ no change │
│ QQuery 5 │ 172.48 / 174.79 ±1.45 / 177.01 ms │ 171.62 / 173.66 ±1.62 / 176.36 ms │ no change │
│ QQuery 6 │ 825.00 / 840.28 ±10.99 / 859.13 ms │ 836.10 / 859.12 ±21.01 / 896.51 ms │ no change │
│ QQuery 7 │ 339.32 / 342.66 ±2.33 / 346.24 ms │ 338.87 / 342.14 ±2.27 / 345.53 ms │ no change │
│ QQuery 8 │ 114.35 / 116.24 ±1.08 / 117.62 ms │ 113.65 / 115.79 ±1.36 / 117.46 ms │ no change │
│ QQuery 9 │ 100.75 / 106.36 ±5.08 / 114.26 ms │ 100.83 / 103.59 ±2.91 / 107.78 ms │ no change │
│ QQuery 10 │ 106.18 / 107.45 ±0.86 / 108.85 ms │ 106.38 / 107.15 ±0.48 / 107.76 ms │ no change │
│ QQuery 11 │ 883.16 / 904.91 ±16.55 / 929.17 ms │ 854.36 / 874.73 ±12.57 / 887.67 ms │ no change │
│ QQuery 12 │ 44.96 / 46.13 ±1.09 / 48.11 ms │ 44.33 / 46.03 ±1.24 / 47.94 ms │ no change │
│ QQuery 13 │ 397.71 / 402.28 ±3.84 / 406.99 ms │ 396.96 / 400.28 ±3.59 / 407.08 ms │ no change │
│ QQuery 14 │ 1005.97 / 1010.66 ±3.37 / 1014.49 ms │ 1002.77 / 1009.77 ±5.68 / 1018.80 ms │ no change │
│ QQuery 15 │ 15.71 / 16.20 ±0.75 / 17.65 ms │ 14.93 / 16.10 ±1.73 / 19.52 ms │ no change │
│ QQuery 16 │ 7.20 / 8.09 ±0.86 / 9.44 ms │ 7.24 / 7.47 ±0.16 / 7.68 ms │ +1.08x faster │
│ QQuery 17 │ 228.47 / 230.64 ±1.83 / 233.98 ms │ 228.54 / 230.39 ±1.32 / 232.42 ms │ no change │
│ QQuery 18 │ 128.31 / 129.14 ±0.69 / 130.25 ms │ 126.94 / 128.25 ±1.10 / 129.77 ms │ no change │
│ QQuery 19 │ 156.22 / 157.78 ±1.00 / 159.02 ms │ 152.35 / 155.47 ±1.80 / 157.15 ms │ no change │
│ QQuery 20 │ 13.78 / 14.29 ±0.48 / 15.05 ms │ 13.63 / 14.12 ±0.42 / 14.88 ms │ no change │
│ QQuery 21 │ 19.45 / 20.00 ±0.45 / 20.77 ms │ 19.16 / 19.53 ±0.26 / 19.95 ms │ no change │
│ QQuery 22 │ 478.23 / 482.50 ±3.08 / 486.49 ms │ 481.28 / 482.77 ±1.23 / 484.69 ms │ no change │
│ QQuery 23 │ 860.58 / 873.52 ±8.36 / 885.41 ms │ 866.87 / 873.69 ±6.93 / 885.97 ms │ no change │
│ QQuery 24 │ 384.24 / 385.29 ±1.22 / 387.54 ms │ 378.52 / 382.12 ±4.26 / 389.99 ms │ no change │
│ QQuery 25 │ 341.08 / 344.36 ±1.81 / 346.64 ms │ 338.12 / 340.96 ±2.05 / 343.37 ms │ no change │
│ QQuery 26 │ 81.30 / 83.34 ±1.29 / 85.13 ms │ 81.46 / 82.94 ±0.96 / 84.46 ms │ no change │
│ QQuery 27 │ 7.07 / 7.26 ±0.14 / 7.50 ms │ 6.96 / 7.13 ±0.18 / 7.45 ms │ no change │
│ QQuery 28 │ 149.05 / 159.81 ±18.20 / 196.16 ms │ 148.17 / 150.33 ±1.33 / 151.80 ms │ +1.06x faster │
│ QQuery 29 │ 280.95 / 284.17 ±3.24 / 289.69 ms │ 280.89 / 282.61 ±1.92 / 286.26 ms │ no change │
│ QQuery 30 │ 42.13 / 44.74 ±1.55 / 46.37 ms │ 43.31 / 45.11 ±1.74 / 48.07 ms │ no change │
│ QQuery 31 │ 169.48 / 172.54 ±2.79 / 176.85 ms │ 168.41 / 170.06 ±1.14 / 171.79 ms │ no change │
│ QQuery 32 │ 57.15 / 58.43 ±1.00 / 59.55 ms │ 56.49 / 57.57 ±0.83 / 58.91 ms │ no change │
│ QQuery 33 │ 139.74 / 141.56 ±1.48 / 143.63 ms │ 139.53 / 141.28 ±1.46 / 143.69 ms │ no change │
│ QQuery 34 │ 7.20 / 7.71 ±0.66 / 8.99 ms │ 7.19 / 7.55 ±0.45 / 8.40 ms │ no change │
│ QQuery 35 │ 107.40 / 108.86 ±1.39 / 111.26 ms │ 105.57 / 107.76 ±1.25 / 109.39 ms │ no change │
│ QQuery 36 │ 6.48 / 6.87 ±0.41 / 7.65 ms │ 6.58 / 6.82 ±0.24 / 7.16 ms │ no change │
│ QQuery 37 │ 8.34 / 8.68 ±0.23 / 8.99 ms │ 8.04 / 8.62 ±0.53 / 9.54 ms │ no change │
│ QQuery 38 │ 84.30 / 88.63 ±3.07 / 92.65 ms │ 80.19 / 86.06 ±3.78 / 90.48 ms │ no change │
│ QQuery 39 │ 125.32 / 128.15 ±1.63 / 129.73 ms │ 125.90 / 129.41 ±3.69 / 135.64 ms │ no change │
│ QQuery 40 │ 109.99 / 117.41 ±6.31 / 128.38 ms │ 107.27 / 115.53 ±9.13 / 133.26 ms │ no change │
│ QQuery 41 │ 14.67 / 15.08 ±0.53 / 16.12 ms │ 14.13 / 15.36 ±0.83 / 16.49 ms │ no change │
│ QQuery 42 │ 106.21 / 109.41 ±1.74 / 111.53 ms │ 105.84 / 107.39 ±1.13 / 109.10 ms │ no change │
│ QQuery 43 │ 6.16 / 6.26 ±0.12 / 6.46 ms │ 6.03 / 6.14 ±0.13 / 6.37 ms │ no change │
│ QQuery 44 │ 11.56 / 12.31 ±0.49 / 12.86 ms │ 11.53 / 12.11 ±0.61 / 13.28 ms │ no change │
│ QQuery 45 │ 50.00 / 51.44 ±0.80 / 52.27 ms │ 49.90 / 51.35 ±0.97 / 52.93 ms │ no change │
│ QQuery 46 │ 8.33 / 8.52 ±0.13 / 8.69 ms │ 8.50 / 8.85 ±0.43 / 9.65 ms │ no change │
│ QQuery 47 │ 692.98 / 700.38 ±5.51 / 708.46 ms │ 681.30 / 689.51 ±5.88 / 698.42 ms │ no change │
│ QQuery 48 │ 287.96 / 291.75 ±3.54 / 296.52 ms │ 288.91 / 291.92 ±3.23 / 298.18 ms │ no change │
│ QQuery 49 │ 252.35 / 256.54 ±4.14 / 262.54 ms │ 249.68 / 252.79 ±2.79 / 256.42 ms │ no change │
│ QQuery 50 │ 222.55 / 231.03 ±4.56 / 235.28 ms │ 220.81 / 226.53 ±3.94 / 232.59 ms │ no change │
│ QQuery 51 │ 182.07 / 184.87 ±2.22 / 188.07 ms │ 178.73 / 180.74 ±2.21 / 184.32 ms │ no change │
│ QQuery 52 │ 107.90 / 109.50 ±1.16 / 111.10 ms │ 106.49 / 108.08 ±0.83 / 108.76 ms │ no change │
│ QQuery 53 │ 103.17 / 104.06 ±0.58 / 104.59 ms │ 101.08 / 102.14 ±0.76 / 103.33 ms │ no change │
│ QQuery 54 │ 146.01 / 147.72 ±1.08 / 149.05 ms │ 145.05 / 147.55 ±1.78 / 150.34 ms │ no change │
│ QQuery 55 │ 106.44 / 107.66 ±1.03 / 109.54 ms │ 106.04 / 106.82 ±0.42 / 107.16 ms │ no change │
│ QQuery 56 │ 140.87 / 142.50 ±1.47 / 144.70 ms │ 139.94 / 143.28 ±2.75 / 146.96 ms │ no change │
│ QQuery 57 │ 173.16 / 175.93 ±3.31 / 181.80 ms │ 174.32 / 176.20 ±1.50 / 177.89 ms │ no change │
│ QQuery 58 │ 290.11 / 300.42 ±8.69 / 311.42 ms │ 287.15 / 296.32 ±7.71 / 308.02 ms │ no change │
│ QQuery 59 │ 198.30 / 202.47 ±7.18 / 216.81 ms │ 197.95 / 198.67 ±0.51 / 199.53 ms │ no change │
│ QQuery 60 │ 144.38 / 145.31 ±0.90 / 146.57 ms │ 142.90 / 143.85 ±0.71 / 144.81 ms │ no change │
│ QQuery 61 │ 12.82 / 13.09 ±0.27 / 13.60 ms │ 13.26 / 13.50 ±0.24 / 13.85 ms │ no change │
│ QQuery 62 │ 872.21 / 939.53 ±62.70 / 1051.30 ms │ 886.69 / 929.12 ±30.04 / 972.26 ms │ no change │
│ QQuery 63 │ 101.16 / 103.63 ±2.10 / 106.70 ms │ 103.55 / 107.32 ±4.22 / 115.50 ms │ no change │
│ QQuery 64 │ 681.83 / 689.59 ±4.33 / 693.07 ms │ 681.37 / 689.37 ±4.11 / 692.91 ms │ no change │
│ QQuery 65 │ 250.45 / 253.15 ±2.79 / 258.41 ms │ 248.13 / 251.27 ±3.09 / 256.50 ms │ no change │
│ QQuery 66 │ 242.73 / 259.94 ±12.30 / 278.25 ms │ 242.64 / 252.34 ±6.69 / 259.90 ms │ no change │
│ QQuery 67 │ 305.72 / 315.77 ±8.79 / 329.56 ms │ 300.55 / 308.77 ±6.45 / 317.29 ms │ no change │
│ QQuery 68 │ 9.19 / 9.96 ±0.81 / 11.46 ms │ 10.16 / 11.10 ±0.87 / 12.36 ms │ 1.12x slower │
│ QQuery 69 │ 101.59 / 103.59 ±1.25 / 105.36 ms │ 102.41 / 104.64 ±2.12 / 107.28 ms │ no change │
│ QQuery 70 │ 347.27 / 359.09 ±6.85 / 366.87 ms │ 339.88 / 344.61 ±3.02 / 348.65 ms │ no change │
│ QQuery 71 │ 133.10 / 134.17 ±0.91 / 135.74 ms │ 135.56 / 137.20 ±2.03 / 141.10 ms │ no change │
│ QQuery 72 │ 618.54 / 628.77 ±6.17 / 637.40 ms │ 617.61 / 624.83 ±8.35 / 640.66 ms │ no change │
│ QQuery 73 │ 6.46 / 7.38 ±0.80 / 8.45 ms │ 7.31 / 8.83 ±1.66 / 11.07 ms │ 1.20x slower │
│ QQuery 74 │ 540.55 / 547.35 ±5.24 / 554.48 ms │ 547.34 / 555.56 ±5.96 / 565.20 ms │ no change │
│ QQuery 75 │ 275.89 / 278.30 ±1.60 / 280.38 ms │ 270.32 / 274.70 ±3.69 / 281.31 ms │ no change │
│ QQuery 76 │ 131.30 / 132.14 ±1.04 / 134.17 ms │ 131.69 / 133.69 ±1.29 / 135.56 ms │ no change │
│ QQuery 77 │ 186.41 / 188.10 ±1.32 / 189.67 ms │ 185.51 / 188.14 ±1.94 / 191.23 ms │ no change │
│ QQuery 78 │ 342.92 / 346.22 ±2.67 / 349.80 ms │ 332.70 / 337.99 ±5.22 / 347.70 ms │ no change │
│ QQuery 79 │ 232.07 / 234.00 ±1.88 / 236.72 ms │ 226.96 / 231.36 ±2.36 / 233.23 ms │ no change │
│ QQuery 80 │ 323.36 / 325.55 ±1.73 / 328.08 ms │ 320.79 / 323.66 ±2.26 / 326.36 ms │ no change │
│ QQuery 81 │ 26.29 / 27.61 ±0.94 / 28.91 ms │ 26.35 / 27.83 ±1.69 / 31.14 ms │ no change │
│ QQuery 82 │ 201.01 / 202.41 ±1.07 / 203.97 ms │ 199.79 / 202.78 ±2.65 / 206.42 ms │ no change │
│ QQuery 83 │ 37.68 / 38.85 ±0.79 / 40.00 ms │ 38.55 / 40.42 ±2.23 / 44.79 ms │ no change │
│ QQuery 84 │ 46.85 / 48.57 ±0.86 / 49.04 ms │ 48.55 / 49.82 ±0.93 / 50.90 ms │ no change │
│ QQuery 85 │ 146.62 / 148.74 ±1.41 / 150.69 ms │ 149.68 / 151.06 ±1.61 / 153.66 ms │ no change │
│ QQuery 86 │ 39.48 / 40.74 ±1.18 / 42.74 ms │ 39.56 / 40.28 ±0.54 / 41.06 ms │ no change │
│ QQuery 87 │ 87.48 / 90.38 ±2.60 / 95.11 ms │ 86.48 / 89.75 ±2.58 / 93.91 ms │ no change │
│ QQuery 88 │ 99.42 / 101.21 ±1.35 / 102.87 ms │ 100.29 / 101.53 ±0.80 / 102.73 ms │ no change │
│ QQuery 89 │ 117.33 / 119.27 ±1.64 / 122.23 ms │ 118.57 / 119.74 ±0.92 / 120.81 ms │ no change │
│ QQuery 90 │ 23.45 / 23.78 ±0.27 / 24.20 ms │ 23.45 / 24.37 ±0.58 / 25.10 ms │ no change │
│ QQuery 91 │ 63.36 / 64.61 ±0.84 / 65.63 ms │ 62.73 / 65.35 ±2.09 / 68.68 ms │ no change │
│ QQuery 92 │ 56.54 / 58.32 ±1.47 / 60.69 ms │ 56.24 / 57.14 ±0.70 / 58.34 ms │ no change │
│ QQuery 93 │ 187.44 / 188.89 ±1.48 / 191.72 ms │ 186.62 / 188.98 ±1.28 / 190.32 ms │ no change │
│ QQuery 94 │ 59.85 / 61.24 ±0.96 / 62.59 ms │ 62.66 / 63.61 ±0.77 / 64.82 ms │ no change │
│ QQuery 95 │ 127.85 / 129.23 ±1.76 / 132.69 ms │ 129.31 / 130.93 ±1.22 / 132.69 ms │ no change │
│ QQuery 96 │ 73.83 / 74.33 ±0.29 / 74.60 ms │ 73.04 / 75.09 ±1.35 / 77.07 ms │ no change │
│ QQuery 97 │ 126.91 / 128.53 ±1.41 / 130.91 ms │ 125.81 / 128.35 ±1.75 / 130.52 ms │ no change │
│ QQuery 98 │ 153.24 / 155.27 ±1.47 / 157.58 ms │ 154.06 / 156.71 ±2.12 / 160.03 ms │ no change │
│ QQuery 99 │ 10717.02 / 10765.75 ±27.57 / 10792.54 ms │ 10676.36 / 10727.46 ±39.92 / 10786.98 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31328.96ms │
│ Total Time (morsel-scheduler) │ 31189.04ms │
│ Average Time (HEAD) │ 316.45ms │
│ Average Time (morsel-scheduler) │ 315.04ms │
│ Queries Faster │ 2 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 95 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 157.0s |
| Peak memory | 5.1 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 259.1s |
| CPU sys | 17.5s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 156.3s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 258.5s |
| CPU sys | 16.6s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273807287
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.20 / 4.53 ±6.50 / 17.53 ms │ 1.25 / 4.56 ±6.47 / 17.49 ms │ no change │
│ QQuery 1 │ 14.99 / 15.56 ±0.50 / 16.50 ms │ 13.40 / 13.70 ±0.34 / 14.36 ms │ +1.14x faster │
│ QQuery 2 │ 44.24 / 44.74 ±0.42 / 45.36 ms │ 43.05 / 43.85 ±0.74 / 44.74 ms │ no change │
│ QQuery 3 │ 41.77 / 45.08 ±2.58 / 47.89 ms │ 41.38 / 45.52 ±3.00 / 48.75 ms │ no change │
│ QQuery 4 │ 292.56 / 297.03 ±3.78 / 302.62 ms │ 485.78 / 535.24 ±45.51 / 615.19 ms │ 1.80x slower │
│ QQuery 5 │ 338.06 / 339.98 ±1.95 / 343.68 ms │ 411.01 / 483.60 ±57.20 / 542.49 ms │ 1.42x slower │
│ QQuery 6 │ 5.56 / 6.34 ±0.68 / 7.44 ms │ 4.13 / 4.94 ±0.72 / 6.25 ms │ +1.28x faster │
│ QQuery 7 │ 17.05 / 17.93 ±1.08 / 20.01 ms │ 15.31 / 16.51 ±0.86 / 17.54 ms │ +1.09x faster │
│ QQuery 8 │ 404.77 / 420.57 ±11.19 / 434.66 ms │ 640.14 / 686.60 ±45.59 / 771.95 ms │ 1.63x slower │
│ QQuery 9 │ 649.79 / 665.67 ±10.78 / 678.64 ms │ 670.41 / 693.70 ±20.67 / 730.41 ms │ no change │
│ QQuery 10 │ 91.35 / 95.12 ±4.09 / 102.99 ms │ 94.95 / 106.99 ±11.44 / 125.49 ms │ 1.12x slower │
│ QQuery 11 │ 105.05 / 106.66 ±1.15 / 108.13 ms │ 111.20 / 118.41 ±7.35 / 130.36 ms │ 1.11x slower │
│ QQuery 12 │ 341.27 / 346.90 ±3.69 / 351.92 ms │ 422.89 / 467.12 ±46.70 / 536.44 ms │ 1.35x slower │
│ QQuery 13 │ 470.16 / 482.74 ±10.17 / 494.98 ms │ 917.44 / 947.66 ±17.93 / 970.67 ms │ 1.96x slower │
│ QQuery 14 │ 339.17 / 342.91 ±2.57 / 346.87 ms │ 456.82 / 502.43 ±35.88 / 567.13 ms │ 1.47x slower │
│ QQuery 15 │ 354.12 / 367.12 ±11.35 / 387.26 ms │ 575.20 / 635.47 ±47.13 / 699.66 ms │ 1.73x slower │
│ QQuery 16 │ 712.94 / 728.71 ±13.32 / 746.22 ms │ 1234.16 / 1288.81 ±50.59 / 1365.98 ms │ 1.77x slower │
│ QQuery 17 │ 704.85 / 711.01 ±8.72 / 728.04 ms │ 1196.72 / 1279.16 ±53.01 / 1360.79 ms │ 1.80x slower │
│ QQuery 18 │ 1354.19 / 1362.66 ±6.35 / 1369.44 ms │ 2617.12 / 2742.63 ±85.62 / 2840.00 ms │ 2.01x slower │
│ QQuery 19 │ 35.95 / 36.95 ±1.46 / 39.86 ms │ 33.44 / 35.62 ±1.80 / 38.60 ms │ no change │
│ QQuery 20 │ 706.61 / 719.82 ±13.79 / 737.90 ms │ 730.95 / 736.94 ±9.99 / 756.70 ms │ no change │
│ QQuery 21 │ 758.82 / 765.07 ±7.89 / 779.95 ms │ 770.46 / 778.55 ±5.38 / 785.98 ms │ no change │
│ QQuery 22 │ 1127.17 / 1135.72 ±6.67 / 1147.05 ms │ 1144.97 / 1151.22 ±4.77 / 1159.62 ms │ no change │
│ QQuery 23 │ 3083.79 / 3100.17 ±12.06 / 3120.93 ms │ 3111.80 / 3123.45 ±12.26 / 3145.55 ms │ no change │
│ QQuery 24 │ 100.11 / 103.35 ±3.05 / 108.16 ms │ 98.90 / 103.63 ±4.27 / 109.23 ms │ no change │
│ QQuery 25 │ 137.47 / 141.45 ±2.31 / 143.41 ms │ 137.81 / 143.20 ±3.88 / 149.41 ms │ no change │
│ QQuery 26 │ 99.34 / 102.88 ±1.88 / 104.43 ms │ 97.73 / 100.18 ±2.69 / 104.90 ms │ no change │
│ QQuery 27 │ 845.21 / 850.48 ±5.95 / 861.12 ms │ 850.50 / 860.83 ±9.89 / 877.72 ms │ no change │
│ QQuery 28 │ 3243.42 / 3268.75 ±14.04 / 3285.59 ms │ 3285.05 / 3311.42 ±19.44 / 3337.75 ms │ no change │
│ QQuery 29 │ 49.47 / 54.71 ±4.76 / 63.61 ms │ 49.06 / 52.83 ±3.50 / 59.19 ms │ no change │
│ QQuery 30 │ 362.56 / 367.93 ±5.10 / 374.25 ms │ 492.82 / 513.51 ±21.53 / 551.08 ms │ 1.40x slower │
│ QQuery 31 │ 369.22 / 381.27 ±9.09 / 392.38 ms │ 582.08 / 601.36 ±16.14 / 622.38 ms │ 1.58x slower │
│ QQuery 32 │ 1027.16 / 1053.53 ±23.96 / 1094.77 ms │ 3169.73 / 3229.68 ±60.51 / 3304.26 ms │ 3.07x slower │
│ QQuery 33 │ 1485.61 / 1493.73 ±5.99 / 1503.21 ms │ 1940.41 / 2066.80 ±88.52 / 2169.34 ms │ 1.38x slower │
│ QQuery 34 │ 1488.36 / 1512.00 ±20.91 / 1550.04 ms │ 1936.38 / 2074.08 ±105.06 / 2201.30 ms │ 1.37x slower │
│ QQuery 35 │ 395.90 / 401.89 ±5.33 / 411.56 ms │ 489.83 / 556.71 ±67.08 / 639.86 ms │ 1.39x slower │
│ QQuery 36 │ 117.94 / 122.60 ±3.18 / 125.34 ms │ 112.59 / 115.51 ±3.03 / 121.35 ms │ +1.06x faster │
│ QQuery 37 │ 47.84 / 49.40 ±0.94 / 50.48 ms │ 45.52 / 46.25 ±0.62 / 47.36 ms │ +1.07x faster │
│ QQuery 38 │ 75.63 / 77.79 ±1.62 / 79.60 ms │ 73.61 / 74.73 ±0.86 / 76.25 ms │ no change │
│ QQuery 39 │ 205.57 / 217.52 ±6.60 / 224.16 ms │ 204.33 / 210.53 ±4.60 / 216.97 ms │ no change │
│ QQuery 40 │ 23.66 / 25.15 ±0.92 / 26.37 ms │ 20.79 / 23.99 ±3.21 / 28.77 ms │ no change │
│ QQuery 41 │ 19.94 / 22.47 ±2.97 / 28.27 ms │ 18.53 / 20.09 ±1.02 / 21.72 ms │ +1.12x faster │
│ QQuery 42 │ 19.69 / 20.88 ±1.04 / 22.66 ms │ 18.37 / 18.85 ±0.40 / 19.35 ms │ +1.11x faster │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22426.72ms │
│ Total Time (morsel-scheduler) │ 30566.83ms │
│ Average Time (HEAD) │ 521.55ms │
│ Average Time (morsel-scheduler) │ 710.86ms │
│ Queries Faster │ 7 │
│ Queries Slower │ 18 │
│ Queries with No Change │ 18 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.3s |
| Peak memory | 39.0 GiB |
| Avg memory | 33.0 GiB |
| CPU user | 1075.1s |
| CPU sys | 80.8s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 154.2s |
| Peak memory | 39.0 GiB |
| Avg memory | 27.3 GiB |
| CPU user | 1079.2s |
| CPU sys | 98.0s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273831617
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273827001-1475-q6697 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (99e872a19bade26aa57b201022f8a3f9b86692a8) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..99e872a19bade26aa57b201022f8a3f9b86692a8) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273870485
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273866779-1479-zg7n7 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (f2cf29a3af1abe7e0996013f65a5becef70cad25) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..f2cf29a3af1abe7e0996013f65a5becef70cad25) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275682067
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275674921-1515-mqcfm 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273862162
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.41 / 6.85 ±0.76 / 8.36 ms │ 6.63 / 7.08 ±0.77 / 8.61 ms │ no change │
│ QQuery 2 │ 144.69 / 145.46 ±0.68 / 146.63 ms │ 145.67 / 146.68 ±1.08 / 148.70 ms │ no change │
│ QQuery 3 │ 112.97 / 113.38 ±0.37 / 114.03 ms │ 112.95 / 114.35 ±0.78 / 115.23 ms │ no change │
│ QQuery 4 │ 1266.43 / 1305.68 ±26.44 / 1348.94 ms │ 1268.89 / 1297.51 ±15.35 / 1315.12 ms │ no change │
│ QQuery 5 │ 172.05 / 173.53 ±1.13 / 175.31 ms │ 171.86 / 173.77 ±1.11 / 175.31 ms │ no change │
│ QQuery 6 │ 818.28 / 852.64 ±23.83 / 882.61 ms │ 831.62 / 860.38 ±17.69 / 877.24 ms │ no change │
│ QQuery 7 │ 339.39 / 341.81 ±1.49 / 343.83 ms │ 339.20 / 342.63 ±1.84 / 344.30 ms │ no change │
│ QQuery 8 │ 115.84 / 117.09 ±0.90 / 118.58 ms │ 117.35 / 118.18 ±0.56 / 119.05 ms │ no change │
│ QQuery 9 │ 105.18 / 110.43 ±8.42 / 127.20 ms │ 101.36 / 105.84 ±2.28 / 107.51 ms │ no change │
│ QQuery 10 │ 103.61 / 105.16 ±1.16 / 107.06 ms │ 105.23 / 106.31 ±0.86 / 107.40 ms │ no change │
│ QQuery 11 │ 898.75 / 914.40 ±12.04 / 930.33 ms │ 897.53 / 910.71 ±8.45 / 921.07 ms │ no change │
│ QQuery 12 │ 46.63 / 47.80 ±0.89 / 49.19 ms │ 44.23 / 46.68 ±1.54 / 48.99 ms │ no change │
│ QQuery 13 │ 396.07 / 402.24 ±3.32 / 404.78 ms │ 396.77 / 400.16 ±2.18 / 402.79 ms │ no change │
│ QQuery 14 │ 992.23 / 1002.95 ±7.68 / 1013.82 ms │ 1003.48 / 1009.30 ±6.15 / 1021.18 ms │ no change │
│ QQuery 15 │ 15.26 / 16.53 ±0.91 / 17.93 ms │ 15.44 / 16.98 ±1.53 / 19.15 ms │ no change │
│ QQuery 16 │ 7.15 / 7.86 ±0.63 / 8.69 ms │ 7.01 / 7.18 ±0.20 / 7.52 ms │ +1.10x faster │
│ QQuery 17 │ 225.71 / 228.11 ±1.67 / 230.96 ms │ 228.92 / 230.58 ±1.34 / 232.60 ms │ no change │
│ QQuery 18 │ 126.13 / 127.28 ±0.87 / 128.35 ms │ 126.35 / 128.05 ±1.13 / 129.57 ms │ no change │
│ QQuery 19 │ 154.26 / 155.69 ±1.52 / 158.61 ms │ 155.74 / 157.25 ±1.64 / 160.40 ms │ no change │
│ QQuery 20 │ 14.24 / 14.93 ±0.93 / 16.76 ms │ 13.65 / 14.08 ±0.39 / 14.54 ms │ +1.06x faster │
│ QQuery 21 │ 18.79 / 19.32 ±0.30 / 19.66 ms │ 19.61 / 20.15 ±0.35 / 20.51 ms │ no change │
│ QQuery 22 │ 471.04 / 477.18 ±4.11 / 483.01 ms │ 474.30 / 478.71 ±4.43 / 486.94 ms │ no change │
│ QQuery 23 │ 865.29 / 874.63 ±7.00 / 884.67 ms │ 842.81 / 846.97 ±3.13 / 850.77 ms │ no change │
│ QQuery 24 │ 378.16 / 381.23 ±3.28 / 387.20 ms │ 379.60 / 382.98 ±2.27 / 386.36 ms │ no change │
│ QQuery 25 │ 338.64 / 340.05 ±1.35 / 342.58 ms │ 340.21 / 342.67 ±1.40 / 344.24 ms │ no change │
│ QQuery 26 │ 78.95 / 81.73 ±1.93 / 84.31 ms │ 79.57 / 80.52 ±0.66 / 81.28 ms │ no change │
│ QQuery 27 │ 6.87 / 7.39 ±0.50 / 8.31 ms │ 7.00 / 7.60 ±0.63 / 8.47 ms │ no change │
│ QQuery 28 │ 147.78 / 148.88 ±0.88 / 149.65 ms │ 148.51 / 150.11 ±1.36 / 152.21 ms │ no change │
│ QQuery 29 │ 278.33 / 280.41 ±1.66 / 282.96 ms │ 282.78 / 283.79 ±0.84 / 284.87 ms │ no change │
│ QQuery 30 │ 42.79 / 44.55 ±1.62 / 46.87 ms │ 44.24 / 44.76 ±0.58 / 45.82 ms │ no change │
│ QQuery 31 │ 167.19 / 171.02 ±2.26 / 173.75 ms │ 170.57 / 171.86 ±1.12 / 173.79 ms │ no change │
│ QQuery 32 │ 56.41 / 58.57 ±1.23 / 60.11 ms │ 56.30 / 57.92 ±1.07 / 59.37 ms │ no change │
│ QQuery 33 │ 139.92 / 140.83 ±0.50 / 141.27 ms │ 139.25 / 141.41 ±2.43 / 145.65 ms │ no change │
│ QQuery 34 │ 6.77 / 7.03 ±0.41 / 7.84 ms │ 6.82 / 7.10 ±0.27 / 7.60 ms │ no change │
│ QQuery 35 │ 103.53 / 107.03 ±2.06 / 109.24 ms │ 107.97 / 108.64 ±0.61 / 109.60 ms │ no change │
│ QQuery 36 │ 6.43 / 6.68 ±0.23 / 7.07 ms │ 6.45 / 6.68 ±0.20 / 7.02 ms │ no change │
│ QQuery 37 │ 8.19 / 8.97 ±0.82 / 10.50 ms │ 8.17 / 8.61 ±0.64 / 9.87 ms │ no change │
│ QQuery 38 │ 86.30 / 88.07 ±2.50 / 93.03 ms │ 82.55 / 88.07 ±3.86 / 94.29 ms │ no change │
│ QQuery 39 │ 123.20 / 124.83 ±1.18 / 126.36 ms │ 124.55 / 127.65 ±2.53 / 131.69 ms │ no change │
│ QQuery 40 │ 107.45 / 114.41 ±5.94 / 124.96 ms │ 110.65 / 115.77 ±5.66 / 126.76 ms │ no change │
│ QQuery 41 │ 14.58 / 15.17 ±0.41 / 15.87 ms │ 14.02 / 14.98 ±0.60 / 15.85 ms │ no change │
│ QQuery 42 │ 107.69 / 109.50 ±1.33 / 110.90 ms │ 108.10 / 109.10 ±0.75 / 110.42 ms │ no change │
│ QQuery 43 │ 5.85 / 6.01 ±0.18 / 6.35 ms │ 5.85 / 6.07 ±0.17 / 6.36 ms │ no change │
│ QQuery 44 │ 11.54 / 12.28 ±0.68 / 13.12 ms │ 11.51 / 12.08 ±0.35 / 12.59 ms │ no change │
│ QQuery 45 │ 49.89 / 51.04 ±0.97 / 52.55 ms │ 50.59 / 51.66 ±0.77 / 52.54 ms │ no change │
│ QQuery 46 │ 8.03 / 8.43 ±0.24 / 8.73 ms │ 8.32 / 8.54 ±0.18 / 8.82 ms │ no change │
│ QQuery 47 │ 687.90 / 694.90 ±5.09 / 701.18 ms │ 691.56 / 700.45 ±7.84 / 714.21 ms │ no change │
│ QQuery 48 │ 286.32 / 292.03 ±4.26 / 297.40 ms │ 286.53 / 290.23 ±4.01 / 296.92 ms │ no change │
│ QQuery 49 │ 250.82 / 254.39 ±2.71 / 257.61 ms │ 251.80 / 253.90 ±1.49 / 256.10 ms │ no change │
│ QQuery 50 │ 217.72 / 221.56 ±2.61 / 224.74 ms │ 220.81 / 227.45 ±3.67 / 230.67 ms │ no change │
│ QQuery 51 │ 179.00 / 179.52 ±0.48 / 180.14 ms │ 179.71 / 181.90 ±2.52 / 185.41 ms │ no change │
│ QQuery 52 │ 106.77 / 108.07 ±2.23 / 112.51 ms │ 106.68 / 107.75 ±1.41 / 110.48 ms │ no change │
│ QQuery 53 │ 101.35 / 102.54 ±0.69 / 103.17 ms │ 101.32 / 102.85 ±1.30 / 105.02 ms │ no change │
│ QQuery 54 │ 145.67 / 147.25 ±1.19 / 149.12 ms │ 143.76 / 146.51 ±3.61 / 153.33 ms │ no change │
│ QQuery 55 │ 106.62 / 107.41 ±1.07 / 109.51 ms │ 105.78 / 107.55 ±1.07 / 109.06 ms │ no change │
│ QQuery 56 │ 140.05 / 141.96 ±1.52 / 143.87 ms │ 140.59 / 141.21 ±0.56 / 142.08 ms │ no change │
│ QQuery 57 │ 173.57 / 174.71 ±1.16 / 176.92 ms │ 173.75 / 176.08 ±2.22 / 179.75 ms │ no change │
│ QQuery 58 │ 284.64 / 295.81 ±11.35 / 312.76 ms │ 285.51 / 295.24 ±6.25 / 305.11 ms │ no change │
│ QQuery 59 │ 194.71 / 196.21 ±1.26 / 198.03 ms │ 197.70 / 198.69 ±1.07 / 200.73 ms │ no change │
│ QQuery 60 │ 142.06 / 143.89 ±1.29 / 145.40 ms │ 142.11 / 144.12 ±1.13 / 145.23 ms │ no change │
│ QQuery 61 │ 12.75 / 13.11 ±0.33 / 13.61 ms │ 13.12 / 13.97 ±0.59 / 14.66 ms │ 1.07x slower │
│ QQuery 62 │ 893.93 / 964.57 ±41.97 / 1015.59 ms │ 893.97 / 946.72 ±41.57 / 1018.15 ms │ no change │
│ QQuery 63 │ 103.35 / 106.16 ±2.06 / 109.14 ms │ 105.96 / 107.32 ±1.19 / 108.97 ms │ no change │
│ QQuery 64 │ 681.46 / 684.67 ±3.77 / 691.97 ms │ 681.77 / 687.59 ±5.84 / 695.43 ms │ no change │
│ QQuery 65 │ 252.17 / 254.70 ±1.92 / 258.08 ms │ 247.43 / 251.62 ±2.28 / 253.96 ms │ no change │
│ QQuery 66 │ 247.78 / 256.63 ±6.31 / 267.48 ms │ 227.62 / 243.86 ±11.90 / 263.80 ms │ no change │
│ QQuery 67 │ 308.33 / 316.94 ±5.42 / 322.40 ms │ 308.34 / 314.58 ±8.42 / 331.12 ms │ no change │
│ QQuery 68 │ 8.59 / 10.33 ±0.97 / 11.56 ms │ 9.16 / 11.07 ±1.60 / 13.48 ms │ 1.07x slower │
│ QQuery 69 │ 103.05 / 104.47 ±1.19 / 106.39 ms │ 101.79 / 104.35 ±2.26 / 108.53 ms │ no change │
│ QQuery 70 │ 345.59 / 352.76 ±11.97 / 376.59 ms │ 327.86 / 341.86 ±12.96 / 365.50 ms │ no change │
│ QQuery 71 │ 135.30 / 137.97 ±1.65 / 140.32 ms │ 134.14 / 135.91 ±1.32 / 137.59 ms │ no change │
│ QQuery 72 │ 614.92 / 622.71 ±4.60 / 627.10 ms │ 621.49 / 623.19 ±1.24 / 624.90 ms │ no change │
│ QQuery 73 │ 6.41 / 7.41 ±1.16 / 9.59 ms │ 7.16 / 8.40 ±1.01 / 9.63 ms │ 1.13x slower │
│ QQuery 74 │ 560.88 / 564.45 ±4.09 / 572.41 ms │ 561.63 / 570.71 ±7.80 / 581.83 ms │ no change │
│ QQuery 75 │ 274.06 / 276.45 ±1.77 / 278.63 ms │ 273.66 / 276.00 ±1.80 / 278.65 ms │ no change │
│ QQuery 76 │ 130.94 / 132.18 ±0.87 / 133.62 ms │ 133.10 / 134.13 ±0.54 / 134.54 ms │ no change │
│ QQuery 77 │ 187.98 / 189.75 ±0.94 / 190.61 ms │ 187.47 / 189.73 ±1.50 / 191.71 ms │ no change │
│ QQuery 78 │ 334.52 / 340.37 ±4.11 / 346.04 ms │ 343.34 / 345.39 ±1.59 / 347.76 ms │ no change │
│ QQuery 79 │ 231.26 / 235.22 ±3.19 / 241.04 ms │ 228.75 / 232.53 ±2.90 / 237.35 ms │ no change │
│ QQuery 80 │ 322.50 / 324.08 ±1.74 / 326.51 ms │ 320.53 / 325.25 ±3.43 / 330.11 ms │ no change │
│ QQuery 81 │ 26.44 / 27.41 ±0.89 / 29.01 ms │ 26.23 / 27.11 ±0.60 / 28.09 ms │ no change │
│ QQuery 82 │ 197.96 / 201.66 ±2.29 / 205.00 ms │ 197.19 / 201.05 ±3.27 / 204.94 ms │ no change │
│ QQuery 83 │ 38.56 / 39.74 ±1.07 / 41.76 ms │ 39.38 / 40.36 ±1.03 / 41.74 ms │ no change │
│ QQuery 84 │ 48.74 / 49.09 ±0.33 / 49.66 ms │ 49.22 / 50.66 ±1.18 / 52.22 ms │ no change │
│ QQuery 85 │ 146.16 / 147.61 ±0.87 / 148.75 ms │ 146.65 / 148.56 ±1.72 / 151.54 ms │ no change │
│ QQuery 86 │ 39.25 / 39.90 ±0.58 / 40.66 ms │ 38.81 / 39.63 ±0.43 / 39.99 ms │ no change │
│ QQuery 87 │ 85.39 / 86.33 ±1.20 / 88.70 ms │ 84.98 / 89.10 ±3.28 / 94.96 ms │ no change │
│ QQuery 88 │ 99.14 / 100.70 ±1.48 / 103.31 ms │ 99.24 / 100.53 ±1.31 / 102.55 ms │ no change │
│ QQuery 89 │ 117.58 / 118.56 ±0.76 / 119.45 ms │ 116.95 / 119.59 ±1.64 / 121.97 ms │ no change │
│ QQuery 90 │ 22.72 / 23.51 ±0.67 / 24.49 ms │ 23.17 / 23.83 ±0.42 / 24.43 ms │ no change │
│ QQuery 91 │ 61.82 / 63.41 ±1.31 / 65.32 ms │ 61.20 / 63.48 ±1.81 / 65.19 ms │ no change │
│ QQuery 92 │ 57.01 / 57.11 ±0.11 / 57.33 ms │ 56.74 / 57.96 ±0.76 / 59.03 ms │ no change │
│ QQuery 93 │ 184.73 / 186.50 ±1.09 / 187.59 ms │ 186.75 / 187.81 ±0.72 / 188.66 ms │ no change │
│ QQuery 94 │ 60.73 / 61.91 ±0.93 / 63.47 ms │ 61.77 / 63.31 ±1.44 / 65.77 ms │ no change │
│ QQuery 95 │ 128.91 / 129.90 ±0.59 / 130.67 ms │ 129.02 / 130.84 ±1.88 / 134.05 ms │ no change │
│ QQuery 96 │ 73.08 / 73.82 ±0.55 / 74.64 ms │ 72.87 / 73.95 ±1.24 / 76.26 ms │ no change │
│ QQuery 97 │ 121.99 / 125.87 ±2.42 / 128.45 ms │ 125.81 / 126.74 ±0.97 / 128.57 ms │ no change │
│ QQuery 98 │ 152.13 / 152.83 ±0.47 / 153.58 ms │ 153.19 / 156.10 ±2.66 / 160.85 ms │ no change │
│ QQuery 99 │ 10681.59 / 10727.99 ±26.94 / 10760.76 ms │ 10708.07 / 10729.98 ±11.98 / 10744.72 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31244.08ms │
│ Total Time (morsel-scheduler) │ 31236.80ms │
│ Average Time (HEAD) │ 315.60ms │
│ Average Time (morsel-scheduler) │ 315.52ms │
│ Queries Faster │ 2 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 94 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 156.5s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 258.2s |
| CPU sys | 17.0s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 156.5s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.6 GiB |
| CPU user | 257.8s |
| CPU sys | 16.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273871433
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273866779-1478-57xwr 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (f2cf29a3af1abe7e0996013f65a5becef70cad25) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..f2cf29a3af1abe7e0996013f65a5becef70cad25) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273904621
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273898806-1482-dqjrs 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (3d4cb822d349992939910578c0645da129345fea) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..3d4cb822d349992939910578c0645da129345fea) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273929339
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.56 / 7.04 ±0.76 / 8.55 ms │ 6.73 / 7.22 ±0.82 / 8.86 ms │ no change │
│ QQuery 2 │ 145.24 / 147.06 ±1.35 / 149.04 ms │ 146.71 / 147.83 ±0.74 / 148.65 ms │ no change │
│ QQuery 3 │ 114.00 / 114.54 ±0.36 / 115.08 ms │ 114.06 / 114.47 ±0.41 / 115.11 ms │ no change │
│ QQuery 4 │ 1312.29 / 1345.98 ±18.67 / 1365.62 ms │ 1311.29 / 1340.89 ±16.85 / 1362.66 ms │ no change │
│ QQuery 5 │ 173.09 / 175.97 ±1.76 / 177.98 ms │ 172.15 / 174.57 ±2.03 / 177.12 ms │ no change │
│ QQuery 6 │ 853.61 / 871.68 ±14.35 / 895.50 ms │ 839.28 / 868.07 ±23.45 / 905.39 ms │ no change │
│ QQuery 7 │ 340.50 / 342.46 ±1.84 / 345.27 ms │ 341.82 / 344.64 ±2.11 / 347.39 ms │ no change │
│ QQuery 8 │ 117.21 / 117.60 ±0.48 / 118.52 ms │ 115.91 / 117.01 ±0.60 / 117.58 ms │ no change │
│ QQuery 9 │ 101.18 / 103.87 ±2.80 / 107.89 ms │ 100.90 / 105.70 ±2.49 / 107.82 ms │ no change │
│ QQuery 10 │ 105.69 / 106.52 ±0.85 / 107.80 ms │ 106.62 / 108.28 ±0.97 / 109.50 ms │ no change │
│ QQuery 11 │ 935.60 / 950.00 ±13.27 / 974.70 ms │ 927.99 / 953.31 ±15.26 / 972.85 ms │ no change │
│ QQuery 12 │ 45.15 / 47.37 ±1.90 / 50.37 ms │ 46.35 / 47.49 ±1.37 / 50.13 ms │ no change │
│ QQuery 13 │ 401.36 / 404.58 ±3.19 / 410.27 ms │ 400.83 / 403.52 ±1.85 / 406.18 ms │ no change │
│ QQuery 14 │ 1006.31 / 1011.62 ±4.50 / 1019.90 ms │ 995.69 / 1006.59 ±7.27 / 1014.58 ms │ no change │
│ QQuery 15 │ 16.13 / 17.79 ±2.06 / 21.76 ms │ 15.46 / 16.47 ±0.63 / 17.43 ms │ +1.08x faster │
│ QQuery 16 │ 7.09 / 7.71 ±0.65 / 8.97 ms │ 7.20 / 7.81 ±0.73 / 9.18 ms │ no change │
│ QQuery 17 │ 227.98 / 231.32 ±1.83 / 233.24 ms │ 228.96 / 231.24 ±1.66 / 233.27 ms │ no change │
│ QQuery 18 │ 127.04 / 127.87 ±0.73 / 128.96 ms │ 128.41 / 129.76 ±0.99 / 131.49 ms │ no change │
│ QQuery 19 │ 155.42 / 157.61 ±2.62 / 162.59 ms │ 154.95 / 156.85 ±0.98 / 157.71 ms │ no change │
│ QQuery 20 │ 13.91 / 14.78 ±0.57 / 15.66 ms │ 14.60 / 14.79 ±0.10 / 14.88 ms │ no change │
│ QQuery 21 │ 19.42 / 20.01 ±0.49 / 20.76 ms │ 19.58 / 20.38 ±0.62 / 21.27 ms │ no change │
│ QQuery 22 │ 480.48 / 483.66 ±2.21 / 486.56 ms │ 481.67 / 485.32 ±1.89 / 486.98 ms │ no change │
│ QQuery 23 │ 876.84 / 880.71 ±3.09 / 883.88 ms │ 876.93 / 889.84 ±10.32 / 904.93 ms │ no change │
│ QQuery 24 │ 382.66 / 385.31 ±3.28 / 391.71 ms │ 381.81 / 385.90 ±3.02 / 389.91 ms │ no change │
│ QQuery 25 │ 340.23 / 343.60 ±2.69 / 348.17 ms │ 339.26 / 342.56 ±3.04 / 347.48 ms │ no change │
│ QQuery 26 │ 81.48 / 83.61 ±1.58 / 86.11 ms │ 81.08 / 84.72 ±2.88 / 89.70 ms │ no change │
│ QQuery 27 │ 7.03 / 7.46 ±0.60 / 8.61 ms │ 7.05 / 7.29 ±0.15 / 7.50 ms │ no change │
│ QQuery 28 │ 149.69 / 151.15 ±1.28 / 153.02 ms │ 148.25 / 149.14 ±0.69 / 149.96 ms │ no change │
│ QQuery 29 │ 282.64 / 286.46 ±2.99 / 290.50 ms │ 282.85 / 284.12 ±1.59 / 287.19 ms │ no change │
│ QQuery 30 │ 44.64 / 45.65 ±0.57 / 46.36 ms │ 43.89 / 45.86 ±1.49 / 47.50 ms │ no change │
│ QQuery 31 │ 170.64 / 174.05 ±2.99 / 177.84 ms │ 170.24 / 171.53 ±0.73 / 172.50 ms │ no change │
│ QQuery 32 │ 57.37 / 67.36 ±17.06 / 101.29 ms │ 56.54 / 57.27 ±0.68 / 58.30 ms │ +1.18x faster │
│ QQuery 33 │ 142.36 / 144.87 ±2.82 / 150.07 ms │ 140.02 / 142.00 ±2.28 / 145.49 ms │ no change │
│ QQuery 34 │ 6.96 / 7.17 ±0.25 / 7.63 ms │ 7.09 / 7.27 ±0.22 / 7.70 ms │ no change │
│ QQuery 35 │ 107.98 / 109.47 ±0.93 / 110.87 ms │ 107.89 / 109.10 ±0.66 / 109.76 ms │ no change │
│ QQuery 36 │ 6.70 / 7.29 ±0.96 / 9.20 ms │ 6.64 / 6.85 ±0.17 / 7.04 ms │ +1.06x faster │
│ QQuery 37 │ 8.52 / 8.94 ±0.25 / 9.28 ms │ 8.01 / 9.15 ±0.96 / 10.79 ms │ no change │
│ QQuery 38 │ 84.47 / 87.92 ±3.68 / 94.85 ms │ 80.19 / 85.33 ±3.86 / 92.18 ms │ no change │
│ QQuery 39 │ 124.16 / 128.13 ±3.38 / 133.43 ms │ 124.90 / 126.55 ±1.90 / 130.22 ms │ no change │
│ QQuery 40 │ 107.72 / 114.88 ±4.65 / 122.38 ms │ 108.68 / 114.87 ±5.83 / 125.56 ms │ no change │
│ QQuery 41 │ 14.24 / 15.16 ±0.95 / 16.89 ms │ 14.30 / 15.85 ±1.28 / 17.99 ms │ no change │
│ QQuery 42 │ 107.56 / 110.93 ±2.67 / 115.53 ms │ 109.18 / 109.99 ±0.54 / 110.69 ms │ no change │
│ QQuery 43 │ 5.80 / 5.99 ±0.21 / 6.39 ms │ 5.83 / 6.01 ±0.16 / 6.29 ms │ no change │
│ QQuery 44 │ 11.59 / 11.71 ±0.11 / 11.89 ms │ 11.51 / 11.99 ±0.56 / 13.09 ms │ no change │
│ QQuery 45 │ 49.89 / 50.90 ±1.05 / 52.72 ms │ 50.72 / 51.32 ±0.56 / 52.20 ms │ no change │
│ QQuery 46 │ 8.21 / 8.41 ±0.17 / 8.67 ms │ 8.30 / 8.93 ±0.48 / 9.71 ms │ 1.06x slower │
│ QQuery 47 │ 714.84 / 722.37 ±5.67 / 731.64 ms │ 710.75 / 721.56 ±7.89 / 731.94 ms │ no change │
│ QQuery 48 │ 288.88 / 294.61 ±3.24 / 297.90 ms │ 284.96 / 291.32 ±5.79 / 298.49 ms │ no change │
│ QQuery 49 │ 253.29 / 254.78 ±1.41 / 257.19 ms │ 254.85 / 256.23 ±1.06 / 257.58 ms │ no change │
│ QQuery 50 │ 220.63 / 230.31 ±5.20 / 234.50 ms │ 227.92 / 230.35 ±1.45 / 232.02 ms │ no change │
│ QQuery 51 │ 177.18 / 181.03 ±2.96 / 186.17 ms │ 180.01 / 182.39 ±2.32 / 186.32 ms │ no change │
│ QQuery 52 │ 108.27 / 109.15 ±0.48 / 109.64 ms │ 106.67 / 108.02 ±1.22 / 110.07 ms │ no change │
│ QQuery 53 │ 102.37 / 103.07 ±0.58 / 104.08 ms │ 102.00 / 102.65 ±0.56 / 103.34 ms │ no change │
│ QQuery 54 │ 144.54 / 147.63 ±1.74 / 149.45 ms │ 145.07 / 146.77 ±1.05 / 147.98 ms │ no change │
│ QQuery 55 │ 107.90 / 108.78 ±1.01 / 110.53 ms │ 107.75 / 109.01 ±1.65 / 112.15 ms │ no change │
│ QQuery 56 │ 140.80 / 142.16 ±1.67 / 145.39 ms │ 140.88 / 142.15 ±1.40 / 144.18 ms │ no change │
│ QQuery 57 │ 172.63 / 175.98 ±2.01 / 178.55 ms │ 171.85 / 176.59 ±2.86 / 179.58 ms │ no change │
│ QQuery 58 │ 287.29 / 303.99 ±10.61 / 316.35 ms │ 285.95 / 293.32 ±4.43 / 298.99 ms │ no change │
│ QQuery 59 │ 196.61 / 198.56 ±1.55 / 200.82 ms │ 197.90 / 200.80 ±1.87 / 203.09 ms │ no change │
│ QQuery 60 │ 145.99 / 146.71 ±0.84 / 147.76 ms │ 143.25 / 144.95 ±1.18 / 146.81 ms │ no change │
│ QQuery 61 │ 12.98 / 13.32 ±0.21 / 13.59 ms │ 13.15 / 13.53 ±0.66 / 14.85 ms │ no change │
│ QQuery 62 │ 897.68 / 949.92 ±45.73 / 1022.12 ms │ 911.44 / 944.14 ±29.29 / 988.30 ms │ no change │
│ QQuery 63 │ 103.90 / 105.32 ±1.36 / 107.04 ms │ 103.67 / 106.32 ±2.99 / 111.82 ms │ no change │
│ QQuery 64 │ 688.27 / 691.65 ±2.27 / 693.59 ms │ 687.74 / 692.69 ±3.78 / 698.01 ms │ no change │
│ QQuery 65 │ 256.22 / 259.53 ±2.10 / 262.18 ms │ 250.93 / 255.49 ±2.65 / 259.18 ms │ no change │
│ QQuery 66 │ 237.97 / 256.30 ±11.76 / 272.03 ms │ 254.82 / 261.07 ±6.85 / 272.44 ms │ no change │
│ QQuery 67 │ 303.25 / 318.11 ±10.01 / 334.50 ms │ 310.90 / 320.42 ±7.53 / 330.86 ms │ no change │
│ QQuery 68 │ 9.58 / 10.20 ±0.46 / 10.74 ms │ 9.73 / 11.27 ±1.52 / 14.08 ms │ 1.10x slower │
│ QQuery 69 │ 103.42 / 105.43 ±1.37 / 107.12 ms │ 101.11 / 103.71 ±1.97 / 106.08 ms │ no change │
│ QQuery 70 │ 323.58 / 343.25 ±9.97 / 351.08 ms │ 343.49 / 352.57 ±11.13 / 373.80 ms │ no change │
│ QQuery 71 │ 134.55 / 136.05 ±1.17 / 137.97 ms │ 136.17 / 137.78 ±1.43 / 140.34 ms │ no change │
│ QQuery 72 │ 619.57 / 636.12 ±10.35 / 650.01 ms │ 613.75 / 624.06 ±6.65 / 632.01 ms │ no change │
│ QQuery 73 │ 7.58 / 7.96 ±0.27 / 8.41 ms │ 6.52 / 7.74 ±1.34 / 10.16 ms │ no change │
│ QQuery 74 │ 596.67 / 603.15 ±4.97 / 611.72 ms │ 593.36 / 596.53 ±2.43 / 599.55 ms │ no change │
│ QQuery 75 │ 276.59 / 278.15 ±1.21 / 280.24 ms │ 275.97 / 276.79 ±0.74 / 278.10 ms │ no change │
│ QQuery 76 │ 131.66 / 133.28 ±1.18 / 135.20 ms │ 132.75 / 134.14 ±1.18 / 135.94 ms │ no change │
│ QQuery 77 │ 188.82 / 190.78 ±1.56 / 193.33 ms │ 189.13 / 190.73 ±1.52 / 193.42 ms │ no change │
│ QQuery 78 │ 340.19 / 343.83 ±4.44 / 351.86 ms │ 339.58 / 344.83 ±2.79 / 347.40 ms │ no change │
│ QQuery 79 │ 233.00 / 235.26 ±1.61 / 237.95 ms │ 234.95 / 236.23 ±1.08 / 238.11 ms │ no change │
│ QQuery 80 │ 323.52 / 325.64 ±1.81 / 327.93 ms │ 321.27 / 323.13 ±1.91 / 326.65 ms │ no change │
│ QQuery 81 │ 25.97 / 27.05 ±1.15 / 29.25 ms │ 26.63 / 27.77 ±1.48 / 30.51 ms │ no change │
│ QQuery 82 │ 200.83 / 202.20 ±1.48 / 204.55 ms │ 195.81 / 199.50 ±3.06 / 203.22 ms │ no change │
│ QQuery 83 │ 38.77 / 39.34 ±0.94 / 41.20 ms │ 38.95 / 39.81 ±1.16 / 42.12 ms │ no change │
│ QQuery 84 │ 48.74 / 50.03 ±1.27 / 52.46 ms │ 47.32 / 48.43 ±1.01 / 50.26 ms │ no change │
│ QQuery 85 │ 147.32 / 150.79 ±4.77 / 160.22 ms │ 150.00 / 150.45 ±0.47 / 151.31 ms │ no change │
│ QQuery 86 │ 38.44 / 39.89 ±1.38 / 42.46 ms │ 39.03 / 41.18 ±1.40 / 42.77 ms │ no change │
│ QQuery 87 │ 85.87 / 89.17 ±3.17 / 93.21 ms │ 85.25 / 90.04 ±4.44 / 98.42 ms │ no change │
│ QQuery 88 │ 99.62 / 100.72 ±1.04 / 102.69 ms │ 99.84 / 105.47 ±9.41 / 124.21 ms │ no change │
│ QQuery 89 │ 117.80 / 119.87 ±1.28 / 121.22 ms │ 118.29 / 119.25 ±0.82 / 120.67 ms │ no change │
│ QQuery 90 │ 24.08 / 24.60 ±0.50 / 25.46 ms │ 23.97 / 24.57 ±0.45 / 25.14 ms │ no change │
│ QQuery 91 │ 63.65 / 64.45 ±0.80 / 65.98 ms │ 64.15 / 64.76 ±0.56 / 65.51 ms │ no change │
│ QQuery 92 │ 57.73 / 59.00 ±1.26 / 61.32 ms │ 57.89 / 58.25 ±0.40 / 58.99 ms │ no change │
│ QQuery 93 │ 187.72 / 190.51 ±1.40 / 191.34 ms │ 186.82 / 188.31 ±0.89 / 189.39 ms │ no change │
│ QQuery 94 │ 61.00 / 62.30 ±1.08 / 64.09 ms │ 61.33 / 62.34 ±0.76 / 63.66 ms │ no change │
│ QQuery 95 │ 128.22 / 129.79 ±0.96 / 130.61 ms │ 128.93 / 129.93 ±1.11 / 131.95 ms │ no change │
│ QQuery 96 │ 73.81 / 74.52 ±0.62 / 75.39 ms │ 73.98 / 74.73 ±0.81 / 76.19 ms │ no change │
│ QQuery 97 │ 125.42 / 128.59 ±2.43 / 132.31 ms │ 125.56 / 127.55 ±1.52 / 129.45 ms │ no change │
│ QQuery 98 │ 151.56 / 154.72 ±2.40 / 157.86 ms │ 156.13 / 156.47 ±0.41 / 157.27 ms │ no change │
│ QQuery 99 │ 10736.99 / 10787.96 ±28.12 / 10812.35 ms │ 10749.37 / 10788.16 ±26.64 / 10828.15 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31602.04ms │
│ Total Time (morsel-scheduler) │ 31561.89ms │
│ Average Time (HEAD) │ 319.21ms │
│ Average Time (morsel-scheduler) │ 318.81ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 94 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 158.4s |
| Peak memory | 5.1 GiB |
| Avg memory | 4.2 GiB |
| CPU user | 261.1s |
| CPU sys | 17.1s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 158.2s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.2 GiB |
| CPU user | 260.8s |
| CPU sys | 17.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275379530
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275376685-1493-7jsxp 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (de8c7f280fa58ce87b040e62103ff2d46db84706) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..de8c7f280fa58ce87b040e62103ff2d46db84706) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273767507
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273756125-1472-4q6hw 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (112637f26a070e610500c18c50e2633fc822588c) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..112637f26a070e610500c18c50e2633fc822588c) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273903617
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273898806-1481-twqbq 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (3d4cb822d349992939910578c0645da129345fea) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..3d4cb822d349992939910578c0645da129345fea) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275707345
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.55 / 6.94 ±0.75 / 8.44 ms │ 6.50 / 6.92 ±0.72 / 8.35 ms │ no change │
│ QQuery 2 │ 147.84 / 148.34 ±0.52 / 149.30 ms │ 145.61 / 146.23 ±0.60 / 147.35 ms │ no change │
│ QQuery 3 │ 113.89 / 114.29 ±0.41 / 115.04 ms │ 112.13 / 113.29 ±1.16 / 115.50 ms │ no change │
│ QQuery 4 │ 1344.57 / 1370.50 ±19.56 / 1394.34 ms │ 1318.05 / 1342.40 ±15.95 / 1365.10 ms │ no change │
│ QQuery 5 │ 172.92 / 175.07 ±1.86 / 178.38 ms │ 170.32 / 172.06 ±1.41 / 174.54 ms │ no change │
│ QQuery 6 │ 838.03 / 856.81 ±13.64 / 875.35 ms │ 841.77 / 867.29 ±18.62 / 890.61 ms │ no change │
│ QQuery 7 │ 338.75 / 343.19 ±2.63 / 346.99 ms │ 340.35 / 342.69 ±1.64 / 345.30 ms │ no change │
│ QQuery 8 │ 118.49 / 118.88 ±0.33 / 119.36 ms │ 114.73 / 116.55 ±1.15 / 118.15 ms │ no change │
│ QQuery 9 │ 102.33 / 105.55 ±2.12 / 107.41 ms │ 100.51 / 101.41 ±0.73 / 102.19 ms │ no change │
│ QQuery 10 │ 106.15 / 107.68 ±1.02 / 108.92 ms │ 104.86 / 105.58 ±0.54 / 106.26 ms │ no change │
│ QQuery 11 │ 949.04 / 968.88 ±10.42 / 978.57 ms │ 958.09 / 971.09 ±9.12 / 983.42 ms │ no change │
│ QQuery 12 │ 46.64 / 48.09 ±1.35 / 50.43 ms │ 45.86 / 46.73 ±0.59 / 47.45 ms │ no change │
│ QQuery 13 │ 397.72 / 404.55 ±5.93 / 414.95 ms │ 404.82 / 407.25 ±2.16 / 410.98 ms │ no change │
│ QQuery 14 │ 1009.57 / 1017.59 ±4.96 / 1023.87 ms │ 1007.80 / 1015.65 ±6.06 / 1024.87 ms │ no change │
│ QQuery 15 │ 16.02 / 17.30 ±1.68 / 20.50 ms │ 15.38 / 16.22 ±0.76 / 17.57 ms │ +1.07x faster │
│ QQuery 16 │ 7.39 / 7.71 ±0.26 / 8.00 ms │ 7.27 / 8.61 ±0.86 / 9.47 ms │ 1.12x slower │
│ QQuery 17 │ 228.59 / 230.24 ±1.40 / 232.70 ms │ 226.46 / 231.32 ±3.94 / 238.21 ms │ no change │
│ QQuery 18 │ 127.96 / 129.82 ±1.50 / 131.77 ms │ 129.26 / 130.15 ±0.96 / 131.91 ms │ no change │
│ QQuery 19 │ 155.85 / 157.06 ±0.93 / 158.40 ms │ 154.07 / 156.63 ±1.45 / 158.59 ms │ no change │
│ QQuery 20 │ 14.22 / 14.79 ±0.36 / 15.24 ms │ 14.12 / 14.58 ±0.52 / 15.59 ms │ no change │
│ QQuery 21 │ 20.20 / 21.00 ±0.68 / 22.00 ms │ 20.09 / 20.37 ±0.21 / 20.67 ms │ no change │
│ QQuery 22 │ 485.00 / 489.71 ±5.50 / 500.39 ms │ 478.86 / 486.09 ±4.17 / 490.56 ms │ no change │
│ QQuery 23 │ 894.80 / 901.28 ±4.01 / 907.28 ms │ 885.92 / 892.65 ±6.95 / 905.63 ms │ no change │
│ QQuery 24 │ 387.51 / 390.72 ±3.72 / 397.97 ms │ 382.66 / 384.58 ±2.10 / 387.41 ms │ no change │
│ QQuery 25 │ 343.65 / 346.61 ±1.82 / 349.03 ms │ 338.52 / 343.09 ±3.27 / 348.17 ms │ no change │
│ QQuery 26 │ 81.14 / 83.02 ±1.44 / 85.17 ms │ 81.89 / 83.09 ±0.89 / 84.50 ms │ no change │
│ QQuery 27 │ 7.24 / 8.10 ±1.31 / 10.68 ms │ 6.86 / 7.55 ±0.60 / 8.61 ms │ +1.07x faster │
│ QQuery 28 │ 150.26 / 151.22 ±1.10 / 153.15 ms │ 147.59 / 149.44 ±1.91 / 152.88 ms │ no change │
│ QQuery 29 │ 284.57 / 288.43 ±2.56 / 292.27 ms │ 279.45 / 282.19 ±1.51 / 283.64 ms │ no change │
│ QQuery 30 │ 43.71 / 45.45 ±1.47 / 47.56 ms │ 43.81 / 45.65 ±1.08 / 46.84 ms │ no change │
│ QQuery 31 │ 174.35 / 175.92 ±1.03 / 177.60 ms │ 165.22 / 169.40 ±3.38 / 172.97 ms │ no change │
│ QQuery 32 │ 57.65 / 58.69 ±0.97 / 60.21 ms │ 57.07 / 66.14 ±17.18 / 100.50 ms │ 1.13x slower │
│ QQuery 33 │ 140.69 / 143.09 ±1.37 / 144.82 ms │ 139.58 / 144.05 ±3.78 / 150.18 ms │ no change │
│ QQuery 34 │ 6.88 / 7.15 ±0.30 / 7.67 ms │ 7.00 / 7.37 ±0.43 / 8.19 ms │ no change │
│ QQuery 35 │ 106.29 / 109.30 ±1.98 / 111.62 ms │ 108.06 / 109.18 ±1.29 / 111.62 ms │ no change │
│ QQuery 36 │ 6.84 / 7.02 ±0.24 / 7.50 ms │ 6.50 / 6.66 ±0.14 / 6.92 ms │ +1.05x faster │
│ QQuery 37 │ 8.73 / 9.06 ±0.23 / 9.32 ms │ 8.46 / 9.12 ±0.89 / 10.86 ms │ no change │
│ QQuery 38 │ 86.44 / 89.71 ±4.26 / 97.44 ms │ 84.54 / 89.22 ±3.80 / 94.30 ms │ no change │
│ QQuery 39 │ 127.88 / 131.69 ±2.18 / 134.39 ms │ 125.27 / 127.16 ±1.51 / 129.16 ms │ no change │
│ QQuery 40 │ 108.85 / 116.06 ±8.09 / 131.47 ms │ 109.13 / 114.56 ±5.72 / 123.84 ms │ no change │
│ QQuery 41 │ 14.75 / 16.43 ±0.92 / 17.49 ms │ 15.15 / 15.55 ±0.44 / 16.26 ms │ +1.06x faster │
│ QQuery 42 │ 110.08 / 110.67 ±0.72 / 111.84 ms │ 106.63 / 108.56 ±1.03 / 109.70 ms │ no change │
│ QQuery 43 │ 5.83 / 6.06 ±0.22 / 6.48 ms │ 5.99 / 6.10 ±0.12 / 6.32 ms │ no change │
│ QQuery 44 │ 11.81 / 11.88 ±0.05 / 11.93 ms │ 11.63 / 12.30 ±0.85 / 13.91 ms │ no change │
│ QQuery 45 │ 50.86 / 51.71 ±0.70 / 52.48 ms │ 49.86 / 50.98 ±0.97 / 52.77 ms │ no change │
│ QQuery 46 │ 8.25 / 8.57 ±0.31 / 9.14 ms │ 8.61 / 8.76 ±0.11 / 8.95 ms │ no change │
│ QQuery 47 │ 735.45 / 741.46 ±7.24 / 755.32 ms │ 734.14 / 738.66 ±3.05 / 742.24 ms │ no change │
│ QQuery 48 │ 289.91 / 293.39 ±1.97 / 295.69 ms │ 282.23 / 290.07 ±5.22 / 297.60 ms │ no change │
│ QQuery 49 │ 250.15 / 253.16 ±2.01 / 255.71 ms │ 247.37 / 250.06 ±2.02 / 253.46 ms │ no change │
│ QQuery 50 │ 219.46 / 224.17 ±3.16 / 228.34 ms │ 225.13 / 225.72 ±0.65 / 226.97 ms │ no change │
│ QQuery 51 │ 181.05 / 183.56 ±1.75 / 186.47 ms │ 177.42 / 181.56 ±3.90 / 188.87 ms │ no change │
│ QQuery 52 │ 108.99 / 112.03 ±4.30 / 120.36 ms │ 106.38 / 107.99 ±1.64 / 111.09 ms │ no change │
│ QQuery 53 │ 102.58 / 103.58 ±0.97 / 105.25 ms │ 102.51 / 103.28 ±0.58 / 104.10 ms │ no change │
│ QQuery 54 │ 145.53 / 149.71 ±4.37 / 157.75 ms │ 143.99 / 145.18 ±0.68 / 146.09 ms │ no change │
│ QQuery 55 │ 106.36 / 108.38 ±1.55 / 110.70 ms │ 106.19 / 108.24 ±2.10 / 110.94 ms │ no change │
│ QQuery 56 │ 139.99 / 142.02 ±1.41 / 144.08 ms │ 138.18 / 139.98 ±1.49 / 142.37 ms │ no change │
│ QQuery 57 │ 174.92 / 177.52 ±2.17 / 181.33 ms │ 172.80 / 175.63 ±2.23 / 179.66 ms │ no change │
│ QQuery 58 │ 290.94 / 298.25 ±6.05 / 305.22 ms │ 283.33 / 292.46 ±4.90 / 297.87 ms │ no change │
│ QQuery 59 │ 196.90 / 200.00 ±3.00 / 205.56 ms │ 196.62 / 198.51 ±1.43 / 200.56 ms │ no change │
│ QQuery 60 │ 143.63 / 146.50 ±1.54 / 148.30 ms │ 144.96 / 146.32 ±0.84 / 147.30 ms │ no change │
│ QQuery 61 │ 12.94 / 13.28 ±0.27 / 13.57 ms │ 12.96 / 13.16 ±0.13 / 13.34 ms │ no change │
│ QQuery 62 │ 902.24 / 949.82 ±27.73 / 972.09 ms │ 898.31 / 946.25 ±41.89 / 1003.04 ms │ no change │
│ QQuery 63 │ 104.85 / 107.17 ±1.70 / 109.41 ms │ 103.83 / 105.51 ±1.30 / 106.99 ms │ no change │
│ QQuery 64 │ 683.73 / 690.84 ±4.29 / 696.99 ms │ 686.61 / 692.24 ±3.30 / 696.14 ms │ no change │
│ QQuery 65 │ 258.09 / 260.70 ±1.87 / 263.64 ms │ 251.19 / 253.89 ±1.63 / 256.06 ms │ no change │
│ QQuery 66 │ 243.11 / 258.53 ±16.94 / 285.84 ms │ 240.43 / 273.53 ±19.16 / 300.26 ms │ 1.06x slower │
│ QQuery 67 │ 319.96 / 324.38 ±5.97 / 335.90 ms │ 311.46 / 320.92 ±8.43 / 336.78 ms │ no change │
│ QQuery 68 │ 8.88 / 10.15 ±1.23 / 12.35 ms │ 9.09 / 12.14 ±4.06 / 20.10 ms │ 1.20x slower │
│ QQuery 69 │ 102.66 / 104.54 ±1.23 / 106.48 ms │ 101.18 / 102.56 ±1.59 / 105.55 ms │ no change │
│ QQuery 70 │ 334.40 / 355.60 ±11.39 / 366.35 ms │ 328.49 / 342.78 ±13.48 / 366.07 ms │ no change │
│ QQuery 71 │ 136.86 / 137.56 ±0.72 / 138.91 ms │ 135.44 / 136.27 ±0.52 / 136.84 ms │ no change │
│ QQuery 72 │ 621.27 / 637.83 ±11.77 / 654.56 ms │ 615.50 / 626.12 ±7.61 / 634.08 ms │ no change │
│ QQuery 73 │ 6.91 / 8.02 ±1.13 / 9.97 ms │ 7.10 / 7.77 ±0.50 / 8.51 ms │ no change │
│ QQuery 74 │ 584.28 / 605.93 ±16.58 / 624.56 ms │ 598.03 / 605.94 ±7.58 / 619.24 ms │ no change │
│ QQuery 75 │ 276.04 / 278.45 ±1.64 / 280.29 ms │ 272.94 / 276.62 ±2.81 / 280.73 ms │ no change │
│ QQuery 76 │ 132.52 / 133.94 ±1.12 / 135.27 ms │ 130.79 / 132.57 ±1.75 / 135.81 ms │ no change │
│ QQuery 77 │ 186.94 / 191.10 ±2.40 / 194.13 ms │ 185.42 / 187.53 ±1.67 / 189.93 ms │ no change │
│ QQuery 78 │ 340.86 / 347.12 ±3.94 / 351.49 ms │ 337.03 / 342.69 ±3.59 / 347.09 ms │ no change │
│ QQuery 79 │ 235.52 / 240.34 ±2.89 / 243.89 ms │ 232.45 / 237.51 ±3.42 / 240.56 ms │ no change │
│ QQuery 80 │ 323.85 / 325.11 ±1.05 / 326.61 ms │ 319.29 / 322.18 ±2.25 / 325.79 ms │ no change │
│ QQuery 81 │ 26.39 / 27.98 ±1.37 / 30.45 ms │ 27.25 / 27.98 ±0.57 / 28.76 ms │ no change │
│ QQuery 82 │ 200.09 / 203.82 ±3.70 / 210.15 ms │ 197.65 / 200.41 ±1.95 / 203.38 ms │ no change │
│ QQuery 83 │ 40.09 / 41.54 ±0.99 / 42.64 ms │ 38.58 / 39.94 ±1.45 / 42.31 ms │ no change │
│ QQuery 84 │ 48.77 / 49.41 ±0.46 / 49.92 ms │ 48.54 / 50.29 ±2.20 / 54.56 ms │ no change │
│ QQuery 85 │ 148.16 / 149.58 ±0.98 / 150.91 ms │ 147.88 / 149.00 ±0.88 / 150.45 ms │ no change │
│ QQuery 86 │ 40.27 / 41.54 ±0.94 / 43.03 ms │ 38.36 / 40.03 ±0.97 / 40.97 ms │ no change │
│ QQuery 87 │ 87.54 / 90.52 ±1.92 / 93.16 ms │ 84.14 / 88.11 ±3.54 / 94.68 ms │ no change │
│ QQuery 88 │ 100.26 / 101.39 ±1.51 / 104.20 ms │ 99.55 / 100.46 ±0.89 / 101.85 ms │ no change │
│ QQuery 89 │ 118.05 / 120.12 ±1.50 / 122.11 ms │ 119.14 / 119.64 ±0.48 / 120.52 ms │ no change │
│ QQuery 90 │ 23.13 / 24.03 ±0.72 / 25.13 ms │ 23.05 / 23.93 ±0.54 / 24.49 ms │ no change │
│ QQuery 91 │ 63.69 / 65.45 ±1.54 / 67.42 ms │ 64.30 / 64.91 ±0.67 / 66.18 ms │ no change │
│ QQuery 92 │ 57.94 / 58.96 ±0.89 / 60.59 ms │ 56.80 / 58.07 ±1.30 / 59.71 ms │ no change │
│ QQuery 93 │ 187.47 / 189.08 ±1.54 / 191.71 ms │ 188.42 / 189.16 ±0.82 / 190.72 ms │ no change │
│ QQuery 94 │ 61.97 / 62.55 ±0.65 / 63.68 ms │ 60.40 / 62.08 ±1.31 / 64.13 ms │ no change │
│ QQuery 95 │ 130.25 / 132.20 ±1.41 / 134.03 ms │ 127.57 / 129.02 ±1.11 / 130.40 ms │ no change │
│ QQuery 96 │ 71.16 / 72.91 ±1.65 / 75.55 ms │ 73.04 / 74.44 ±0.77 / 75.43 ms │ no change │
│ QQuery 97 │ 126.14 / 128.99 ±2.08 / 131.66 ms │ 124.33 / 126.39 ±1.19 / 127.39 ms │ no change │
│ QQuery 98 │ 152.66 / 155.54 ±2.20 / 159.43 ms │ 151.42 / 153.83 ±2.57 / 158.26 ms │ no change │
│ QQuery 99 │ 10749.50 / 10782.75 ±27.22 / 10819.94 ms │ 10784.15 / 10818.52 ±17.95 / 10833.90 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31734.32ms │
│ Total Time (morsel-scheduler) │ 31610.52ms │
│ Average Time (HEAD) │ 320.55ms │
│ Average Time (morsel-scheduler) │ 319.30ms │
│ Queries Faster │ 4 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 91 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 159.0s |
| Peak memory | 5.5 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 262.9s |
| CPU sys | 16.8s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 158.4s |
| Peak memory | 5.3 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 262.1s |
| CPU sys | 16.5s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275711166
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.18 / 4.44 ±6.37 / 17.18 ms │ 1.27 / 4.51 ±6.33 / 17.16 ms │ no change │
│ QQuery 1 │ 14.17 / 14.62 ±0.24 / 14.82 ms │ 15.97 / 17.87 ±1.31 / 19.34 ms │ 1.22x slower │
│ QQuery 2 │ 43.86 / 44.01 ±0.09 / 44.11 ms │ 57.11 / 61.21 ±2.94 / 66.09 ms │ 1.39x slower │
│ QQuery 3 │ 38.90 / 40.56 ±1.23 / 42.22 ms │ 49.07 / 57.68 ±5.38 / 63.94 ms │ 1.42x slower │
│ QQuery 4 │ 284.83 / 290.55 ±6.29 / 302.30 ms │ 594.03 / 737.18 ±90.09 / 864.06 ms │ 2.54x slower │
│ QQuery 5 │ 337.30 / 340.07 ±2.50 / 344.67 ms │ 689.91 / 811.60 ±103.42 / 985.25 ms │ 2.39x slower │
│ QQuery 6 │ 5.62 / 6.60 ±0.79 / 7.85 ms │ 3.40 / 5.23 ±1.18 / 6.62 ms │ +1.26x faster │
│ QQuery 7 │ 16.87 / 17.61 ±0.83 / 18.73 ms │ 19.65 / 22.36 ±3.33 / 28.42 ms │ 1.27x slower │
│ QQuery 8 │ 420.24 / 428.23 ±4.88 / 434.46 ms │ 959.07 / 1082.91 ±90.37 / 1208.34 ms │ 2.53x slower │
│ QQuery 9 │ 648.04 / 659.69 ±8.61 / 669.07 ms │ 929.13 / 1070.72 ±89.18 / 1170.87 ms │ 1.62x slower │
│ QQuery 10 │ 91.09 / 93.60 ±2.10 / 97.19 ms │ 144.33 / 168.79 ±17.12 / 195.49 ms │ 1.80x slower │
│ QQuery 11 │ 103.76 / 104.94 ±1.21 / 107.19 ms │ 168.76 / 185.51 ±14.04 / 210.92 ms │ 1.77x slower │
│ QQuery 12 │ 341.87 / 345.51 ±2.32 / 348.50 ms │ 636.04 / 816.92 ±159.66 / 1038.83 ms │ 2.36x slower │
│ QQuery 13 │ 455.93 / 467.27 ±7.68 / 477.43 ms │ 881.60 / 1284.03 ±251.34 / 1580.26 ms │ 2.75x slower │
│ QQuery 14 │ 337.36 / 346.34 ±5.73 / 354.85 ms │ 666.00 / 820.54 ±157.38 / 1078.51 ms │ 2.37x slower │
│ QQuery 15 │ 350.14 / 364.77 ±16.60 / 395.76 ms │ 760.89 / 1150.93 ±239.66 / 1455.82 ms │ 3.16x slower │
│ QQuery 16 │ 712.85 / 721.63 ±10.69 / 742.49 ms │ 1801.25 / 2019.34 ±185.97 / 2301.35 ms │ 2.80x slower │
│ QQuery 17 │ 703.56 / 709.52 ±3.96 / 713.91 ms │ 1628.17 / 2100.24 ±262.81 / 2370.97 ms │ 2.96x slower │
│ QQuery 18 │ 1437.38 / 1467.44 ±19.80 / 1499.14 ms │ 2819.78 / 3700.08 ±682.07 / 4920.47 ms │ 2.52x slower │
│ QQuery 19 │ 35.00 / 49.45 ±25.98 / 101.37 ms │ 56.09 / 64.14 ±8.66 / 78.40 ms │ 1.30x slower │
│ QQuery 20 │ 715.90 / 735.21 ±15.75 / 756.29 ms │ 837.15 / 948.67 ±67.87 / 1034.48 ms │ 1.29x slower │
│ QQuery 21 │ 755.86 / 762.93 ±3.74 / 766.54 ms │ 904.31 / 1046.31 ±98.48 / 1169.00 ms │ 1.37x slower │
│ QQuery 22 │ 1130.31 / 1140.48 ±7.82 / 1152.93 ms │ 1836.22 / 2271.08 ±377.16 / 2912.08 ms │ 1.99x slower │
│ QQuery 23 │ 3084.56 / 3101.65 ±19.82 / 3140.21 ms │ 4807.75 / 5341.80 ±295.89 / 5625.67 ms │ 1.72x slower │
│ QQuery 24 │ 99.24 / 102.95 ±2.58 / 105.73 ms │ 65.03 / 81.41 ±11.82 / 101.67 ms │ +1.26x faster │
│ QQuery 25 │ 138.28 / 140.01 ±1.20 / 141.69 ms │ 180.35 / 200.44 ±23.03 / 244.83 ms │ 1.43x slower │
│ QQuery 26 │ 97.25 / 101.19 ±2.09 / 102.86 ms │ 64.41 / 84.21 ±17.06 / 115.16 ms │ +1.20x faster │
│ QQuery 27 │ 847.83 / 850.25 ±1.91 / 852.78 ms │ 908.85 / 1252.95 ±338.98 / 1727.75 ms │ 1.47x slower │
│ QQuery 28 │ 3242.21 / 3274.52 ±18.11 / 3290.84 ms │ 5496.92 / 6085.26 ±540.20 / 6929.65 ms │ 1.86x slower │
│ QQuery 29 │ 50.02 / 55.12 ±6.09 / 66.29 ms │ 63.58 / 79.76 ±10.90 / 97.80 ms │ 1.45x slower │
│ QQuery 30 │ 359.81 / 363.54 ±2.54 / 366.44 ms │ 650.52 / 782.30 ±107.90 / 902.54 ms │ 2.15x slower │
│ QQuery 31 │ 362.48 / 381.64 ±14.13 / 401.17 ms │ 506.12 / 737.36 ±169.72 / 908.51 ms │ 1.93x slower │
│ QQuery 32 │ 1044.56 / 1059.07 ±15.70 / 1087.27 ms │ 3284.19 / 4541.69 ±1119.89 / 6289.03 ms │ 4.29x slower │
│ QQuery 33 │ 1450.63 / 1452.26 ±1.69 / 1454.91 ms │ 3153.48 / 4067.28 ±778.56 / 5159.59 ms │ 2.80x slower │
│ QQuery 34 │ 1438.96 / 1479.65 ±28.03 / 1511.97 ms │ 3171.75 / 3586.55 ±293.99 / 4072.26 ms │ 2.42x slower │
│ QQuery 35 │ 382.52 / 389.06 ±4.12 / 393.26 ms │ 780.93 / 874.83 ±106.20 / 1066.66 ms │ 2.25x slower │
│ QQuery 36 │ 110.95 / 118.43 ±4.10 / 123.41 ms │ 64.73 / 81.70 ±9.57 / 94.16 ms │ +1.45x faster │
│ QQuery 37 │ 46.62 / 48.52 ±1.54 / 51.04 ms │ 38.47 / 40.02 ±1.16 / 41.67 ms │ +1.21x faster │
│ QQuery 38 │ 72.74 / 74.70 ±1.69 / 77.78 ms │ 44.09 / 49.71 ±6.04 / 57.58 ms │ +1.50x faster │
│ QQuery 39 │ 196.91 / 206.00 ±5.10 / 211.47 ms │ 112.52 / 139.44 ±16.25 / 156.91 ms │ +1.48x faster │
│ QQuery 40 │ 20.94 / 24.57 ±2.51 / 27.04 ms │ 16.33 / 20.06 ±2.53 / 22.86 ms │ +1.23x faster │
│ QQuery 41 │ 18.90 / 21.32 ±1.84 / 23.93 ms │ 12.93 / 14.83 ±1.52 / 16.54 ms │ +1.44x faster │
│ QQuery 42 │ 19.20 / 19.83 ±0.43 / 20.41 ms │ 11.89 / 14.69 ±2.45 / 19.06 ms │ +1.35x faster │
└───────────┴───────────────────────────────────────┴─────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22419.78ms │
│ Total Time (morsel-scheduler) │ 48524.14ms │
│ Average Time (HEAD) │ 521.39ms │
│ Average Time (morsel-scheduler) │ 1128.47ms │
│ Queries Faster │ 10 │
│ Queries Slower │ 32 │
│ Queries with No Change │ 1 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.1s |
| Peak memory | 37.5 GiB |
| Avg memory | 27.0 GiB |
| CPU user | 1067.1s |
| CPU sys | 86.5s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 244.2s |
| Peak memory | 42.0 GiB |
| Avg memory | 29.9 GiB |
| CPU user | 1059.5s |
| CPU sys | 111.9s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275679397
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275639877-1511-f97mc 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275711813
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275674921-1516-xwg7c 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275380098
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275376685-1492-c26k8 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (de8c7f280fa58ce87b040e62103ff2d46db84706) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..de8c7f280fa58ce87b040e62103ff2d46db84706) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275711143
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.24 / 4.60 ±6.57 / 17.73 ms │ 1.21 / 4.52 ±6.44 / 17.40 ms │ no change │
│ QQuery 1 │ 14.31 / 14.85 ±0.32 / 15.30 ms │ 18.87 / 20.82 ±1.88 / 24.19 ms │ 1.40x slower │
│ QQuery 2 │ 44.20 / 44.78 ±0.32 / 45.05 ms │ 50.98 / 56.09 ±2.91 / 58.94 ms │ 1.25x slower │
│ QQuery 3 │ 41.31 / 42.50 ±0.97 / 44.19 ms │ 44.94 / 57.54 ±9.62 / 72.80 ms │ 1.35x slower │
│ QQuery 4 │ 289.61 / 293.97 ±3.96 / 301.18 ms │ 727.68 / 852.46 ±98.05 / 973.18 ms │ 2.90x slower │
│ QQuery 5 │ 339.08 / 343.50 ±4.62 / 351.86 ms │ 788.14 / 903.39 ±117.92 / 1110.92 ms │ 2.63x slower │
│ QQuery 6 │ 5.12 / 6.39 ±0.68 / 7.01 ms │ 3.62 / 5.07 ±1.05 / 6.89 ms │ +1.26x faster │
│ QQuery 7 │ 16.56 / 16.94 ±0.57 / 18.07 ms │ 15.81 / 19.13 ±3.19 / 24.07 ms │ 1.13x slower │
│ QQuery 8 │ 415.98 / 425.40 ±9.06 / 441.88 ms │ 1008.61 / 1260.70 ±184.02 / 1540.48 ms │ 2.96x slower │
│ QQuery 9 │ 657.90 / 663.56 ±4.58 / 671.23 ms │ 872.96 / 1073.74 ±215.33 / 1483.53 ms │ 1.62x slower │
│ QQuery 10 │ 91.74 / 95.40 ±3.01 / 100.19 ms │ 135.13 / 155.81 ±13.30 / 170.91 ms │ 1.63x slower │
│ QQuery 11 │ 105.01 / 106.11 ±0.64 / 106.72 ms │ 165.65 / 220.24 ±30.55 / 249.32 ms │ 2.08x slower │
│ QQuery 12 │ 338.48 / 343.31 ±4.78 / 351.86 ms │ 762.40 / 1000.04 ±176.99 / 1189.43 ms │ 2.91x slower │
│ QQuery 13 │ 458.24 / 466.54 ±8.58 / 479.96 ms │ 1109.11 / 1291.00 ±120.09 / 1429.28 ms │ 2.77x slower │
│ QQuery 14 │ 347.96 / 355.18 ±5.10 / 363.92 ms │ 686.39 / 836.60 ±112.14 / 1023.16 ms │ 2.36x slower │
│ QQuery 15 │ 356.59 / 362.37 ±6.33 / 373.04 ms │ 1023.63 / 1220.20 ±169.22 / 1530.43 ms │ 3.37x slower │
│ QQuery 16 │ 716.66 / 728.11 ±10.75 / 746.92 ms │ 1919.57 / 2252.97 ±182.44 / 2448.95 ms │ 3.09x slower │
│ QQuery 17 │ 708.30 / 717.60 ±5.74 / 724.89 ms │ 1708.63 / 1984.89 ±224.80 / 2382.07 ms │ 2.77x slower │
│ QQuery 18 │ 1438.52 / 1478.75 ±22.49 / 1508.11 ms │ 3279.25 / 3628.50 ±321.29 / 4123.33 ms │ 2.45x slower │
│ QQuery 19 │ 38.17 / 46.83 ±16.23 / 79.27 ms │ 49.30 / 58.44 ±7.33 / 71.63 ms │ 1.25x slower │
│ QQuery 20 │ 723.21 / 739.30 ±16.85 / 762.88 ms │ 811.66 / 966.18 ±170.38 / 1191.84 ms │ 1.31x slower │
│ QQuery 21 │ 764.93 / 770.75 ±5.06 / 780.19 ms │ 1014.10 / 1154.90 ±229.63 / 1609.24 ms │ 1.50x slower │
│ QQuery 22 │ 1144.32 / 1150.89 ±9.68 / 1169.87 ms │ 1850.34 / 2298.76 ±331.99 / 2679.79 ms │ 2.00x slower │
│ QQuery 23 │ 3110.53 / 3157.34 ±27.58 / 3190.25 ms │ 4929.14 / 5387.12 ±352.33 / 5886.63 ms │ 1.71x slower │
│ QQuery 24 │ 104.31 / 105.33 ±0.91 / 106.99 ms │ 69.92 / 74.32 ±4.32 / 81.78 ms │ +1.42x faster │
│ QQuery 25 │ 140.79 / 142.50 ±1.22 / 143.87 ms │ 177.00 / 220.74 ±24.09 / 246.54 ms │ 1.55x slower │
│ QQuery 26 │ 105.42 / 128.72 ±32.16 / 190.10 ms │ 61.69 / 84.94 ±18.78 / 118.67 ms │ +1.52x faster │
│ QQuery 27 │ 861.71 / 880.62 ±25.26 / 930.13 ms │ 910.08 / 1054.06 ±82.84 / 1129.80 ms │ 1.20x slower │
│ QQuery 28 │ 3262.61 / 3298.87 ±25.05 / 3325.58 ms │ 5440.10 / 6752.22 ±994.05 / 8512.85 ms │ 2.05x slower │
│ QQuery 29 │ 53.17 / 56.99 ±5.92 / 68.64 ms │ 63.69 / 73.43 ±5.93 / 80.04 ms │ 1.29x slower │
│ QQuery 30 │ 365.98 / 372.37 ±3.62 / 377.03 ms │ 654.29 / 838.73 ±189.62 / 1186.00 ms │ 2.25x slower │
│ QQuery 31 │ 390.71 / 401.72 ±7.92 / 412.62 ms │ 566.16 / 603.82 ±43.06 / 657.65 ms │ 1.50x slower │
│ QQuery 32 │ 1193.16 / 1252.91 ±53.94 / 1336.10 ms │ 3356.37 / 4114.10 ±869.96 / 5756.83 ms │ 3.28x slower │
│ QQuery 33 │ 1497.79 / 1526.98 ±34.27 / 1592.55 ms │ 3309.47 / 3546.35 ±173.34 / 3767.46 ms │ 2.32x slower │
│ QQuery 34 │ 1536.04 / 1562.60 ±20.95 / 1583.51 ms │ 3402.60 / 3927.95 ±357.73 / 4454.29 ms │ 2.51x slower │
│ QQuery 35 │ 415.39 / 422.06 ±5.66 / 431.63 ms │ 700.56 / 997.22 ±188.50 / 1214.36 ms │ 2.36x slower │
│ QQuery 36 │ 115.75 / 124.69 ±5.76 / 132.70 ms │ 72.56 / 78.56 ±4.45 / 84.90 ms │ +1.59x faster │
│ QQuery 37 │ 48.60 / 49.79 ±1.01 / 51.25 ms │ 36.89 / 41.53 ±6.73 / 54.77 ms │ +1.20x faster │
│ QQuery 38 │ 76.74 / 78.80 ±2.18 / 81.77 ms │ 42.90 / 57.56 ±8.92 / 70.72 ms │ +1.37x faster │
│ QQuery 39 │ 214.97 / 228.31 ±8.35 / 237.10 ms │ 144.74 / 166.35 ±12.74 / 179.92 ms │ +1.37x faster │
│ QQuery 40 │ 24.41 / 25.87 ±1.17 / 27.32 ms │ 15.92 / 17.90 ±2.28 / 22.17 ms │ +1.45x faster │
│ QQuery 41 │ 21.30 / 22.87 ±1.16 / 24.58 ms │ 13.77 / 16.42 ±2.27 / 19.98 ms │ +1.39x faster │
│ QQuery 42 │ 20.16 / 21.49 ±1.53 / 24.48 ms │ 11.22 / 12.48 ±0.94 / 13.31 ms │ +1.72x faster │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 23078.44ms │
│ Total Time (morsel-scheduler) │ 49387.80ms │
│ Average Time (HEAD) │ 536.71ms │
│ Average Time (morsel-scheduler) │ 1148.55ms │
│ Queries Faster │ 10 │
│ Queries Slower │ 32 │
│ Queries with No Change │ 1 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 116.4s |
| Peak memory | 35.1 GiB |
| Avg memory | 25.6 GiB |
| CPU user | 1079.2s |
| CPU sys | 102.0s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 248.6s |
| Peak memory | 43.0 GiB |
| Avg memory | 30.9 GiB |
| CPU user | 1076.8s |
| CPU sys | 114.3s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273727484
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273723891-1466-9hf2t 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (580bb936213d6513f753d97bcb4728ffe91c8e21) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..580bb936213d6513f753d97bcb4728ffe91c8e21) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273903686
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.21 / 4.45 ±6.36 / 17.17 ms │ 1.24 / 4.53 ±6.43 / 17.39 ms │ no change │
│ QQuery 1 │ 14.98 / 15.39 ±0.24 / 15.68 ms │ 21.17 / 32.48 ±8.65 / 46.03 ms │ 2.11x slower │
│ QQuery 2 │ 45.97 / 46.40 ±0.31 / 46.75 ms │ 76.20 / 111.93 ±23.30 / 146.80 ms │ 2.41x slower │
│ QQuery 3 │ 44.02 / 46.88 ±2.20 / 49.26 ms │ 88.73 / 99.25 ±10.77 / 119.71 ms │ 2.12x slower │
│ QQuery 4 │ 289.54 / 294.00 ±4.03 / 300.09 ms │ 614.48 / 692.06 ±91.82 / 865.54 ms │ 2.35x slower │
│ QQuery 5 │ 339.18 / 341.90 ±1.98 / 345.30 ms │ 645.51 / 955.64 ±294.00 / 1477.86 ms │ 2.80x slower │
│ QQuery 6 │ 5.66 / 6.69 ±1.29 / 9.12 ms │ 3.53 / 4.66 ±1.19 / 6.65 ms │ +1.44x faster │
│ QQuery 7 │ 17.29 / 17.65 ±0.30 / 18.05 ms │ 27.05 / 29.88 ±2.17 / 32.55 ms │ 1.69x slower │
│ QQuery 8 │ 408.61 / 416.77 ±7.39 / 428.69 ms │ 855.52 / 1243.74 ±281.46 / 1653.84 ms │ 2.98x slower │
│ QQuery 9 │ 647.07 / 659.26 ±8.83 / 670.52 ms │ 1160.36 / 1584.69 ±257.92 / 1931.62 ms │ 2.40x slower │
│ QQuery 10 │ 91.73 / 95.51 ±3.36 / 101.22 ms │ 164.26 / 183.20 ±17.66 / 213.01 ms │ 1.92x slower │
│ QQuery 11 │ 105.98 / 106.81 ±0.66 / 107.86 ms │ 175.59 / 230.00 ±29.07 / 258.67 ms │ 2.15x slower │
│ QQuery 12 │ 335.63 / 342.41 ±4.57 / 348.39 ms │ 565.08 / 702.26 ±93.31 / 824.89 ms │ 2.05x slower │
│ QQuery 13 │ 461.08 / 471.92 ±10.98 / 493.04 ms │ 1192.67 / 1316.24 ±124.02 / 1493.58 ms │ 2.79x slower │
│ QQuery 14 │ 345.12 / 348.24 ±2.65 / 351.32 ms │ 850.98 / 944.67 ±69.51 / 1040.53 ms │ 2.71x slower │
│ QQuery 15 │ 352.42 / 364.39 ±8.84 / 378.74 ms │ 780.89 / 1170.63 ±336.53 / 1783.97 ms │ 3.21x slower │
│ QQuery 16 │ 709.16 / 719.36 ±10.02 / 735.74 ms │ 1664.07 / 1808.84 ±139.57 / 2042.04 ms │ 2.51x slower │
│ QQuery 17 │ 713.97 / 716.78 ±2.14 / 719.50 ms │ 1865.37 / 2291.46 ±275.92 / 2685.67 ms │ 3.20x slower │
│ QQuery 18 │ 1397.67 / 1474.65 ±52.41 / 1547.25 ms │ 3486.08 / 3977.01 ±422.16 / 4671.66 ms │ 2.70x slower │
│ QQuery 19 │ 37.11 / 66.71 ±55.46 / 177.57 ms │ 78.94 / 97.94 ±22.38 / 141.63 ms │ 1.47x slower │
│ QQuery 20 │ 726.89 / 747.65 ±22.36 / 784.96 ms │ 1037.24 / 1148.84 ±121.26 / 1369.76 ms │ 1.54x slower │
│ QQuery 21 │ 763.86 / 769.94 ±5.50 / 779.13 ms │ 1175.30 / 1707.30 ±504.19 / 2349.44 ms │ 2.22x slower │
│ QQuery 22 │ 1138.45 / 1145.35 ±8.21 / 1160.91 ms │ 2345.04 / 2508.81 ±93.13 / 2624.77 ms │ 2.19x slower │
│ QQuery 23 │ 3114.01 / 3132.49 ±16.22 / 3161.63 ms │ 3233.21 / 4054.77 ±474.30 / 4660.38 ms │ 1.29x slower │
│ QQuery 24 │ 99.93 / 104.98 ±4.40 / 112.43 ms │ 105.84 / 116.64 ±7.19 / 125.19 ms │ 1.11x slower │
│ QQuery 25 │ 138.37 / 141.77 ±2.45 / 145.84 ms │ 251.83 / 325.49 ±102.68 / 528.80 ms │ 2.30x slower │
│ QQuery 26 │ 102.35 / 103.97 ±1.02 / 105.37 ms │ 117.20 / 134.66 ±13.53 / 154.01 ms │ 1.30x slower │
│ QQuery 27 │ 850.69 / 857.30 ±6.63 / 870.00 ms │ 1377.50 / 1811.40 ±434.54 / 2466.04 ms │ 2.11x slower │
│ QQuery 28 │ 3240.73 / 3272.84 ±17.56 / 3290.28 ms │ 7983.30 / 10609.13 ±1713.98 / 13152.92 ms │ 3.24x slower │
│ QQuery 29 │ 51.70 / 57.10 ±4.64 / 63.38 ms │ 113.64 / 129.23 ±14.64 / 157.12 ms │ 2.26x slower │
│ QQuery 30 │ 356.11 / 367.40 ±6.45 / 373.89 ms │ 653.06 / 908.38 ±195.23 / 1129.64 ms │ 2.47x slower │
│ QQuery 31 │ 375.72 / 384.26 ±8.21 / 397.71 ms │ 630.04 / 774.71 ±143.17 / 969.03 ms │ 2.02x slower │
│ QQuery 32 │ 1046.60 / 1064.40 ±19.15 / 1100.16 ms │ 3170.73 / 3680.52 ±372.79 / 4185.70 ms │ 3.46x slower │
│ QQuery 33 │ 1442.48 / 1451.78 ±9.11 / 1468.82 ms │ 3511.61 / 4148.91 ±455.35 / 4846.24 ms │ 2.86x slower │
│ QQuery 34 │ 1427.57 / 1452.62 ±23.30 / 1492.43 ms │ 3264.33 / 3691.20 ±320.86 / 4060.16 ms │ 2.54x slower │
│ QQuery 35 │ 391.29 / 398.38 ±5.07 / 405.62 ms │ 766.19 / 973.38 ±181.21 / 1279.01 ms │ 2.44x slower │
│ QQuery 36 │ 111.88 / 117.24 ±4.65 / 125.19 ms │ 120.78 / 129.58 ±6.83 / 135.74 ms │ 1.11x slower │
│ QQuery 37 │ 46.89 / 47.73 ±0.71 / 48.86 ms │ 46.47 / 48.97 ±2.12 / 52.50 ms │ no change │
│ QQuery 38 │ 76.45 / 76.84 ±0.34 / 77.27 ms │ 74.65 / 75.54 ±0.61 / 76.20 ms │ no change │
│ QQuery 39 │ 202.35 / 222.07 ±11.25 / 235.17 ms │ 221.44 / 244.09 ±21.73 / 280.30 ms │ 1.10x slower │
│ QQuery 40 │ 25.47 / 27.90 ±1.72 / 30.00 ms │ 20.91 / 24.22 ±2.01 / 26.99 ms │ +1.15x faster │
│ QQuery 41 │ 19.50 / 22.46 ±1.83 / 24.43 ms │ 17.93 / 20.39 ±1.61 / 22.52 ms │ +1.10x faster │
│ QQuery 42 │ 20.01 / 20.23 ±0.16 / 20.50 ms │ 17.74 / 19.19 ±1.47 / 21.82 ms │ +1.05x faster │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22542.88ms │
│ Total Time (morsel-scheduler) │ 54766.45ms │
│ Average Time (HEAD) │ 524.25ms │
│ Average Time (morsel-scheduler) │ 1273.64ms │
│ Queries Faster │ 4 │
│ Queries Slower │ 36 │
│ Queries with No Change │ 3 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.8s |
| Peak memory | 39.1 GiB |
| Avg memory | 29.3 GiB |
| CPU user | 1068.7s |
| CPU sys | 88.1s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 275.4s |
| Peak memory | 43.5 GiB |
| Avg memory | 28.8 GiB |
| CPU user | 987.1s |
| CPU sys | 112.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273908500
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273898806-1480-9ffll 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (3d4cb822d349992939910578c0645da129345fea) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..3d4cb822d349992939910578c0645da129345fea) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275680443
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275639877-1513-f2776 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: tpch
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273804919
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.40 / 6.89 ±0.78 / 8.44 ms │ 6.67 / 7.11 ±0.79 / 8.70 ms │ no change │
│ QQuery 2 │ 145.40 / 146.24 ±0.51 / 146.92 ms │ 144.88 / 146.76 ±1.06 / 147.99 ms │ no change │
│ QQuery 3 │ 113.49 / 114.02 ±0.52 / 114.80 ms │ 113.63 / 114.63 ±0.51 / 115.04 ms │ no change │
│ QQuery 4 │ 1326.32 / 1347.05 ±15.86 / 1368.12 ms │ 1329.71 / 1341.84 ±12.00 / 1361.57 ms │ no change │
│ QQuery 5 │ 170.04 / 172.36 ±1.48 / 174.59 ms │ 172.79 / 174.20 ±1.15 / 175.70 ms │ no change │
│ QQuery 6 │ 824.66 / 865.70 ±29.30 / 906.54 ms │ 816.47 / 855.39 ±23.29 / 887.01 ms │ no change │
│ QQuery 7 │ 340.42 / 343.50 ±2.51 / 347.17 ms │ 341.76 / 344.69 ±1.79 / 346.75 ms │ no change │
│ QQuery 8 │ 114.69 / 116.10 ±0.89 / 117.17 ms │ 115.03 / 116.35 ±0.87 / 117.68 ms │ no change │
│ QQuery 9 │ 100.69 / 103.57 ±3.17 / 108.03 ms │ 102.27 / 106.86 ±2.51 / 109.93 ms │ no change │
│ QQuery 10 │ 105.57 / 106.09 ±0.39 / 106.58 ms │ 104.34 / 106.55 ±1.64 / 109.43 ms │ no change │
│ QQuery 11 │ 946.45 / 955.02 ±6.33 / 964.27 ms │ 956.29 / 962.79 ±3.97 / 968.38 ms │ no change │
│ QQuery 12 │ 45.71 / 47.84 ±1.70 / 50.54 ms │ 44.12 / 45.52 ±1.18 / 47.40 ms │ no change │
│ QQuery 13 │ 398.52 / 402.68 ±2.40 / 405.69 ms │ 399.68 / 402.51 ±1.92 / 405.64 ms │ no change │
│ QQuery 14 │ 993.23 / 1006.17 ±10.18 / 1022.94 ms │ 1005.20 / 1011.89 ±5.54 / 1020.04 ms │ no change │
│ QQuery 15 │ 15.75 / 16.57 ±1.05 / 18.64 ms │ 16.14 / 17.27 ±0.67 / 17.93 ms │ no change │
│ QQuery 16 │ 7.15 / 7.81 ±0.81 / 9.40 ms │ 7.13 / 7.42 ±0.22 / 7.74 ms │ no change │
│ QQuery 17 │ 227.70 / 231.74 ±3.10 / 235.99 ms │ 230.00 / 231.53 ±1.08 / 233.31 ms │ no change │
│ QQuery 18 │ 126.95 / 128.24 ±0.84 / 129.45 ms │ 127.90 / 129.03 ±1.10 / 130.41 ms │ no change │
│ QQuery 19 │ 155.76 / 156.48 ±0.92 / 158.25 ms │ 154.56 / 156.02 ±1.22 / 157.56 ms │ no change │
│ QQuery 20 │ 13.80 / 14.20 ±0.35 / 14.81 ms │ 13.97 / 14.54 ±0.38 / 14.96 ms │ no change │
│ QQuery 21 │ 18.79 / 19.69 ±0.49 / 20.12 ms │ 20.00 / 20.93 ±0.89 / 22.44 ms │ 1.06x slower │
│ QQuery 22 │ 475.80 / 477.05 ±0.92 / 478.26 ms │ 482.08 / 488.03 ±3.30 / 491.00 ms │ no change │
│ QQuery 23 │ 863.05 / 881.23 ±13.26 / 902.15 ms │ 875.75 / 883.45 ±6.40 / 895.10 ms │ no change │
│ QQuery 24 │ 385.29 / 387.70 ±2.07 / 391.08 ms │ 383.51 / 387.00 ±1.91 / 388.97 ms │ no change │
│ QQuery 25 │ 338.51 / 341.97 ±1.83 / 343.61 ms │ 341.77 / 344.04 ±2.10 / 347.59 ms │ no change │
│ QQuery 26 │ 80.18 / 82.67 ±1.44 / 84.17 ms │ 81.19 / 82.43 ±0.99 / 84.14 ms │ no change │
│ QQuery 27 │ 6.93 / 7.59 ±0.76 / 8.97 ms │ 6.77 / 7.16 ±0.22 / 7.35 ms │ +1.06x faster │
│ QQuery 28 │ 148.38 / 149.03 ±0.92 / 150.83 ms │ 149.23 / 150.49 ±0.98 / 151.76 ms │ no change │
│ QQuery 29 │ 282.70 / 283.55 ±1.22 / 285.91 ms │ 283.43 / 284.70 ±0.95 / 286.02 ms │ no change │
│ QQuery 30 │ 42.64 / 44.90 ±1.74 / 47.20 ms │ 42.84 / 46.03 ±2.01 / 49.01 ms │ no change │
│ QQuery 31 │ 170.67 / 173.01 ±1.66 / 175.30 ms │ 169.51 / 173.00 ±1.95 / 175.54 ms │ no change │
│ QQuery 32 │ 56.77 / 58.36 ±1.19 / 60.05 ms │ 57.01 / 66.87 ±17.63 / 102.10 ms │ 1.15x slower │
│ QQuery 33 │ 140.39 / 141.79 ±1.04 / 143.18 ms │ 140.38 / 142.13 ±0.88 / 142.74 ms │ no change │
│ QQuery 34 │ 6.92 / 7.13 ±0.22 / 7.52 ms │ 6.87 / 7.06 ±0.24 / 7.51 ms │ no change │
│ QQuery 35 │ 104.95 / 107.63 ±1.69 / 109.52 ms │ 107.61 / 108.29 ±0.80 / 109.84 ms │ no change │
│ QQuery 36 │ 6.34 / 6.51 ±0.17 / 6.84 ms │ 6.56 / 6.75 ±0.21 / 7.15 ms │ no change │
│ QQuery 37 │ 8.45 / 8.62 ±0.27 / 9.15 ms │ 8.48 / 9.27 ±0.65 / 10.44 ms │ 1.08x slower │
│ QQuery 38 │ 85.05 / 89.79 ±3.03 / 94.35 ms │ 86.80 / 89.94 ±4.80 / 99.49 ms │ no change │
│ QQuery 39 │ 123.44 / 126.38 ±2.32 / 130.37 ms │ 123.65 / 125.95 ±2.36 / 130.19 ms │ no change │
│ QQuery 40 │ 107.64 / 116.34 ±8.85 / 132.24 ms │ 112.96 / 115.32 ±2.43 / 119.64 ms │ no change │
│ QQuery 41 │ 14.09 / 15.29 ±1.06 / 17.06 ms │ 14.53 / 15.76 ±1.18 / 17.50 ms │ no change │
│ QQuery 42 │ 107.50 / 109.91 ±1.71 / 112.82 ms │ 106.28 / 109.23 ±1.58 / 110.74 ms │ no change │
│ QQuery 43 │ 5.89 / 6.05 ±0.20 / 6.45 ms │ 5.93 / 6.05 ±0.16 / 6.37 ms │ no change │
│ QQuery 44 │ 11.45 / 11.63 ±0.13 / 11.84 ms │ 11.50 / 11.93 ±0.38 / 12.49 ms │ no change │
│ QQuery 45 │ 50.09 / 51.48 ±1.43 / 54.13 ms │ 50.03 / 51.63 ±1.20 / 52.99 ms │ no change │
│ QQuery 46 │ 8.38 / 8.57 ±0.16 / 8.79 ms │ 8.26 / 8.53 ±0.20 / 8.80 ms │ no change │
│ QQuery 47 │ 711.31 / 723.53 ±6.27 / 728.48 ms │ 715.55 / 725.69 ±8.08 / 737.33 ms │ no change │
│ QQuery 48 │ 286.81 / 290.38 ±2.76 / 293.91 ms │ 286.05 / 290.84 ±3.78 / 294.65 ms │ no change │
│ QQuery 49 │ 251.12 / 254.37 ±1.71 / 255.82 ms │ 253.59 / 256.54 ±3.43 / 263.15 ms │ no change │
│ QQuery 50 │ 230.30 / 233.32 ±1.80 / 235.61 ms │ 223.05 / 227.96 ±3.10 / 232.10 ms │ no change │
│ QQuery 51 │ 177.81 / 181.24 ±2.10 / 183.46 ms │ 182.29 / 184.73 ±1.96 / 187.55 ms │ no change │
│ QQuery 52 │ 108.42 / 109.21 ±0.61 / 109.91 ms │ 106.88 / 108.21 ±1.51 / 111.04 ms │ no change │
│ QQuery 53 │ 102.38 / 103.37 ±0.61 / 104.13 ms │ 102.75 / 103.85 ±1.29 / 106.22 ms │ no change │
│ QQuery 54 │ 145.57 / 146.84 ±0.65 / 147.38 ms │ 147.32 / 148.33 ±1.29 / 150.86 ms │ no change │
│ QQuery 55 │ 107.26 / 109.16 ±1.51 / 110.93 ms │ 107.06 / 108.83 ±1.83 / 112.20 ms │ no change │
│ QQuery 56 │ 140.02 / 141.99 ±1.94 / 145.27 ms │ 139.83 / 142.71 ±2.63 / 146.00 ms │ no change │
│ QQuery 57 │ 172.82 / 175.19 ±1.87 / 177.49 ms │ 172.12 / 174.62 ±1.70 / 176.71 ms │ no change │
│ QQuery 58 │ 289.06 / 299.37 ±6.62 / 308.32 ms │ 285.89 / 299.65 ±8.54 / 311.89 ms │ no change │
│ QQuery 59 │ 195.14 / 197.90 ±1.62 / 199.68 ms │ 197.72 / 198.79 ±0.97 / 200.44 ms │ no change │
│ QQuery 60 │ 141.46 / 144.04 ±1.51 / 145.81 ms │ 143.12 / 145.76 ±1.95 / 148.36 ms │ no change │
│ QQuery 61 │ 12.81 / 13.25 ±0.35 / 13.73 ms │ 13.22 / 13.38 ±0.21 / 13.77 ms │ no change │
│ QQuery 62 │ 909.48 / 954.30 ±32.74 / 991.20 ms │ 915.33 / 938.77 ±17.12 / 960.94 ms │ no change │
│ QQuery 63 │ 104.55 / 106.29 ±2.46 / 111.16 ms │ 102.42 / 106.12 ±3.53 / 110.40 ms │ no change │
│ QQuery 64 │ 688.64 / 692.15 ±3.52 / 698.49 ms │ 685.34 / 689.23 ±2.55 / 692.06 ms │ no change │
│ QQuery 65 │ 250.53 / 252.63 ±1.07 / 253.43 ms │ 251.35 / 254.53 ±2.39 / 258.59 ms │ no change │
│ QQuery 66 │ 241.06 / 254.64 ±9.60 / 269.75 ms │ 253.77 / 258.12 ±4.61 / 265.96 ms │ no change │
│ QQuery 67 │ 312.57 / 318.86 ±6.16 / 327.23 ms │ 311.84 / 318.74 ±4.83 / 325.21 ms │ no change │
│ QQuery 68 │ 8.48 / 10.13 ±2.02 / 14.04 ms │ 9.27 / 10.72 ±1.37 / 13.29 ms │ 1.06x slower │
│ QQuery 69 │ 100.52 / 103.11 ±2.24 / 107.22 ms │ 102.02 / 105.21 ±2.30 / 108.30 ms │ no change │
│ QQuery 70 │ 332.36 / 348.22 ±12.28 / 369.63 ms │ 333.26 / 345.30 ±10.44 / 364.05 ms │ no change │
│ QQuery 71 │ 135.24 / 137.94 ±2.01 / 140.93 ms │ 134.94 / 137.48 ±1.74 / 139.73 ms │ no change │
│ QQuery 72 │ 607.18 / 627.43 ±15.78 / 643.76 ms │ 615.91 / 621.33 ±5.21 / 630.62 ms │ no change │
│ QQuery 73 │ 6.57 / 7.07 ±0.34 / 7.62 ms │ 7.17 / 7.57 ±0.32 / 8.13 ms │ 1.07x slower │
│ QQuery 74 │ 594.34 / 607.43 ±6.55 / 611.30 ms │ 583.89 / 599.03 ±8.35 / 609.09 ms │ no change │
│ QQuery 75 │ 276.87 / 277.95 ±1.33 / 280.47 ms │ 277.36 / 277.57 ±0.19 / 277.84 ms │ no change │
│ QQuery 76 │ 132.91 / 134.28 ±1.27 / 136.43 ms │ 131.14 / 133.16 ±1.85 / 136.26 ms │ no change │
│ QQuery 77 │ 188.39 / 190.19 ±0.97 / 191.30 ms │ 188.33 / 191.63 ±2.22 / 194.21 ms │ no change │
│ QQuery 78 │ 339.58 / 344.01 ±3.66 / 348.96 ms │ 340.34 / 344.02 ±2.91 / 348.83 ms │ no change │
│ QQuery 79 │ 233.08 / 236.39 ±3.54 / 242.04 ms │ 233.71 / 238.08 ±2.78 / 242.04 ms │ no change │
│ QQuery 80 │ 322.50 / 326.84 ±3.03 / 329.91 ms │ 321.35 / 325.52 ±2.99 / 330.62 ms │ no change │
│ QQuery 81 │ 26.95 / 27.65 ±0.68 / 28.52 ms │ 26.25 / 27.46 ±1.28 / 29.92 ms │ no change │
│ QQuery 82 │ 202.32 / 203.66 ±0.71 / 204.40 ms │ 198.88 / 203.36 ±3.44 / 206.93 ms │ no change │
│ QQuery 83 │ 41.14 / 41.68 ±0.55 / 42.73 ms │ 39.16 / 40.09 ±0.97 / 41.70 ms │ no change │
│ QQuery 84 │ 48.48 / 49.84 ±1.08 / 51.46 ms │ 48.30 / 49.71 ±0.95 / 50.90 ms │ no change │
│ QQuery 85 │ 149.40 / 150.28 ±0.85 / 151.58 ms │ 147.77 / 148.87 ±0.70 / 149.63 ms │ no change │
│ QQuery 86 │ 39.68 / 40.68 ±0.56 / 41.32 ms │ 39.18 / 40.10 ±0.51 / 40.59 ms │ no change │
│ QQuery 87 │ 84.70 / 88.53 ±3.34 / 94.56 ms │ 85.57 / 88.23 ±3.42 / 94.98 ms │ no change │
│ QQuery 88 │ 100.19 / 104.25 ±6.66 / 117.54 ms │ 99.73 / 103.41 ±5.12 / 113.48 ms │ no change │
│ QQuery 89 │ 119.52 / 121.03 ±0.99 / 122.00 ms │ 116.68 / 119.06 ±1.50 / 120.66 ms │ no change │
│ QQuery 90 │ 23.95 / 24.67 ±0.72 / 26.02 ms │ 22.90 / 24.08 ±0.62 / 24.71 ms │ no change │
│ QQuery 91 │ 60.46 / 64.27 ±2.08 / 66.61 ms │ 64.92 / 65.67 ±0.81 / 67.09 ms │ no change │
│ QQuery 92 │ 57.57 / 58.50 ±0.56 / 59.35 ms │ 57.84 / 58.87 ±1.08 / 60.65 ms │ no change │
│ QQuery 93 │ 185.97 / 189.04 ±2.30 / 192.19 ms │ 185.09 / 188.07 ±2.17 / 191.20 ms │ no change │
│ QQuery 94 │ 60.87 / 61.63 ±0.72 / 62.74 ms │ 61.16 / 62.37 ±1.10 / 64.46 ms │ no change │
│ QQuery 95 │ 129.79 / 131.26 ±1.18 / 133.40 ms │ 128.31 / 130.66 ±1.55 / 132.52 ms │ no change │
│ QQuery 96 │ 73.89 / 74.50 ±0.51 / 75.17 ms │ 69.19 / 72.63 ±1.82 / 74.33 ms │ no change │
│ QQuery 97 │ 123.77 / 126.59 ±2.56 / 131.22 ms │ 127.29 / 128.18 ±0.69 / 129.42 ms │ no change │
│ QQuery 98 │ 153.68 / 155.63 ±1.53 / 158.33 ms │ 152.21 / 155.89 ±2.46 / 158.87 ms │ no change │
│ QQuery 99 │ 10756.15 / 10795.01 ±21.85 / 10819.43 ms │ 10757.58 / 10789.46 ±24.20 / 10828.29 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31563.86ms │
│ Total Time (morsel-scheduler) │ 31563.11ms │
│ Average Time (HEAD) │ 318.83ms │
│ Average Time (morsel-scheduler) │ 318.82ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 5 │
│ Queries with No Change │ 93 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 158.2s |
| Peak memory | 5.7 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 260.8s |
| CPU sys | 17.0s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 158.1s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 260.8s |
| CPU sys | 17.4s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273785813
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.19 / 4.36 ±6.26 / 16.87 ms │ 1.25 / 4.43 ±6.30 / 17.03 ms │ no change │
│ QQuery 1 │ 14.82 / 14.97 ±0.09 / 15.07 ms │ 13.28 / 13.64 ±0.44 / 14.51 ms │ +1.10x faster │
│ QQuery 2 │ 45.04 / 45.64 ±0.52 / 46.58 ms │ 44.04 / 45.05 ±0.88 / 46.30 ms │ no change │
│ QQuery 3 │ 41.66 / 43.53 ±1.80 / 46.43 ms │ 40.60 / 46.47 ±3.21 / 49.13 ms │ 1.07x slower │
│ QQuery 4 │ 278.14 / 285.87 ±5.18 / 293.08 ms │ 1916.28 / 2026.95 ±100.66 / 2162.53 ms │ 7.09x slower │
│ QQuery 5 │ 328.96 / 335.53 ±3.50 / 339.39 ms │ 1930.81 / 2035.39 ±55.51 / 2087.83 ms │ 6.07x slower │
│ QQuery 6 │ 4.90 / 5.80 ±0.78 / 7.13 ms │ 4.11 / 4.82 ±0.92 / 6.52 ms │ +1.20x faster │
│ QQuery 7 │ 16.99 / 17.84 ±0.88 / 19.48 ms │ 60.07 / 61.65 ±1.41 / 64.28 ms │ 3.46x slower │
│ QQuery 8 │ 419.84 / 426.55 ±5.14 / 434.13 ms │ 3845.69 / 3940.60 ±57.40 / 4012.46 ms │ 9.24x slower │
│ QQuery 9 │ 636.88 / 646.54 ±5.27 / 651.92 ms │ 3710.13 / 3778.38 ±36.98 / 3816.00 ms │ 5.84x slower │
│ QQuery 10 │ 91.14 / 93.79 ±2.33 / 96.66 ms │ 460.23 / 470.46 ±6.59 / 477.99 ms │ 5.02x slower │
│ QQuery 11 │ 103.49 / 105.30 ±1.61 / 107.69 ms │ 500.88 / 517.16 ±11.05 / 528.20 ms │ 4.91x slower │
│ QQuery 12 │ 332.83 / 343.16 ±8.62 / 356.24 ms │ 1721.01 / 1802.90 ±45.78 / 1851.66 ms │ 5.25x slower │
│ QQuery 13 │ 443.81 / 466.64 ±17.20 / 488.75 ms │ 2618.56 / 2656.94 ±27.87 / 2701.26 ms │ 5.69x slower │
│ QQuery 14 │ 339.64 / 344.06 ±3.62 / 349.04 ms │ 1782.40 / 1808.14 ±19.64 / 1839.84 ms │ 5.26x slower │
│ QQuery 15 │ 343.82 / 359.82 ±16.93 / 390.12 ms │ 2219.35 / 2238.93 ±30.43 / 2299.27 ms │ 6.22x slower │
│ QQuery 16 │ 701.78 / 710.44 ±6.59 / 718.20 ms │ 4362.28 / 4435.23 ±51.74 / 4517.67 ms │ 6.24x slower │
│ QQuery 17 │ 695.92 / 705.40 ±7.02 / 716.92 ms │ 4404.67 / 4431.03 ±22.99 / 4465.62 ms │ 6.28x slower │
│ QQuery 18 │ 1400.70 / 1454.88 ±35.22 / 1488.13 ms │ 7641.28 / 7741.01 ±75.76 / 7819.26 ms │ 5.32x slower │
│ QQuery 19 │ 34.98 / 47.74 ±22.82 / 93.30 ms │ 34.59 / 35.82 ±1.14 / 37.16 ms │ +1.33x faster │
│ QQuery 20 │ 715.54 / 737.91 ±25.25 / 775.64 ms │ 716.34 / 731.54 ±10.26 / 744.43 ms │ no change │
│ QQuery 21 │ 754.81 / 762.64 ±4.52 / 766.76 ms │ 1797.95 / 1884.62 ±96.80 / 2069.25 ms │ 2.47x slower │
│ QQuery 22 │ 1122.99 / 1129.16 ±6.57 / 1141.47 ms │ 2249.31 / 2325.61 ±71.94 / 2462.41 ms │ 2.06x slower │
│ QQuery 23 │ 3035.25 / 3070.03 ±19.95 / 3094.00 ms │ 3105.59 / 3125.35 ±21.32 / 3161.02 ms │ no change │
│ QQuery 24 │ 100.97 / 102.69 ±1.00 / 103.94 ms │ 95.85 / 99.95 ±2.29 / 102.55 ms │ no change │
│ QQuery 25 │ 138.29 / 140.30 ±1.52 / 142.22 ms │ 137.85 / 142.80 ±4.62 / 150.30 ms │ no change │
│ QQuery 26 │ 100.26 / 102.19 ±1.48 / 104.23 ms │ 97.36 / 99.59 ±1.41 / 101.15 ms │ no change │
│ QQuery 27 │ 845.04 / 849.40 ±6.35 / 861.80 ms │ 3373.55 / 3426.86 ±32.23 / 3463.77 ms │ 4.03x slower │
│ QQuery 28 │ 3219.77 / 3249.54 ±15.98 / 3266.75 ms │ 28406.93 / 28575.09 ±105.00 / 28704.24 ms │ 8.79x slower │
│ QQuery 29 │ 51.63 / 54.07 ±2.63 / 57.91 ms │ 50.56 / 57.24 ±5.49 / 67.09 ms │ 1.06x slower │
│ QQuery 30 │ 355.78 / 365.08 ±7.72 / 374.40 ms │ 1621.00 / 1662.77 ±32.68 / 1702.82 ms │ 4.55x slower │
│ QQuery 31 │ 366.05 / 380.97 ±10.04 / 395.87 ms │ 959.36 / 972.16 ±11.85 / 989.58 ms │ 2.55x slower │
│ QQuery 32 │ 1249.66 / 1297.45 ±43.27 / 1362.80 ms │ 3141.30 / 3175.09 ±35.84 / 3240.55 ms │ 2.45x slower │
│ QQuery 33 │ 1420.03 / 1452.22 ±43.60 / 1537.33 ms │ 7790.46 / 8010.96 ±145.23 / 8240.79 ms │ 5.52x slower │
│ QQuery 34 │ 1397.41 / 1422.15 ±16.14 / 1446.54 ms │ 7635.48 / 8049.88 ±363.17 / 8606.36 ms │ 5.66x slower │
│ QQuery 35 │ 376.49 / 379.56 ±1.97 / 382.71 ms │ 2292.35 / 2363.45 ±71.15 / 2495.92 ms │ 6.23x slower │
│ QQuery 36 │ 110.78 / 120.35 ±5.18 / 126.45 ms │ 87.48 / 105.01 ±13.74 / 129.09 ms │ +1.15x faster │
│ QQuery 37 │ 46.54 / 47.84 ±0.93 / 49.12 ms │ 46.49 / 50.22 ±3.04 / 55.21 ms │ no change │
│ QQuery 38 │ 74.85 / 75.57 ±0.77 / 76.98 ms │ 69.38 / 72.31 ±1.92 / 74.33 ms │ no change │
│ QQuery 39 │ 206.39 / 213.34 ±6.90 / 224.61 ms │ 174.98 / 183.98 ±6.74 / 195.37 ms │ +1.16x faster │
│ QQuery 40 │ 21.52 / 24.28 ±1.49 / 25.38 ms │ 18.52 / 20.71 ±1.74 / 23.37 ms │ +1.17x faster │
│ QQuery 41 │ 19.56 / 21.99 ±1.68 / 24.45 ms │ 15.56 / 17.74 ±3.01 / 23.55 ms │ +1.24x faster │
│ QQuery 42 │ 18.38 / 19.82 ±1.21 / 22.05 ms │ 17.24 / 18.30 ±1.25 / 20.73 ms │ +1.08x faster │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22476.40ms │
│ Total Time (morsel-scheduler) │ 103266.21ms │
│ Average Time (HEAD) │ 522.71ms │
│ Average Time (morsel-scheduler) │ 2401.54ms │
│ Queries Faster │ 8 │
│ Queries Slower │ 26 │
│ Queries with No Change │ 9 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴─────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.4s |
| Peak memory | 39.7 GiB |
| Avg memory | 26.9 GiB |
| CPU user | 1055.0s |
| CPU sys | 96.6s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 517.6s |
| Peak memory | 37.0 GiB |
| Avg memory | 29.8 GiB |
| CPU user | 1036.3s |
| CPU sys | 131.6s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273756125
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan closed pull request #21718: feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype
URL: https://github.com/apache/datafusion/pull/21718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan closed pull request #21718: feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype
URL: https://github.com/apache/datafusion/pull/21718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273902663
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.60 / 7.12 ±0.84 / 8.79 ms │ 6.67 / 7.07 ±0.75 / 8.56 ms │ no change │
│ QQuery 2 │ 145.99 / 147.82 ±1.40 / 150.30 ms │ 145.41 / 147.69 ±1.41 / 149.34 ms │ no change │
│ QQuery 3 │ 114.31 / 114.74 ±0.29 / 115.09 ms │ 113.73 / 114.82 ±0.87 / 115.97 ms │ no change │
│ QQuery 4 │ 1310.27 / 1347.88 ±20.85 / 1371.25 ms │ 1316.78 / 1344.00 ±20.56 / 1380.51 ms │ no change │
│ QQuery 5 │ 173.01 / 175.31 ±1.72 / 178.35 ms │ 172.26 / 174.29 ±1.49 / 176.02 ms │ no change │
│ QQuery 6 │ 845.91 / 861.90 ±8.97 / 871.57 ms │ 854.57 / 879.54 ±22.34 / 918.11 ms │ no change │
│ QQuery 7 │ 341.25 / 344.39 ±2.49 / 347.11 ms │ 339.97 / 342.39 ±2.55 / 346.59 ms │ no change │
│ QQuery 8 │ 115.54 / 116.36 ±0.55 / 117.04 ms │ 114.60 / 116.92 ±1.83 / 119.77 ms │ no change │
│ QQuery 9 │ 100.67 / 106.65 ±5.63 / 116.40 ms │ 100.08 / 102.88 ±2.22 / 106.77 ms │ no change │
│ QQuery 10 │ 105.37 / 106.86 ±1.10 / 108.34 ms │ 105.12 / 105.65 ±0.55 / 106.58 ms │ no change │
│ QQuery 11 │ 943.38 / 953.81 ±6.61 / 964.10 ms │ 948.82 / 956.50 ±7.77 / 969.55 ms │ no change │
│ QQuery 12 │ 44.70 / 46.65 ±1.25 / 47.83 ms │ 47.27 / 48.25 ±1.35 / 50.89 ms │ no change │
│ QQuery 13 │ 398.68 / 404.01 ±3.33 / 409.14 ms │ 398.15 / 402.60 ±2.58 / 405.83 ms │ no change │
│ QQuery 14 │ 994.12 / 1012.20 ±10.34 / 1021.36 ms │ 998.25 / 1007.63 ±8.74 / 1017.95 ms │ no change │
│ QQuery 15 │ 15.58 / 16.69 ±0.68 / 17.37 ms │ 16.74 / 17.66 ±0.85 / 19.07 ms │ 1.06x slower │
│ QQuery 16 │ 6.94 / 7.64 ±0.92 / 9.42 ms │ 7.38 / 7.71 ±0.44 / 8.58 ms │ no change │
│ QQuery 17 │ 231.27 / 232.56 ±0.78 / 233.47 ms │ 229.23 / 231.07 ±1.21 / 232.91 ms │ no change │
│ QQuery 18 │ 126.68 / 128.12 ±0.93 / 129.56 ms │ 127.46 / 128.85 ±1.30 / 131.14 ms │ no change │
│ QQuery 19 │ 154.33 / 156.11 ±1.03 / 156.99 ms │ 155.00 / 155.59 ±0.46 / 156.19 ms │ no change │
│ QQuery 20 │ 13.66 / 14.71 ±0.91 / 16.40 ms │ 13.46 / 14.29 ±0.57 / 15.19 ms │ no change │
│ QQuery 21 │ 19.31 / 20.05 ±0.43 / 20.57 ms │ 19.35 / 20.28 ±0.62 / 21.01 ms │ no change │
│ QQuery 22 │ 474.37 / 478.01 ±4.16 / 486.10 ms │ 486.57 / 488.90 ±2.40 / 493.16 ms │ no change │
│ QQuery 23 │ 877.52 / 890.37 ±11.70 / 906.75 ms │ 874.76 / 880.55 ±4.15 / 886.31 ms │ no change │
│ QQuery 24 │ 384.32 / 387.43 ±2.13 / 390.25 ms │ 381.10 / 385.47 ±5.01 / 395.04 ms │ no change │
│ QQuery 25 │ 340.78 / 343.56 ±1.40 / 344.47 ms │ 340.30 / 342.80 ±1.77 / 345.76 ms │ no change │
│ QQuery 26 │ 80.02 / 82.26 ±1.99 / 85.60 ms │ 79.25 / 81.94 ±1.94 / 85.24 ms │ no change │
│ QQuery 27 │ 6.82 / 7.08 ±0.21 / 7.36 ms │ 6.88 / 7.72 ±1.02 / 9.63 ms │ 1.09x slower │
│ QQuery 28 │ 149.13 / 149.69 ±0.47 / 150.50 ms │ 147.69 / 149.30 ±1.20 / 151.39 ms │ no change │
│ QQuery 29 │ 283.08 / 285.23 ±2.11 / 289.20 ms │ 280.21 / 282.62 ±1.86 / 285.96 ms │ no change │
│ QQuery 30 │ 44.98 / 46.02 ±1.24 / 48.46 ms │ 42.81 / 44.33 ±0.96 / 45.74 ms │ no change │
│ QQuery 31 │ 171.65 / 173.59 ±2.62 / 178.66 ms │ 170.34 / 171.72 ±0.95 / 173.21 ms │ no change │
│ QQuery 32 │ 58.70 / 59.65 ±1.05 / 61.55 ms │ 57.49 / 66.72 ±17.29 / 101.28 ms │ 1.12x slower │
│ QQuery 33 │ 141.42 / 142.81 ±1.00 / 144.17 ms │ 139.63 / 141.50 ±1.29 / 143.39 ms │ no change │
│ QQuery 34 │ 6.99 / 7.57 ±0.44 / 8.12 ms │ 6.92 / 7.53 ±0.73 / 8.83 ms │ no change │
│ QQuery 35 │ 106.63 / 109.25 ±1.41 / 110.66 ms │ 107.85 / 108.16 ±0.24 / 108.43 ms │ no change │
│ QQuery 36 │ 6.65 / 6.78 ±0.14 / 7.06 ms │ 6.64 / 6.88 ±0.19 / 7.21 ms │ no change │
│ QQuery 37 │ 8.46 / 9.00 ±0.52 / 9.97 ms │ 8.13 / 8.63 ±0.38 / 9.15 ms │ no change │
│ QQuery 38 │ 86.70 / 88.48 ±1.69 / 91.58 ms │ 81.90 / 85.21 ±4.50 / 94.09 ms │ no change │
│ QQuery 39 │ 126.58 / 129.79 ±3.15 / 135.67 ms │ 124.03 / 128.01 ±2.52 / 130.75 ms │ no change │
│ QQuery 40 │ 107.22 / 114.04 ±7.39 / 128.19 ms │ 107.81 / 113.84 ±7.86 / 128.86 ms │ no change │
│ QQuery 41 │ 14.70 / 16.17 ±0.85 / 17.33 ms │ 15.00 / 16.27 ±1.07 / 18.08 ms │ no change │
│ QQuery 42 │ 108.65 / 109.46 ±0.88 / 111.14 ms │ 106.31 / 109.17 ±1.80 / 111.39 ms │ no change │
│ QQuery 43 │ 5.97 / 6.14 ±0.18 / 6.48 ms │ 5.86 / 6.11 ±0.19 / 6.45 ms │ no change │
│ QQuery 44 │ 11.58 / 12.14 ±0.87 / 13.87 ms │ 11.97 / 12.19 ±0.25 / 12.67 ms │ no change │
│ QQuery 45 │ 50.36 / 51.41 ±0.81 / 52.67 ms │ 50.81 / 52.00 ±0.69 / 52.69 ms │ no change │
│ QQuery 46 │ 8.51 / 8.90 ±0.56 / 10.00 ms │ 8.64 / 8.84 ±0.16 / 9.09 ms │ no change │
│ QQuery 47 │ 730.82 / 737.08 ±5.06 / 744.56 ms │ 716.48 / 722.46 ±5.23 / 732.10 ms │ no change │
│ QQuery 48 │ 285.51 / 291.97 ±6.06 / 301.98 ms │ 280.06 / 291.01 ±6.12 / 298.11 ms │ no change │
│ QQuery 49 │ 251.78 / 256.01 ±2.51 / 258.06 ms │ 252.16 / 253.17 ±1.23 / 255.28 ms │ no change │
│ QQuery 50 │ 223.68 / 229.99 ±4.08 / 236.09 ms │ 220.58 / 224.42 ±2.83 / 228.42 ms │ no change │
│ QQuery 51 │ 182.20 / 184.81 ±1.81 / 187.58 ms │ 179.28 / 182.60 ±2.51 / 186.08 ms │ no change │
│ QQuery 52 │ 107.71 / 109.38 ±1.29 / 110.74 ms │ 107.84 / 109.04 ±0.82 / 110.34 ms │ no change │
│ QQuery 53 │ 101.65 / 103.54 ±1.07 / 104.86 ms │ 103.20 / 105.40 ±2.01 / 108.56 ms │ no change │
│ QQuery 54 │ 145.80 / 147.98 ±1.93 / 150.45 ms │ 146.49 / 147.53 ±1.10 / 149.59 ms │ no change │
│ QQuery 55 │ 106.97 / 110.05 ±2.36 / 113.69 ms │ 106.94 / 108.20 ±1.49 / 111.06 ms │ no change │
│ QQuery 56 │ 141.46 / 142.72 ±1.24 / 144.87 ms │ 140.28 / 142.13 ±1.08 / 143.60 ms │ no change │
│ QQuery 57 │ 173.52 / 175.45 ±2.35 / 180.08 ms │ 172.70 / 175.01 ±1.83 / 177.48 ms │ no change │
│ QQuery 58 │ 290.48 / 300.48 ±8.41 / 315.55 ms │ 287.09 / 294.02 ±6.09 / 301.73 ms │ no change │
│ QQuery 59 │ 197.53 / 199.45 ±1.31 / 201.40 ms │ 196.14 / 199.00 ±3.33 / 205.37 ms │ no change │
│ QQuery 60 │ 143.28 / 145.73 ±2.03 / 147.84 ms │ 143.34 / 145.61 ±1.59 / 147.42 ms │ no change │
│ QQuery 61 │ 12.98 / 13.18 ±0.17 / 13.42 ms │ 13.09 / 13.37 ±0.21 / 13.60 ms │ no change │
│ QQuery 62 │ 879.61 / 921.20 ±27.79 / 953.09 ms │ 885.14 / 899.38 ±8.25 / 910.73 ms │ no change │
│ QQuery 63 │ 103.02 / 105.89 ±1.91 / 108.79 ms │ 104.08 / 105.54 ±1.08 / 106.74 ms │ no change │
│ QQuery 64 │ 686.68 / 690.45 ±2.25 / 693.04 ms │ 682.08 / 688.41 ±4.19 / 693.66 ms │ no change │
│ QQuery 65 │ 251.11 / 255.79 ±3.00 / 259.51 ms │ 247.76 / 256.07 ±4.99 / 262.06 ms │ no change │
│ QQuery 66 │ 246.40 / 254.89 ±9.62 / 272.49 ms │ 236.96 / 255.12 ±12.73 / 276.38 ms │ no change │
│ QQuery 67 │ 310.81 / 319.27 ±8.02 / 333.45 ms │ 312.62 / 320.44 ±4.66 / 325.19 ms │ no change │
│ QQuery 68 │ 8.99 / 9.98 ±0.65 / 10.94 ms │ 8.79 / 9.94 ±1.21 / 12.18 ms │ no change │
│ QQuery 69 │ 101.09 / 103.97 ±2.10 / 106.55 ms │ 100.73 / 103.04 ±1.85 / 105.40 ms │ no change │
│ QQuery 70 │ 330.77 / 347.07 ±15.94 / 375.83 ms │ 338.18 / 349.74 ±8.55 / 363.92 ms │ no change │
│ QQuery 71 │ 135.99 / 137.27 ±0.89 / 138.60 ms │ 136.75 / 138.40 ±1.64 / 141.06 ms │ no change │
│ QQuery 72 │ 618.68 / 627.61 ±6.53 / 636.53 ms │ 609.55 / 623.03 ±11.22 / 634.73 ms │ no change │
│ QQuery 73 │ 7.09 / 8.08 ±0.55 / 8.62 ms │ 7.08 / 8.23 ±0.81 / 9.58 ms │ no change │
│ QQuery 74 │ 595.61 / 600.80 ±4.89 / 609.91 ms │ 597.66 / 601.97 ±2.64 / 605.96 ms │ no change │
│ QQuery 75 │ 276.58 / 279.08 ±2.16 / 281.75 ms │ 275.22 / 279.17 ±2.36 / 282.13 ms │ no change │
│ QQuery 76 │ 131.89 / 133.23 ±1.25 / 135.60 ms │ 132.11 / 133.79 ±1.46 / 136.25 ms │ no change │
│ QQuery 77 │ 189.89 / 192.00 ±1.43 / 193.65 ms │ 187.63 / 190.40 ±1.88 / 193.40 ms │ no change │
│ QQuery 78 │ 339.88 / 344.50 ±4.01 / 351.08 ms │ 337.57 / 343.73 ±3.32 / 346.63 ms │ no change │
│ QQuery 79 │ 234.43 / 237.64 ±2.35 / 241.42 ms │ 235.46 / 236.90 ±1.06 / 238.49 ms │ no change │
│ QQuery 80 │ 320.98 / 325.81 ±3.98 / 331.08 ms │ 323.80 / 324.75 ±0.83 / 326.01 ms │ no change │
│ QQuery 81 │ 26.72 / 27.67 ±0.94 / 28.90 ms │ 26.65 / 27.33 ±0.76 / 28.76 ms │ no change │
│ QQuery 82 │ 201.99 / 205.01 ±1.93 / 206.98 ms │ 199.78 / 201.05 ±1.65 / 204.27 ms │ no change │
│ QQuery 83 │ 38.82 / 40.97 ±2.06 / 44.61 ms │ 39.07 / 40.71 ±1.14 / 42.20 ms │ no change │
│ QQuery 84 │ 49.47 / 49.97 ±0.36 / 50.48 ms │ 48.35 / 49.21 ±0.74 / 50.08 ms │ no change │
│ QQuery 85 │ 148.41 / 149.55 ±0.71 / 150.65 ms │ 148.69 / 150.47 ±1.43 / 152.63 ms │ no change │
│ QQuery 86 │ 39.04 / 39.75 ±0.48 / 40.39 ms │ 38.04 / 39.70 ±1.14 / 40.98 ms │ no change │
│ QQuery 87 │ 86.43 / 88.53 ±2.51 / 93.34 ms │ 84.10 / 87.96 ±3.76 / 94.92 ms │ no change │
│ QQuery 88 │ 99.82 / 100.92 ±0.76 / 102.09 ms │ 99.66 / 100.36 ±0.60 / 101.31 ms │ no change │
│ QQuery 89 │ 118.00 / 120.18 ±1.41 / 121.95 ms │ 118.64 / 119.85 ±0.99 / 121.36 ms │ no change │
│ QQuery 90 │ 23.71 / 24.29 ±0.41 / 24.79 ms │ 24.09 / 24.91 ±0.74 / 25.93 ms │ no change │
│ QQuery 91 │ 63.00 / 64.18 ±1.20 / 65.88 ms │ 63.15 / 64.50 ±1.16 / 66.07 ms │ no change │
│ QQuery 92 │ 58.36 / 59.58 ±0.89 / 60.68 ms │ 58.39 / 59.15 ±0.52 / 59.78 ms │ no change │
│ QQuery 93 │ 186.60 / 188.58 ±1.41 / 190.48 ms │ 188.16 / 190.78 ±1.47 / 192.41 ms │ no change │
│ QQuery 94 │ 61.94 / 62.45 ±0.69 / 63.80 ms │ 62.43 / 63.00 ±0.32 / 63.39 ms │ no change │
│ QQuery 95 │ 130.26 / 131.93 ±1.50 / 133.81 ms │ 129.60 / 131.15 ±1.42 / 133.07 ms │ no change │
│ QQuery 96 │ 74.34 / 74.99 ±0.47 / 75.69 ms │ 71.42 / 74.46 ±1.68 / 76.35 ms │ no change │
│ QQuery 97 │ 126.35 / 127.32 ±0.87 / 128.83 ms │ 126.38 / 128.43 ±1.59 / 130.34 ms │ no change │
│ QQuery 98 │ 153.94 / 155.02 ±0.79 / 156.33 ms │ 154.93 / 157.41 ±2.05 / 161.04 ms │ no change │
│ QQuery 99 │ 10806.18 / 10829.11 ±16.46 / 10855.00 ms │ 10740.92 / 10805.60 ±36.85 / 10847.25 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31618.77ms │
│ Total Time (morsel-scheduler) │ 31529.18ms │
│ Average Time (HEAD) │ 319.38ms │
│ Average Time (morsel-scheduler) │ 318.48ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 96 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 158.4s |
| Peak memory | 5.0 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 260.9s |
| CPU sys | 17.3s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 157.9s |
| Peak memory | 5.6 GiB |
| Avg memory | 4.6 GiB |
| CPU user | 260.5s |
| CPU sys | 16.9s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273867327
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273866779)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273866779-1477-r6zmh 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (f2cf29a3af1abe7e0996013f65a5becef70cad25) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..f2cf29a3af1abe7e0996013f65a5becef70cad25) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273829062
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273827001)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4273827001-1474-hgpm7 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (99e872a19bade26aa57b201022f8a3f9b86692a8) to 5c653be (merge-base) [diff](https://github.com/apache/datafusion/compare/5c653bee5da64003915f6dfeb3da15759b091a8d..99e872a19bade26aa57b201022f8a3f9b86692a8) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275706964
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.67 / 7.11 ±0.75 / 8.60 ms │ 6.52 / 7.00 ±0.70 / 8.40 ms │ no change │
│ QQuery 2 │ 145.75 / 146.89 ±1.14 / 149.02 ms │ 146.42 / 147.37 ±0.86 / 148.49 ms │ no change │
│ QQuery 3 │ 113.26 / 114.34 ±0.56 / 114.82 ms │ 114.16 / 114.61 ±0.41 / 115.30 ms │ no change │
│ QQuery 4 │ 1262.45 / 1286.34 ±13.87 / 1299.19 ms │ 1278.08 / 1309.03 ±18.96 / 1329.98 ms │ no change │
│ QQuery 5 │ 172.91 / 173.65 ±1.12 / 175.87 ms │ 172.30 / 173.42 ±1.41 / 176.05 ms │ no change │
│ QQuery 6 │ 806.89 / 823.95 ±9.96 / 835.84 ms │ 835.32 / 840.27 ±6.08 / 851.90 ms │ no change │
│ QQuery 7 │ 343.01 / 345.66 ±2.18 / 348.69 ms │ 340.79 / 343.83 ±2.23 / 347.52 ms │ no change │
│ QQuery 8 │ 117.07 / 118.01 ±0.94 / 119.76 ms │ 115.24 / 117.47 ±1.82 / 120.17 ms │ no change │
│ QQuery 9 │ 101.44 / 103.94 ±2.58 / 107.90 ms │ 100.11 / 103.68 ±2.89 / 107.85 ms │ no change │
│ QQuery 10 │ 104.42 / 106.35 ±1.59 / 108.75 ms │ 106.05 / 107.27 ±0.84 / 108.47 ms │ no change │
│ QQuery 11 │ 895.39 / 918.13 ±21.73 / 956.00 ms │ 884.80 / 909.79 ±17.82 / 928.04 ms │ no change │
│ QQuery 12 │ 46.46 / 47.08 ±0.51 / 47.80 ms │ 43.69 / 46.34 ±1.42 / 47.71 ms │ no change │
│ QQuery 13 │ 399.22 / 402.79 ±2.06 / 405.32 ms │ 397.22 / 400.85 ±1.99 / 402.68 ms │ no change │
│ QQuery 14 │ 1004.79 / 1010.63 ±4.77 / 1016.89 ms │ 1001.29 / 1008.89 ±6.18 / 1016.40 ms │ no change │
│ QQuery 15 │ 15.13 / 15.93 ±0.92 / 17.67 ms │ 15.61 / 16.76 ±0.98 / 17.95 ms │ 1.05x slower │
│ QQuery 16 │ 6.96 / 7.64 ±0.63 / 8.79 ms │ 7.21 / 7.77 ±0.56 / 8.82 ms │ no change │
│ QQuery 17 │ 226.97 / 228.96 ±1.85 / 231.96 ms │ 228.72 / 231.56 ±1.84 / 233.47 ms │ no change │
│ QQuery 18 │ 127.19 / 129.09 ±1.59 / 131.81 ms │ 125.91 / 128.63 ±1.74 / 130.75 ms │ no change │
│ QQuery 19 │ 157.09 / 160.27 ±3.32 / 166.58 ms │ 154.72 / 157.26 ±1.95 / 160.20 ms │ no change │
│ QQuery 20 │ 14.11 / 14.58 ±0.39 / 15.12 ms │ 14.09 / 14.45 ±0.45 / 15.33 ms │ no change │
│ QQuery 21 │ 19.04 / 19.78 ±0.40 / 20.24 ms │ 19.01 / 19.49 ±0.33 / 19.89 ms │ no change │
│ QQuery 22 │ 487.45 / 493.32 ±4.51 / 499.03 ms │ 476.82 / 481.95 ±3.93 / 488.00 ms │ no change │
│ QQuery 23 │ 886.20 / 893.48 ±5.88 / 901.35 ms │ 869.75 / 879.80 ±9.27 / 894.49 ms │ no change │
│ QQuery 24 │ 384.98 / 387.83 ±1.97 / 389.87 ms │ 382.50 / 385.93 ±2.13 / 388.86 ms │ no change │
│ QQuery 25 │ 342.48 / 345.00 ±1.39 / 346.39 ms │ 339.98 / 343.20 ±1.74 / 345.27 ms │ no change │
│ QQuery 26 │ 80.88 / 83.79 ±1.64 / 85.25 ms │ 81.52 / 82.48 ±1.08 / 84.47 ms │ no change │
│ QQuery 27 │ 6.79 / 7.02 ±0.19 / 7.28 ms │ 7.03 / 7.37 ±0.44 / 8.20 ms │ 1.05x slower │
│ QQuery 28 │ 149.19 / 149.87 ±0.90 / 151.61 ms │ 148.34 / 149.36 ±1.29 / 151.87 ms │ no change │
│ QQuery 29 │ 279.67 / 282.96 ±2.58 / 286.48 ms │ 279.53 / 283.96 ±3.00 / 287.62 ms │ no change │
│ QQuery 30 │ 43.23 / 43.81 ±0.74 / 45.23 ms │ 42.89 / 44.32 ±1.13 / 45.99 ms │ no change │
│ QQuery 31 │ 169.90 / 172.92 ±1.56 / 174.33 ms │ 168.98 / 171.93 ±1.78 / 174.31 ms │ no change │
│ QQuery 32 │ 57.48 / 58.45 ±0.61 / 59.00 ms │ 55.96 / 58.16 ±1.81 / 61.11 ms │ no change │
│ QQuery 33 │ 139.61 / 141.73 ±1.65 / 143.77 ms │ 140.43 / 142.30 ±1.14 / 143.68 ms │ no change │
│ QQuery 34 │ 6.88 / 7.28 ±0.34 / 7.72 ms │ 7.10 / 7.26 ±0.17 / 7.58 ms │ no change │
│ QQuery 35 │ 107.01 / 108.32 ±0.92 / 109.89 ms │ 105.50 / 107.57 ±1.40 / 109.35 ms │ no change │
│ QQuery 36 │ 6.39 / 6.55 ±0.14 / 6.72 ms │ 6.50 / 6.71 ±0.16 / 6.89 ms │ no change │
│ QQuery 37 │ 8.12 / 8.79 ±0.36 / 9.17 ms │ 8.22 / 9.16 ±1.36 / 11.84 ms │ no change │
│ QQuery 38 │ 81.37 / 87.83 ±4.15 / 92.12 ms │ 82.45 / 86.86 ±4.74 / 95.41 ms │ no change │
│ QQuery 39 │ 127.37 / 128.69 ±0.96 / 130.07 ms │ 125.36 / 128.42 ±2.37 / 132.61 ms │ no change │
│ QQuery 40 │ 112.15 / 114.86 ±2.96 / 120.33 ms │ 110.17 / 114.77 ±6.68 / 127.87 ms │ no change │
│ QQuery 41 │ 14.25 / 15.24 ±0.98 / 17.07 ms │ 14.32 / 15.08 ±0.67 / 16.31 ms │ no change │
│ QQuery 42 │ 110.21 / 110.65 ±0.26 / 110.88 ms │ 110.35 / 111.46 ±1.18 / 113.71 ms │ no change │
│ QQuery 43 │ 5.92 / 6.04 ±0.11 / 6.25 ms │ 5.91 / 6.04 ±0.17 / 6.38 ms │ no change │
│ QQuery 44 │ 11.33 / 11.72 ±0.39 / 12.45 ms │ 11.34 / 11.64 ±0.38 / 12.36 ms │ no change │
│ QQuery 45 │ 50.57 / 51.10 ±0.71 / 52.45 ms │ 50.40 / 51.71 ±1.18 / 53.69 ms │ no change │
│ QQuery 46 │ 8.39 / 8.77 ±0.32 / 9.28 ms │ 8.16 / 8.43 ±0.22 / 8.79 ms │ no change │
│ QQuery 47 │ 694.87 / 698.71 ±3.28 / 704.11 ms │ 700.03 / 704.26 ±4.46 / 712.71 ms │ no change │
│ QQuery 48 │ 286.08 / 293.50 ±5.12 / 300.71 ms │ 283.64 / 288.21 ±2.40 / 290.18 ms │ no change │
│ QQuery 49 │ 250.23 / 252.35 ±1.20 / 253.94 ms │ 252.35 / 254.65 ±1.78 / 257.35 ms │ no change │
│ QQuery 50 │ 223.24 / 226.41 ±1.78 / 228.58 ms │ 219.92 / 228.05 ±5.98 / 234.16 ms │ no change │
│ QQuery 51 │ 180.72 / 184.00 ±4.00 / 191.66 ms │ 179.73 / 183.33 ±2.51 / 187.16 ms │ no change │
│ QQuery 52 │ 107.00 / 109.14 ±2.29 / 113.54 ms │ 106.93 / 107.54 ±0.36 / 107.93 ms │ no change │
│ QQuery 53 │ 102.80 / 104.04 ±0.72 / 105.06 ms │ 102.29 / 104.18 ±1.49 / 106.75 ms │ no change │
│ QQuery 54 │ 146.15 / 148.62 ±1.96 / 151.31 ms │ 145.36 / 145.75 ±0.33 / 146.33 ms │ no change │
│ QQuery 55 │ 106.11 / 108.97 ±2.19 / 111.66 ms │ 106.69 / 108.05 ±1.30 / 110.07 ms │ no change │
│ QQuery 56 │ 141.30 / 143.02 ±1.72 / 146.18 ms │ 138.89 / 141.68 ±1.77 / 144.12 ms │ no change │
│ QQuery 57 │ 173.71 / 176.43 ±1.63 / 178.68 ms │ 170.57 / 174.30 ±2.08 / 176.87 ms │ no change │
│ QQuery 58 │ 291.58 / 296.16 ±4.50 / 303.48 ms │ 286.57 / 295.97 ±5.38 / 302.29 ms │ no change │
│ QQuery 59 │ 198.26 / 199.28 ±1.20 / 201.33 ms │ 196.80 / 198.32 ±0.86 / 199.43 ms │ no change │
│ QQuery 60 │ 142.80 / 145.48 ±1.47 / 146.79 ms │ 143.22 / 144.71 ±1.00 / 146.21 ms │ no change │
│ QQuery 61 │ 12.88 / 13.22 ±0.27 / 13.69 ms │ 12.60 / 12.86 ±0.25 / 13.31 ms │ no change │
│ QQuery 62 │ 889.05 / 915.80 ±29.33 / 972.34 ms │ 878.13 / 943.26 ±49.48 / 1006.60 ms │ no change │
│ QQuery 63 │ 104.97 / 105.53 ±0.33 / 105.91 ms │ 103.02 / 105.95 ±3.67 / 112.93 ms │ no change │
│ QQuery 64 │ 688.85 / 695.10 ±6.85 / 706.83 ms │ 683.37 / 686.35 ±1.77 / 688.87 ms │ no change │
│ QQuery 65 │ 245.31 / 251.86 ±4.91 / 259.00 ms │ 249.72 / 253.45 ±3.86 / 260.73 ms │ no change │
│ QQuery 66 │ 233.81 / 250.25 ±12.95 / 267.86 ms │ 250.46 / 260.30 ±6.90 / 271.51 ms │ no change │
│ QQuery 67 │ 311.82 / 317.62 ±4.78 / 323.75 ms │ 302.29 / 314.06 ±8.50 / 327.77 ms │ no change │
│ QQuery 68 │ 10.07 / 11.52 ±1.31 / 13.63 ms │ 9.42 / 10.29 ±0.59 / 10.95 ms │ +1.12x faster │
│ QQuery 69 │ 102.05 / 103.60 ±2.16 / 107.78 ms │ 100.18 / 103.37 ±1.70 / 104.90 ms │ no change │
│ QQuery 70 │ 320.12 / 346.23 ±17.26 / 374.25 ms │ 331.83 / 351.93 ±14.28 / 370.37 ms │ no change │
│ QQuery 71 │ 134.24 / 136.43 ±2.26 / 140.52 ms │ 135.35 / 136.87 ±1.24 / 138.79 ms │ no change │
│ QQuery 72 │ 609.31 / 619.52 ±8.73 / 630.44 ms │ 613.72 / 618.49 ±3.88 / 623.59 ms │ no change │
│ QQuery 73 │ 6.94 / 8.05 ±1.09 / 9.98 ms │ 7.04 / 7.99 ±0.96 / 9.68 ms │ no change │
│ QQuery 74 │ 561.55 / 569.06 ±5.00 / 575.03 ms │ 557.00 / 563.20 ±4.90 / 571.73 ms │ no change │
│ QQuery 75 │ 274.51 / 277.62 ±1.76 / 279.79 ms │ 274.75 / 275.63 ±0.75 / 276.62 ms │ no change │
│ QQuery 76 │ 130.69 / 132.77 ±1.39 / 134.84 ms │ 131.35 / 132.59 ±1.08 / 134.18 ms │ no change │
│ QQuery 77 │ 187.49 / 188.45 ±0.85 / 189.77 ms │ 186.80 / 188.63 ±1.84 / 191.24 ms │ no change │
│ QQuery 78 │ 338.86 / 342.70 ±2.47 / 345.49 ms │ 343.35 / 346.01 ±1.39 / 347.19 ms │ no change │
│ QQuery 79 │ 233.07 / 236.34 ±2.65 / 240.27 ms │ 229.48 / 231.50 ±2.03 / 235.02 ms │ no change │
│ QQuery 80 │ 320.44 / 325.59 ±3.63 / 331.11 ms │ 319.69 / 323.88 ±3.61 / 328.86 ms │ no change │
│ QQuery 81 │ 26.49 / 27.97 ±1.62 / 30.93 ms │ 25.75 / 26.34 ±0.74 / 27.80 ms │ +1.06x faster │
│ QQuery 82 │ 199.96 / 201.93 ±1.60 / 203.87 ms │ 197.32 / 200.09 ±2.76 / 204.02 ms │ no change │
│ QQuery 83 │ 38.01 / 39.30 ±1.42 / 42.02 ms │ 38.09 / 39.67 ±1.60 / 42.07 ms │ no change │
│ QQuery 84 │ 47.70 / 49.08 ±0.72 / 49.64 ms │ 47.42 / 48.59 ±0.70 / 49.43 ms │ no change │
│ QQuery 85 │ 147.64 / 150.71 ±1.68 / 152.29 ms │ 147.57 / 148.75 ±0.90 / 150.24 ms │ no change │
│ QQuery 86 │ 38.09 / 39.69 ±1.34 / 41.56 ms │ 39.10 / 40.38 ±1.17 / 42.46 ms │ no change │
│ QQuery 87 │ 84.78 / 87.46 ±3.17 / 93.23 ms │ 83.70 / 87.48 ±3.38 / 93.45 ms │ no change │
│ QQuery 88 │ 99.76 / 100.63 ±0.60 / 101.42 ms │ 98.96 / 100.84 ±1.07 / 102.15 ms │ no change │
│ QQuery 89 │ 119.97 / 120.91 ±0.59 / 121.71 ms │ 118.16 / 119.82 ±1.46 / 121.66 ms │ no change │
│ QQuery 90 │ 23.62 / 23.91 ±0.26 / 24.28 ms │ 23.41 / 23.91 ±0.32 / 24.39 ms │ no change │
│ QQuery 91 │ 63.23 / 64.81 ±1.30 / 66.86 ms │ 63.17 / 64.10 ±0.85 / 65.12 ms │ no change │
│ QQuery 92 │ 57.17 / 58.11 ±1.05 / 60.08 ms │ 56.97 / 57.77 ±0.78 / 59.15 ms │ no change │
│ QQuery 93 │ 185.08 / 189.21 ±2.40 / 191.61 ms │ 185.73 / 186.58 ±0.97 / 188.00 ms │ no change │
│ QQuery 94 │ 60.37 / 60.85 ±0.42 / 61.49 ms │ 61.09 / 61.53 ±0.31 / 61.94 ms │ no change │
│ QQuery 95 │ 128.38 / 130.07 ±1.07 / 131.76 ms │ 127.85 / 129.30 ±1.00 / 130.57 ms │ no change │
│ QQuery 96 │ 71.71 / 73.66 ±1.05 / 74.77 ms │ 72.03 / 73.28 ±0.79 / 74.39 ms │ no change │
│ QQuery 97 │ 125.29 / 128.46 ±2.11 / 130.62 ms │ 124.92 / 126.74 ±1.53 / 129.30 ms │ no change │
│ QQuery 98 │ 154.33 / 156.93 ±2.47 / 161.38 ms │ 152.97 / 154.30 ±0.93 / 155.46 ms │ no change │
│ QQuery 99 │ 10658.85 / 10700.96 ±32.70 / 10737.35 ms │ 10709.05 / 10748.52 ±23.71 / 10770.78 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 31245.18ms │
│ Total Time (morsel-scheduler) │ 31285.26ms │
│ Average Time (HEAD) │ 315.61ms │
│ Average Time (morsel-scheduler) │ 316.01ms │
│ Queries Faster │ 2 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 95 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 156.6s |
| Peak memory | 5.5 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 258.5s |
| CPU sys | 17.7s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 156.7s |
| Peak memory | 6.0 GiB |
| Avg memory | 4.5 GiB |
| CPU user | 258.7s |
| CPU sys | 17.5s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806
run benchmarks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273936494
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273898806)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.21 / 4.46 ±6.36 / 17.19 ms │ 1.24 / 4.55 ±6.45 / 17.46 ms │ no change │
│ QQuery 1 │ 14.30 / 14.84 ±0.29 / 15.15 ms │ 24.87 / 29.01 ±2.65 / 31.62 ms │ 1.96x slower │
│ QQuery 2 │ 45.02 / 45.41 ±0.30 / 45.91 ms │ 85.67 / 97.46 ±9.30 / 108.26 ms │ 2.15x slower │
│ QQuery 3 │ 42.58 / 44.62 ±1.85 / 46.81 ms │ 78.85 / 106.36 ±18.56 / 136.00 ms │ 2.38x slower │
│ QQuery 4 │ 294.44 / 299.91 ±5.64 / 310.58 ms │ 718.68 / 951.52 ±180.89 / 1243.39 ms │ 3.17x slower │
│ QQuery 5 │ 340.41 / 344.30 ±3.13 / 348.03 ms │ 604.28 / 720.80 ±146.14 / 1008.74 ms │ 2.09x slower │
│ QQuery 6 │ 5.48 / 6.19 ±0.91 / 7.93 ms │ 3.68 / 4.93 ±0.95 / 6.53 ms │ +1.26x faster │
│ QQuery 7 │ 16.79 / 17.03 ±0.21 / 17.41 ms │ 26.24 / 32.14 ±6.75 / 44.98 ms │ 1.89x slower │
│ QQuery 8 │ 414.40 / 420.59 ±4.18 / 424.77 ms │ 1106.57 / 1360.46 ±243.12 / 1695.43 ms │ 3.23x slower │
│ QQuery 9 │ 654.26 / 666.77 ±8.87 / 676.47 ms │ 1665.64 / 1864.18 ±176.98 / 2126.92 ms │ 2.80x slower │
│ QQuery 10 │ 91.75 / 93.94 ±1.46 / 95.42 ms │ 177.90 / 192.45 ±18.86 / 229.01 ms │ 2.05x slower │
│ QQuery 11 │ 104.41 / 105.04 ±0.49 / 105.66 ms │ 242.82 / 278.82 ±29.36 / 306.87 ms │ 2.65x slower │
│ QQuery 12 │ 337.11 / 340.91 ±3.46 / 345.75 ms │ 697.05 / 924.72 ±220.10 / 1342.84 ms │ 2.71x slower │
│ QQuery 13 │ 472.74 / 479.62 ±8.17 / 494.22 ms │ 1286.54 / 1417.16 ±102.23 / 1570.07 ms │ 2.95x slower │
│ QQuery 14 │ 344.43 / 350.91 ±6.80 / 360.88 ms │ 715.25 / 880.78 ±141.58 / 1122.31 ms │ 2.51x slower │
│ QQuery 15 │ 355.41 / 363.29 ±4.45 / 367.13 ms │ 1061.81 / 1371.20 ±241.02 / 1778.85 ms │ 3.77x slower │
│ QQuery 16 │ 717.94 / 729.94 ±17.11 / 763.51 ms │ 1996.04 / 2302.60 ±321.05 / 2917.75 ms │ 3.15x slower │
│ QQuery 17 │ 709.33 / 714.13 ±4.23 / 721.37 ms │ 1972.18 / 2229.91 ±299.90 / 2764.33 ms │ 3.12x slower │
│ QQuery 18 │ 1412.58 / 1472.64 ±45.95 / 1518.78 ms │ 3133.06 / 3506.88 ±329.02 / 4062.15 ms │ 2.38x slower │
│ QQuery 19 │ 34.89 / 36.77 ±2.09 / 40.78 ms │ 71.45 / 88.26 ±25.82 / 139.29 ms │ 2.40x slower │
│ QQuery 20 │ 722.39 / 740.77 ±19.30 / 766.32 ms │ 1162.89 / 1206.97 ±31.98 / 1257.12 ms │ 1.63x slower │
│ QQuery 21 │ 759.47 / 765.21 ±3.39 / 770.09 ms │ 1088.09 / 1441.66 ±295.13 / 1850.59 ms │ 1.88x slower │
│ QQuery 22 │ 1132.10 / 1137.03 ±4.48 / 1144.72 ms │ 2886.92 / 3074.45 ±169.17 / 3354.22 ms │ 2.70x slower │
│ QQuery 23 │ 3084.03 / 3103.80 ±16.77 / 3133.09 ms │ 3765.83 / 4251.42 ±393.95 / 4786.21 ms │ 1.37x slower │
│ QQuery 24 │ 101.93 / 104.31 ±2.09 / 108.16 ms │ 112.18 / 128.36 ±14.56 / 151.20 ms │ 1.23x slower │
│ QQuery 25 │ 139.10 / 141.17 ±1.84 / 144.07 ms │ 241.99 / 309.10 ±48.20 / 391.07 ms │ 2.19x slower │
│ QQuery 26 │ 101.06 / 102.93 ±1.30 / 104.77 ms │ 104.72 / 136.59 ±34.20 / 194.85 ms │ 1.33x slower │
│ QQuery 27 │ 850.09 / 855.95 ±5.14 / 862.76 ms │ 1216.70 / 1474.15 ±224.64 / 1790.89 ms │ 1.72x slower │
│ QQuery 28 │ 3253.19 / 3272.96 ±11.44 / 3287.93 ms │ 7096.07 / 11997.71 ±3397.21 / 17132.01 ms │ 3.67x slower │
│ QQuery 29 │ 50.58 / 55.74 ±4.44 / 62.88 ms │ 101.19 / 134.11 ±21.24 / 162.46 ms │ 2.41x slower │
│ QQuery 30 │ 356.54 / 365.17 ±5.01 / 370.99 ms │ 761.12 / 832.11 ±121.84 / 1074.84 ms │ 2.28x slower │
│ QQuery 31 │ 362.29 / 387.08 ±17.15 / 415.66 ms │ 666.55 / 785.27 ±78.86 / 887.89 ms │ 2.03x slower │
│ QQuery 32 │ 1215.22 / 1273.88 ±37.66 / 1333.16 ms │ 3723.45 / 4906.59 ±761.63 / 5876.98 ms │ 3.85x slower │
│ QQuery 33 │ 1491.06 / 1527.63 ±31.62 / 1569.40 ms │ 2977.09 / 3829.76 ±463.04 / 4369.49 ms │ 2.51x slower │
│ QQuery 34 │ 1470.94 / 1490.39 ±22.68 / 1528.36 ms │ 3576.91 / 4533.04 ±633.94 / 5373.09 ms │ 3.04x slower │
│ QQuery 35 │ 388.84 / 397.21 ±6.42 / 404.87 ms │ 945.36 / 1118.99 ±198.54 / 1488.54 ms │ 2.82x slower │
│ QQuery 36 │ 110.13 / 120.56 ±5.64 / 127.06 ms │ 103.37 / 122.46 ±15.56 / 145.12 ms │ no change │
│ QQuery 37 │ 46.75 / 49.52 ±3.13 / 55.46 ms │ 49.56 / 50.55 ±0.85 / 52.02 ms │ no change │
│ QQuery 38 │ 72.94 / 75.99 ±1.93 / 78.58 ms │ 71.64 / 73.47 ±1.39 / 75.48 ms │ no change │
│ QQuery 39 │ 206.64 / 214.05 ±6.94 / 226.54 ms │ 174.14 / 219.41 ±33.20 / 248.75 ms │ no change │
│ QQuery 40 │ 24.99 / 26.01 ±0.65 / 26.71 ms │ 22.44 / 23.30 ±1.06 / 24.98 ms │ +1.12x faster │
│ QQuery 41 │ 19.60 / 20.52 ±0.62 / 21.36 ms │ 19.51 / 21.01 ±1.02 / 22.02 ms │ no change │
│ QQuery 42 │ 19.37 / 19.74 ±0.25 / 20.13 ms │ 18.25 / 18.74 ±0.45 / 19.36 ms │ +1.05x faster │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22798.95ms │
│ Total Time (morsel-scheduler) │ 59053.42ms │
│ Average Time (HEAD) │ 530.21ms │
│ Average Time (morsel-scheduler) │ 1373.34ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 34 │
│ Queries with No Change │ 6 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 115.1s |
| Peak memory | 37.1 GiB |
| Avg memory | 27.2 GiB |
| CPU user | 1072.0s |
| CPU sys | 98.9s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 296.8s |
| Peak memory | 39.8 GiB |
| Avg memory | 30.5 GiB |
| CPU user | 988.3s |
| CPU sys | 112.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4273773731
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4273723891)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 6.53 / 7.02 ±0.80 / 8.63 ms │ 6.85 / 7.30 ±0.81 / 8.91 ms │ no change │
│ QQuery 2 │ 145.60 / 147.81 ±2.43 / 151.93 ms │ 147.27 / 148.72 ±1.20 / 150.87 ms │ no change │
│ QQuery 3 │ 113.61 / 114.76 ±1.26 / 117.14 ms │ 114.48 / 115.41 ±0.60 / 116.30 ms │ no change │
│ QQuery 4 │ 1343.72 / 1381.99 ±34.17 / 1435.16 ms │ 1389.93 / 1435.94 ±31.93 / 1469.77 ms │ no change │
│ QQuery 5 │ 174.81 / 176.60 ±1.37 / 178.95 ms │ 173.61 / 177.46 ±2.82 / 180.60 ms │ no change │
│ QQuery 6 │ 844.23 / 874.39 ±21.97 / 911.89 ms │ 871.57 / 892.64 ±16.87 / 907.99 ms │ no change │
│ QQuery 7 │ 345.11 / 349.59 ±4.50 / 356.04 ms │ 345.14 / 349.07 ±2.10 / 351.13 ms │ no change │
│ QQuery 8 │ 117.53 / 118.48 ±0.81 / 119.47 ms │ 115.68 / 117.80 ±1.46 / 119.65 ms │ no change │
│ QQuery 9 │ 100.83 / 104.77 ±3.12 / 107.52 ms │ 101.42 / 106.07 ±2.66 / 108.24 ms │ no change │
│ QQuery 10 │ 105.13 / 108.85 ±2.68 / 113.33 ms │ 105.60 / 107.84 ±2.20 / 111.33 ms │ no change │
│ QQuery 11 │ 969.64 / 989.08 ±11.78 / 1003.05 ms │ 934.56 / 959.45 ±14.77 / 976.56 ms │ no change │
│ QQuery 12 │ 48.31 / 48.89 ±0.55 / 49.65 ms │ 49.06 / 49.89 ±0.66 / 51.07 ms │ no change │
│ QQuery 13 │ 405.04 / 410.58 ±3.85 / 415.70 ms │ 399.68 / 406.70 ±5.92 / 413.96 ms │ no change │
│ QQuery 14 │ 1006.05 / 1018.75 ±9.25 / 1033.10 ms │ 1003.65 / 1019.53 ±9.61 / 1033.22 ms │ no change │
│ QQuery 15 │ 16.77 / 17.14 ±0.30 / 17.66 ms │ 14.83 / 16.01 ±0.68 / 16.82 ms │ +1.07x faster │
│ QQuery 16 │ 7.51 / 8.01 ±0.61 / 9.18 ms │ 6.96 / 7.95 ±0.95 / 9.39 ms │ no change │
│ QQuery 17 │ 229.92 / 233.99 ±3.88 / 241.08 ms │ 227.17 / 234.18 ±5.33 / 241.54 ms │ no change │
│ QQuery 18 │ 125.73 / 128.14 ±2.94 / 133.63 ms │ 128.42 / 131.86 ±1.92 / 134.38 ms │ no change │
│ QQuery 19 │ 154.90 / 158.26 ±3.58 / 165.19 ms │ 157.03 / 160.68 ±4.38 / 167.81 ms │ no change │
│ QQuery 20 │ 13.46 / 14.19 ±0.55 / 15.09 ms │ 14.07 / 14.92 ±0.52 / 15.69 ms │ 1.05x slower │
│ QQuery 21 │ 19.75 / 20.82 ±0.59 / 21.51 ms │ 19.15 / 19.68 ±0.47 / 20.34 ms │ +1.06x faster │
│ QQuery 22 │ 484.19 / 489.82 ±4.29 / 495.19 ms │ 476.66 / 491.59 ±12.42 / 511.43 ms │ no change │
│ QQuery 23 │ 918.74 / 934.44 ±18.24 / 966.95 ms │ 898.95 / 915.23 ±11.12 / 933.88 ms │ no change │
│ QQuery 24 │ 383.07 / 388.20 ±4.32 / 394.59 ms │ 385.00 / 395.48 ±7.07 / 405.20 ms │ no change │
│ QQuery 25 │ 343.09 / 351.16 ±6.49 / 361.23 ms │ 342.55 / 348.67 ±5.24 / 357.38 ms │ no change │
│ QQuery 26 │ 83.06 / 85.16 ±1.75 / 87.34 ms │ 81.04 / 82.30 ±1.10 / 84.29 ms │ no change │
│ QQuery 27 │ 7.35 / 8.24 ±0.48 / 8.67 ms │ 7.08 / 7.82 ±0.70 / 8.75 ms │ +1.05x faster │
│ QQuery 28 │ 154.20 / 155.82 ±1.28 / 157.73 ms │ 150.64 / 153.55 ±3.13 / 158.67 ms │ no change │
│ QQuery 29 │ 291.46 / 296.85 ±3.57 / 301.87 ms │ 283.84 / 291.01 ±5.10 / 297.86 ms │ no change │
│ QQuery 30 │ 44.15 / 46.45 ±1.54 / 48.76 ms │ 44.92 / 46.91 ±1.24 / 48.46 ms │ no change │
│ QQuery 31 │ 171.39 / 176.16 ±3.08 / 180.64 ms │ 171.20 / 175.16 ±2.29 / 177.75 ms │ no change │
│ QQuery 32 │ 57.87 / 69.04 ±16.74 / 102.21 ms │ 57.14 / 57.74 ±0.44 / 58.41 ms │ +1.20x faster │
│ QQuery 33 │ 145.30 / 147.76 ±2.00 / 151.23 ms │ 141.45 / 143.94 ±1.70 / 145.96 ms │ no change │
│ QQuery 34 │ 7.51 / 7.81 ±0.38 / 8.56 ms │ 6.93 / 7.22 ±0.21 / 7.52 ms │ +1.08x faster │
│ QQuery 35 │ 112.04 / 112.70 ±0.45 / 113.42 ms │ 107.82 / 109.53 ±1.55 / 112.20 ms │ no change │
│ QQuery 36 │ 7.17 / 7.57 ±0.42 / 8.34 ms │ 6.58 / 6.80 ±0.19 / 7.13 ms │ +1.11x faster │
│ QQuery 37 │ 8.91 / 9.58 ±0.41 / 10.20 ms │ 8.62 / 9.04 ±0.51 / 9.83 ms │ +1.06x faster │
│ QQuery 38 │ 83.89 / 87.94 ±4.12 / 95.47 ms │ 85.83 / 92.93 ±6.13 / 103.22 ms │ 1.06x slower │
│ QQuery 39 │ 127.75 / 133.90 ±5.39 / 141.41 ms │ 125.07 / 131.91 ±7.81 / 147.17 ms │ no change │
│ QQuery 40 │ 107.19 / 114.47 ±8.20 / 130.10 ms │ 111.71 / 118.68 ±9.08 / 136.28 ms │ no change │
│ QQuery 41 │ 14.12 / 14.84 ±0.58 / 15.53 ms │ 14.56 / 15.15 ±0.50 / 15.89 ms │ no change │
│ QQuery 42 │ 109.55 / 112.35 ±1.55 / 113.96 ms │ 109.18 / 111.76 ±1.64 / 113.40 ms │ no change │
│ QQuery 43 │ 6.03 / 6.17 ±0.19 / 6.54 ms │ 5.90 / 6.11 ±0.22 / 6.55 ms │ no change │
│ QQuery 44 │ 11.61 / 12.20 ±0.35 / 12.64 ms │ 11.71 / 11.82 ±0.21 / 12.25 ms │ no change │
│ QQuery 45 │ 52.13 / 52.82 ±0.46 / 53.35 ms │ 51.52 / 51.78 ±0.21 / 52.01 ms │ no change │
│ QQuery 46 │ 8.36 / 8.69 ±0.48 / 9.63 ms │ 8.33 / 8.53 ±0.17 / 8.80 ms │ no change │
│ QQuery 47 │ 755.67 / 782.46 ±17.05 / 807.39 ms │ 711.20 / 747.42 ±20.01 / 767.41 ms │ no change │
│ QQuery 48 │ 288.58 / 304.85 ±10.72 / 318.30 ms │ 294.12 / 298.40 ±2.33 / 300.81 ms │ no change │
│ QQuery 49 │ 257.27 / 261.85 ±3.45 / 265.91 ms │ 254.77 / 257.82 ±1.94 / 259.94 ms │ no change │
│ QQuery 50 │ 226.26 / 234.95 ±8.22 / 250.02 ms │ 226.81 / 232.71 ±3.54 / 237.13 ms │ no change │
│ QQuery 51 │ 185.13 / 188.03 ±2.72 / 191.67 ms │ 182.12 / 184.58 ±1.60 / 186.38 ms │ no change │
│ QQuery 52 │ 109.21 / 110.87 ±1.39 / 113.19 ms │ 110.88 / 112.99 ±2.37 / 117.56 ms │ no change │
│ QQuery 53 │ 102.55 / 104.79 ±2.32 / 109.26 ms │ 103.84 / 105.19 ±0.97 / 106.36 ms │ no change │
│ QQuery 54 │ 147.13 / 151.30 ±3.54 / 155.93 ms │ 146.39 / 147.72 ±1.56 / 150.72 ms │ no change │
│ QQuery 55 │ 107.79 / 108.83 ±1.04 / 110.52 ms │ 109.57 / 111.98 ±1.69 / 114.68 ms │ no change │
│ QQuery 56 │ 141.11 / 144.85 ±3.45 / 151.10 ms │ 144.81 / 146.56 ±1.03 / 147.78 ms │ no change │
│ QQuery 57 │ 174.25 / 178.62 ±4.41 / 186.24 ms │ 181.05 / 184.45 ±1.96 / 187.08 ms │ no change │
│ QQuery 58 │ 287.49 / 302.63 ±13.87 / 326.02 ms │ 286.28 / 302.47 ±9.48 / 312.08 ms │ no change │
│ QQuery 59 │ 197.34 / 201.85 ±3.31 / 207.04 ms │ 199.57 / 203.67 ±3.30 / 208.55 ms │ no change │
│ QQuery 60 │ 147.26 / 149.51 ±1.18 / 150.45 ms │ 147.74 / 149.80 ±1.80 / 152.47 ms │ no change │
│ QQuery 61 │ 13.51 / 13.73 ±0.15 / 13.94 ms │ 13.77 / 14.07 ±0.23 / 14.45 ms │ no change │
│ QQuery 62 │ 942.16 / 968.20 ±21.52 / 1003.33 ms │ 906.38 / 948.17 ±32.87 / 988.73 ms │ no change │
│ QQuery 63 │ 104.28 / 108.14 ±2.66 / 111.91 ms │ 105.20 / 107.29 ±1.41 / 109.43 ms │ no change │
│ QQuery 64 │ 681.19 / 699.27 ±11.06 / 713.20 ms │ 700.56 / 709.53 ±10.02 / 726.21 ms │ no change │
│ QQuery 65 │ 255.79 / 264.90 ±8.95 / 276.39 ms │ 259.52 / 265.85 ±4.39 / 271.00 ms │ no change │
│ QQuery 66 │ 252.78 / 261.35 ±8.58 / 274.42 ms │ 250.40 / 257.96 ±8.91 / 275.07 ms │ no change │
│ QQuery 67 │ 314.26 / 330.14 ±13.26 / 348.19 ms │ 313.39 / 326.12 ±10.26 / 339.44 ms │ no change │
│ QQuery 68 │ 9.13 / 10.29 ±1.23 / 12.29 ms │ 9.22 / 11.26 ±1.70 / 13.36 ms │ 1.09x slower │
│ QQuery 69 │ 102.08 / 104.09 ±1.80 / 106.53 ms │ 103.06 / 104.37 ±0.76 / 105.42 ms │ no change │
│ QQuery 70 │ 328.19 / 348.55 ±12.90 / 366.48 ms │ 343.52 / 356.32 ±13.27 / 380.19 ms │ no change │
│ QQuery 71 │ 136.27 / 140.32 ±2.59 / 144.18 ms │ 136.46 / 139.08 ±1.48 / 141.04 ms │ no change │
│ QQuery 72 │ 618.97 / 632.41 ±8.57 / 644.22 ms │ 634.79 / 640.93 ±6.54 / 649.18 ms │ no change │
│ QQuery 73 │ 6.74 / 8.14 ±1.17 / 9.94 ms │ 7.11 / 7.70 ±0.44 / 8.29 ms │ +1.06x faster │
│ QQuery 74 │ 581.95 / 611.53 ±24.14 / 632.53 ms │ 594.53 / 621.63 ±19.59 / 649.78 ms │ no change │
│ QQuery 75 │ 274.74 / 282.12 ±5.32 / 288.33 ms │ 279.85 / 283.39 ±1.79 / 284.80 ms │ no change │
│ QQuery 76 │ 132.42 / 134.63 ±1.74 / 137.25 ms │ 133.34 / 135.46 ±2.29 / 139.68 ms │ no change │
│ QQuery 77 │ 188.28 / 190.72 ±2.89 / 196.31 ms │ 188.20 / 192.39 ±2.59 / 195.62 ms │ no change │
│ QQuery 78 │ 341.71 / 352.42 ±10.57 / 367.09 ms │ 339.26 / 351.54 ±8.08 / 360.59 ms │ no change │
│ QQuery 79 │ 229.89 / 240.80 ±11.38 / 261.70 ms │ 229.26 / 240.90 ±10.95 / 259.24 ms │ no change │
│ QQuery 80 │ 327.29 / 329.79 ±2.47 / 334.08 ms │ 324.56 / 328.20 ±2.68 / 332.85 ms │ no change │
│ QQuery 81 │ 27.36 / 28.22 ±1.12 / 30.41 ms │ 26.21 / 27.64 ±0.88 / 28.71 ms │ no change │
│ QQuery 82 │ 204.29 / 207.25 ±2.40 / 210.22 ms │ 199.53 / 204.34 ±4.78 / 211.26 ms │ no change │
│ QQuery 83 │ 39.27 / 40.60 ±0.80 / 41.49 ms │ 38.81 / 39.48 ±0.51 / 40.35 ms │ no change │
│ QQuery 84 │ 49.06 / 49.75 ±0.69 / 51.05 ms │ 49.01 / 50.58 ±1.18 / 52.54 ms │ no change │
│ QQuery 85 │ 148.94 / 150.36 ±1.18 / 152.11 ms │ 147.79 / 150.05 ±2.55 / 154.85 ms │ no change │
│ QQuery 86 │ 40.19 / 41.71 ±1.06 / 43.08 ms │ 38.08 / 41.27 ±1.80 / 43.16 ms │ no change │
│ QQuery 87 │ 84.52 / 89.52 ±3.28 / 94.18 ms │ 86.06 / 92.20 ±6.76 / 105.06 ms │ no change │
│ QQuery 88 │ 100.51 / 101.86 ±1.16 / 103.41 ms │ 100.22 / 106.29 ±8.56 / 123.15 ms │ no change │
│ QQuery 89 │ 121.21 / 123.81 ±1.53 / 125.41 ms │ 121.65 / 122.69 ±0.83 / 124.15 ms │ no change │
│ QQuery 90 │ 24.70 / 25.75 ±0.93 / 27.22 ms │ 25.31 / 26.00 ±0.41 / 26.54 ms │ no change │
│ QQuery 91 │ 63.69 / 65.88 ±1.33 / 67.15 ms │ 62.22 / 63.88 ±1.81 / 67.33 ms │ no change │
│ QQuery 92 │ 58.48 / 59.07 ±0.59 / 60.20 ms │ 57.71 / 58.99 ±1.09 / 60.99 ms │ no change │
│ QQuery 93 │ 185.88 / 189.03 ±2.08 / 192.21 ms │ 185.87 / 191.18 ±4.84 / 199.50 ms │ no change │
│ QQuery 94 │ 61.50 / 62.95 ±0.94 / 64.27 ms │ 61.20 / 63.34 ±2.17 / 67.36 ms │ no change │
│ QQuery 95 │ 132.46 / 133.53 ±0.82 / 134.64 ms │ 129.28 / 132.55 ±1.85 / 134.81 ms │ no change │
│ QQuery 96 │ 71.08 / 74.77 ±2.94 / 79.31 ms │ 71.32 / 75.08 ±1.90 / 76.50 ms │ no change │
│ QQuery 97 │ 133.32 / 135.31 ±1.37 / 137.19 ms │ 128.78 / 133.35 ±4.69 / 139.70 ms │ no change │
│ QQuery 98 │ 164.54 / 166.24 ±1.63 / 169.03 ms │ 155.41 / 160.37 ±4.10 / 165.75 ms │ no change │
│ QQuery 99 │ 10795.17 / 10868.49 ±43.81 / 10925.20 ms │ 10833.95 / 10901.79 ±55.45 / 10987.63 ms │ no change │
└───────────┴──────────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 32133.28ms │
│ Total Time (morsel-scheduler) │ 32138.78ms │
│ Average Time (HEAD) │ 324.58ms │
│ Average Time (morsel-scheduler) │ 324.63ms │
│ Queries Faster │ 8 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 88 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 161.1s |
| Peak memory | 5.7 GiB |
| Avg memory | 4.7 GiB |
| CPU user | 265.7s |
| CPU sys | 18.3s |
| Peak spill | 0 B |
**tpcds — branch**
| Metric | Value |
|--------|-------|
| Wall time | 161.0s |
| Peak memory | 5.4 GiB |
| Avg memory | 4.4 GiB |
| CPU user | 265.7s |
| CPU sys | 18.4s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275402429
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275376685)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and morsel-scheduler
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ morsel-scheduler ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.20 / 4.51 ±6.46 / 17.43 ms │ 1.23 / 4.52 ±6.42 / 17.36 ms │ no change │
│ QQuery 1 │ 14.25 / 14.75 ±0.26 / 14.95 ms │ 21.86 / 24.92 ±2.50 / 28.49 ms │ 1.69x slower │
│ QQuery 2 │ 44.10 / 44.47 ±0.34 / 45.01 ms │ 67.53 / 83.69 ±11.33 / 101.71 ms │ 1.88x slower │
│ QQuery 3 │ 42.88 / 45.04 ±1.46 / 47.07 ms │ 66.18 / 83.43 ±14.73 / 104.59 ms │ 1.85x slower │
│ QQuery 4 │ 288.49 / 294.00 ±3.46 / 297.29 ms │ 758.01 / 983.26 ±148.80 / 1164.79 ms │ 3.34x slower │
│ QQuery 5 │ 335.92 / 339.01 ±3.57 / 345.77 ms │ 632.05 / 697.58 ±68.86 / 788.88 ms │ 2.06x slower │
│ QQuery 6 │ 6.12 / 8.24 ±2.43 / 12.87 ms │ 3.77 / 6.21 ±1.86 / 9.42 ms │ +1.33x faster │
│ QQuery 7 │ 16.94 / 17.50 ±0.41 / 18.02 ms │ 21.63 / 24.63 ±2.26 / 27.33 ms │ 1.41x slower │
│ QQuery 8 │ 409.88 / 417.85 ±4.16 / 421.53 ms │ 990.50 / 1246.22 ±194.50 / 1559.15 ms │ 2.98x slower │
│ QQuery 9 │ 650.77 / 658.79 ±4.98 / 664.88 ms │ 1406.59 / 1662.93 ±270.35 / 2158.80 ms │ 2.52x slower │
│ QQuery 10 │ 93.69 / 94.54 ±0.71 / 95.84 ms │ 152.89 / 191.64 ±38.57 / 255.61 ms │ 2.03x slower │
│ QQuery 11 │ 104.90 / 105.70 ±0.66 / 106.60 ms │ 211.09 / 227.47 ±13.54 / 250.07 ms │ 2.15x slower │
│ QQuery 12 │ 337.37 / 341.73 ±4.56 / 350.40 ms │ 628.41 / 793.91 ±85.92 / 866.51 ms │ 2.32x slower │
│ QQuery 13 │ 454.66 / 474.46 ±13.12 / 491.34 ms │ 1172.12 / 1407.56 ±214.57 / 1809.47 ms │ 2.97x slower │
│ QQuery 14 │ 344.36 / 347.98 ±2.65 / 352.26 ms │ 827.14 / 883.03 ±67.02 / 1012.24 ms │ 2.54x slower │
│ QQuery 15 │ 351.01 / 362.60 ±14.16 / 386.47 ms │ 888.21 / 1115.48 ±153.65 / 1352.28 ms │ 3.08x slower │
│ QQuery 16 │ 716.99 / 748.63 ±17.44 / 767.19 ms │ 2111.22 / 2348.47 ±232.56 / 2723.32 ms │ 3.14x slower │
│ QQuery 17 │ 706.87 / 746.26 ±31.15 / 779.33 ms │ 1885.73 / 2254.92 ±329.53 / 2715.48 ms │ 3.02x slower │
│ QQuery 18 │ 1446.98 / 1477.27 ±25.68 / 1511.35 ms │ 4165.07 / 4705.34 ±430.23 / 5442.32 ms │ 3.19x slower │
│ QQuery 19 │ 36.43 / 38.29 ±3.12 / 44.51 ms │ 61.99 / 72.39 ±8.80 / 84.69 ms │ 1.89x slower │
│ QQuery 20 │ 721.07 / 739.62 ±19.72 / 767.11 ms │ 919.16 / 1084.93 ±130.66 / 1297.48 ms │ 1.47x slower │
│ QQuery 21 │ 759.96 / 768.12 ±5.68 / 774.93 ms │ 980.85 / 1173.17 ±124.56 / 1316.24 ms │ 1.53x slower │
│ QQuery 22 │ 1129.80 / 1136.82 ±6.57 / 1146.65 ms │ 2016.11 / 2288.27 ±207.49 / 2532.53 ms │ 2.01x slower │
│ QQuery 23 │ 3077.30 / 3093.51 ±9.55 / 3106.66 ms │ 3586.78 / 4137.25 ±430.69 / 4662.61 ms │ 1.34x slower │
│ QQuery 24 │ 103.01 / 106.75 ±3.60 / 113.27 ms │ 113.41 / 131.67 ±17.65 / 164.94 ms │ 1.23x slower │
│ QQuery 25 │ 137.55 / 140.05 ±1.87 / 142.82 ms │ 242.10 / 321.54 ±49.41 / 364.97 ms │ 2.30x slower │
│ QQuery 26 │ 100.34 / 103.29 ±1.83 / 105.70 ms │ 106.57 / 134.42 ±24.41 / 177.61 ms │ 1.30x slower │
│ QQuery 27 │ 850.88 / 855.49 ±5.32 / 865.36 ms │ 1207.17 / 1507.63 ±202.83 / 1844.03 ms │ 1.76x slower │
│ QQuery 28 │ 3250.62 / 3285.19 ±18.22 / 3299.88 ms │ 7680.28 / 9528.29 ±1643.27 / 12533.14 ms │ 2.90x slower │
│ QQuery 29 │ 50.47 / 54.16 ±4.95 / 63.59 ms │ 73.23 / 109.52 ±26.59 / 152.32 ms │ 2.02x slower │
│ QQuery 30 │ 358.12 / 361.43 ±2.93 / 365.66 ms │ 632.23 / 733.29 ±67.28 / 824.84 ms │ 2.03x slower │
│ QQuery 31 │ 353.82 / 369.88 ±12.20 / 387.24 ms │ 629.40 / 887.15 ±221.24 / 1203.31 ms │ 2.40x slower │
│ QQuery 32 │ 1050.13 / 1064.27 ±24.42 / 1112.97 ms │ 3701.94 / 4394.44 ±709.72 / 5674.05 ms │ 4.13x slower │
│ QQuery 33 │ 1457.52 / 1465.72 ±7.59 / 1479.35 ms │ 3004.17 / 3728.44 ±626.16 / 4855.86 ms │ 2.54x slower │
│ QQuery 34 │ 1446.08 / 1464.53 ±22.30 / 1505.00 ms │ 3143.30 / 3777.05 ±542.43 / 4760.94 ms │ 2.58x slower │
│ QQuery 35 │ 387.56 / 394.21 ±5.93 / 401.71 ms │ 762.88 / 1010.67 ±243.31 / 1411.22 ms │ 2.56x slower │
│ QQuery 36 │ 119.14 / 122.01 ±1.70 / 124.23 ms │ 125.25 / 131.11 ±4.80 / 135.74 ms │ 1.07x slower │
│ QQuery 37 │ 50.87 / 52.04 ±1.04 / 53.83 ms │ 47.91 / 49.12 ±1.65 / 52.38 ms │ +1.06x faster │
│ QQuery 38 │ 77.22 / 78.56 ±1.50 / 81.10 ms │ 76.09 / 80.78 ±6.40 / 92.74 ms │ no change │
│ QQuery 39 │ 204.60 / 216.35 ±7.23 / 226.50 ms │ 189.97 / 233.91 ±27.04 / 265.80 ms │ 1.08x slower │
│ QQuery 40 │ 24.58 / 25.36 ±0.69 / 26.40 ms │ 20.58 / 26.84 ±5.77 / 37.11 ms │ 1.06x slower │
│ QQuery 41 │ 19.79 / 21.33 ±1.15 / 23.14 ms │ 17.40 / 19.68 ±1.92 / 22.56 ms │ +1.08x faster │
│ QQuery 42 │ 20.04 / 20.38 ±0.24 / 20.66 ms │ 18.56 / 19.41 ±0.70 / 20.55 ms │ no change │
└───────────┴───────────────────────────────────────┴──────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 22520.66ms │
│ Total Time (morsel-scheduler) │ 54326.23ms │
│ Average Time (HEAD) │ 523.74ms │
│ Average Time (morsel-scheduler) │ 1263.40ms │
│ Queries Faster │ 3 │
│ Queries Slower │ 37 │
│ Queries with No Change │ 3 │
│ Queries with Failure │ 0 │
└─────────────────────────────────┴────────────┘
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned — base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 113.8s |
| Peak memory | 41.9 GiB |
| Avg memory | 30.2 GiB |
| CPU user | 1073.0s |
| CPU sys | 86.1s |
| Peak spill | 0 B |
**clickbench_partitioned — branch**
| Metric | Value |
|--------|-------|
| Wall time | 273.3s |
| Peak memory | 41.3 GiB |
| Avg memory | 33.9 GiB |
| CPU user | 1011.6s |
| CPU sys | 112.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275682061
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275674921)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275674921-1514-rwpkz 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(morsel-scheduler): thread-per-core morsel-driven scheduler prototype [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #21718:
URL: https://github.com/apache/datafusion/pull/21718#issuecomment-4275681058
🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/21718#issuecomment-4275639877)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4275639877-1512-4xhqm 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing morsel-scheduler (1fd587df8d347995910cfbeb359b84dbcad1d6c9) to 7731130 (merge-base) [diff](https://github.com/apache/datafusion/compare/773113086191b87b5e21b2c0fa64af91c2a19298..1fd587df8d347995910cfbeb359b84dbcad1d6c9) using: tpcds
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org