You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Dandandan (via GitHub)" <gi...@apache.org> on 2026/08/30 09:46:19 UTC
[PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Dandandan opened a new pull request, #24794:
URL: https://github.com/apache/datafusion/pull/24794
## Which issue does this PR close?
- Closes #.
## Rationale for this change
A `LeftSemi` hash join emits nothing while probing. `adjust_indices_by_join_type`
returns empty index arrays for it, and every matched build row comes out at the
end from the visited bitmap, the same way `LeftAnti` and `LeftMark` work.
`HashJoinExec::compute_properties` nevertheless reports
`EmissionType::Incremental` for `LeftSemi`, grouped with `Inner` and
`RightSemi` under the comment "If we only need to generate matched rows from
the probe side". That is not what `LeftSemi` does: its rows come from the build
side. `LeftAnti` and `LeftMark` are correctly classified `Both`.
So the join is blocking, and a `LIMIT` above it buys nothing. On TPC-DS SF1:
```sql
select ws_order_number from web_sales ws
where exists (select 1 from web_returns wr
where wr.wr_order_number = ws.ws_order_number)
limit 10
```
| | |
|---|---|
| plans as `LeftSemi`, with `LIMIT 10` | 9.2 ms |
| the same query with no `LIMIT` at all | 8.6 ms |
| mirrored so it plans as `RightSemi`, with `LIMIT 10` | 2.8 ms |
## What changes are included in this PR?
A semi join only needs to know whether a build row has matched yet, and the
visited bitmap already carries that. This emits a build row when its bit flips
from unset to set, which produces the same rows in the same order but while
probing rather than after it. The final stage then has nothing left to do.
The comment at that site already described the fix: "When visit the right
batch, we can output the matched left row and don't need to wait the end of
loop".
The query above now takes 2.7 ms, matching the `RightSemi` form, and
`EmissionType::Incremental` becomes true rather than aspirational.
Only the hash join changes. `NestedLoopJoinExec` still emits its `LeftSemi`
rows at the end, so `need_produce_result_in_final` is untouched.
## Are these changes tested?
Yes, by the existing coverage: the full sqllogictest suite (504 files) and the
1117 join unit tests pass. Semi joins are heavily covered by `joins.slt`,
`subquery.slt` and the hash join's own tests, including across partition modes
and batch sizes.
One expectation changed, in `push_down_filter_parquet.slt`. It is an `EXPLAIN
ANALYZE` whose join reports `input_batches=1, input_rows=2` where it used to
report `2` and `4`; `output_rows` is 2 either way. The join no longer holds its
output back, so the consumer finishes and the probe scan stops earlier.
## Are there any user-facing changes?
`LeftSemi` hash joins are no longer blocking, so a `LIMIT` or any other
early-terminating consumer above one can now stop it early. Results are
unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "github-actions[bot] (via GitHub) via github" <gi...@datafusion.apache.org>.
github-actions[bot] closed pull request #24794: perf: Emit LeftSemi hash join rows while probing
URL: https://github.com/apache/datafusion/pull/24794
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5471125244
run benchmark tpch tpcds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5468066020
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/24794?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `95.83333%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 81.52%. Comparing base ([`61bf6b9`](https://app.codecov.io/gh/apache/datafusion/commit/61bf6b96cc07d6a0518014dec7fe86fbef76576e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd3dfdc`](https://app.codecov.io/gh/apache/datafusion/commit/cd3dfdcf871de36b4698de7d74e248e575799f83?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/24794?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...fusion/physical-plan/src/joins/hash\_join/stream.rs](https://app.codecov.io/gh/apache/datafusion/pull/24794?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fhash_join%2Fstream.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9oYXNoX2pvaW4vc3RyZWFtLnJz) | 95.83% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/24794?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #24794 +/- ##
=======================================
Coverage 81.52% 81.52%
=======================================
Files 1123 1123
Lines 405970 406054 +84
Branches 405970 406054 +84
=======================================
+ Hits 330978 331048 +70
- Misses 55627 55641 +14
Partials 19365 19365
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/24794?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5471214645
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/24794#issuecomment-5471125244)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing perf/left-semi-incremental (9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a) to 61bf6b9 (merge-base) [diff](https://github.com/apache/datafusion/compare/61bf6b96cc07d6a0518014dec7fe86fbef76576e..9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a)
<details><summary>Run configuration</summary>
```yaml
run benchmark tpcds
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and perf_left-semi-incremental
--------------------
Benchmark tpcds_sf1.json
--------------------
βββββββββββββ³βββββββββββββ³βββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β perf_left-semi-incremental β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 1 β 5.71 ms β 5.61 ms β no change β
β QQuery 2 β 80.79 ms β 81.04 ms β no change β
β QQuery 3 β 28.58 ms β 29.30 ms β no change β
β QQuery 4 β 478.10 ms β 485.17 ms β no change β
β QQuery 5 β 51.65 ms β 51.40 ms β no change β
β QQuery 6 β 36.26 ms β 35.87 ms β no change β
β QQuery 7 β 74.93 ms β 75.09 ms β no change β
β QQuery 8 β 36.36 ms β 36.06 ms β no change β
β QQuery 9 β 52.91 ms β 51.25 ms β no change β
β QQuery 10 β 63.00 ms β 63.08 ms β no change β
β QQuery 11 β 298.31 ms β 301.56 ms β no change β
β QQuery 12 β 31.13 ms β 28.46 ms β +1.09x faster β
β QQuery 13 β 124.53 ms β 117.36 ms β +1.06x faster β
β QQuery 14 β 433.43 ms β 415.44 ms β no change β
β QQuery 15 β 68.82 ms β 57.84 ms β +1.19x faster β
β QQuery 16 β 7.71 ms β 6.79 ms β +1.14x faster β
β QQuery 17 β 84.30 ms β 79.17 ms β +1.06x faster β
β QQuery 18 β 115.27 ms β 103.81 ms β +1.11x faster β
β QQuery 19 β 43.73 ms β 40.89 ms β +1.07x faster β
β QQuery 20 β 40.52 ms β 35.41 ms β +1.14x faster β
β QQuery 21 β 18.50 ms β 17.38 ms β +1.06x faster β
β QQuery 22 β 72.73 ms β 63.70 ms β +1.14x faster β
β QQuery 23 β 315.99 ms β 309.37 ms β no change β
β QQuery 24 β 195.01 ms β 195.90 ms β no change β
β QQuery 25 β 113.37 ms β 111.75 ms β no change β
β QQuery 26 β 50.00 ms β 52.26 ms β no change β
β QQuery 27 β 6.26 ms β 7.03 ms β 1.12x slower β
β QQuery 28 β 60.50 ms β 63.04 ms β no change β
β QQuery 29 β 97.16 ms β 102.46 ms β 1.05x slower β
β QQuery 30 β 32.41 ms β 35.74 ms β 1.10x slower β
β QQuery 31 β 110.74 ms β 117.15 ms β 1.06x slower β
β QQuery 32 β 20.15 ms β 22.66 ms β 1.12x slower β
β QQuery 33 β 37.69 ms β 40.61 ms β 1.08x slower β
β QQuery 34 β 9.95 ms β 11.02 ms β 1.11x slower β
β QQuery 35 β 73.69 ms β 82.45 ms β 1.12x slower β
β QQuery 36 β 6.03 ms β 6.47 ms β 1.07x slower β
β QQuery 37 β 6.93 ms β 7.66 ms β 1.11x slower β
β QQuery 38 β 61.58 ms β 69.46 ms β 1.13x slower β
β QQuery 39 β 90.24 ms β 103.87 ms β 1.15x slower β
β QQuery 40 β 26.24 ms β 26.43 ms β no change β
β QQuery 41 β 12.11 ms β 12.42 ms β no change β
β QQuery 42 β 24.59 ms β 25.58 ms β no change β
β QQuery 43 β 5.86 ms β 5.88 ms β no change β
β QQuery 44 β 9.65 ms β 10.47 ms β 1.09x slower β
β QQuery 45 β 37.72 ms β 47.53 ms β 1.26x slower β
β QQuery 46 β 12.64 ms β 12.91 ms β no change β
β QQuery 47 β 226.43 ms β 280.53 ms β 1.24x slower β
β QQuery 48 β 95.19 ms β 101.08 ms β 1.06x slower β
β QQuery 49 β 74.41 ms β 75.79 ms β no change β
β QQuery 50 β 59.09 ms β 62.32 ms β 1.05x slower β
β QQuery 51 β 92.17 ms β 90.97 ms β no change β
β QQuery 52 β 24.32 ms β 24.01 ms β no change β
β QQuery 53 β 29.16 ms β 28.93 ms β no change β
β QQuery 54 β 54.43 ms β 54.01 ms β no change β
β QQuery 55 β 23.00 ms β 22.92 ms β no change β
β QQuery 56 β 38.34 ms β 39.44 ms β no change β
β QQuery 57 β 175.75 ms β 176.04 ms β no change β
β QQuery 58 β 110.85 ms β 111.70 ms β no change β
β QQuery 59 β 117.35 ms β 120.66 ms β no change β
β QQuery 60 β 39.48 ms β 40.91 ms β no change β
β QQuery 61 β 12.37 ms β 13.18 ms β 1.06x slower β
β QQuery 62 β 46.03 ms β 46.41 ms β no change β
β QQuery 63 β 29.36 ms β 28.84 ms β no change β
β QQuery 64 β 364.77 ms β 366.42 ms β no change β
β QQuery 65 β 120.58 ms β 124.53 ms β no change β
β QQuery 66 β 80.40 ms β 81.54 ms β no change β
β QQuery 67 β 269.42 ms β 270.79 ms β no change β
β QQuery 68 β 12.57 ms β 11.83 ms β +1.06x faster β
β QQuery 69 β 58.07 ms β 59.29 ms β no change β
β QQuery 70 β 106.19 ms β 104.73 ms β no change β
β QQuery 71 β 35.54 ms β 35.16 ms β no change β
β QQuery 72 β 1860.30 ms β 1639.37 ms β +1.13x faster β
β QQuery 73 β 9.93 ms β 9.55 ms β no change β
β QQuery 74 β 169.66 ms β 167.85 ms β no change β
β QQuery 75 β 145.45 ms β 155.29 ms β 1.07x slower β
β QQuery 76 β 34.60 ms β 35.54 ms β no change β
β QQuery 77 β 61.02 ms β 60.96 ms β no change β
β QQuery 78 β 212.49 ms β 215.92 ms β no change β
β QQuery 79 β 66.15 ms β 66.10 ms β no change β
β QQuery 80 β 98.21 ms β 97.56 ms β no change β
β QQuery 81 β 25.54 ms β 26.23 ms β no change β
β QQuery 82 β 16.25 ms β 16.26 ms β no change β
β QQuery 83 β 33.47 ms β 33.53 ms β no change β
β QQuery 84 β 28.95 ms β 28.96 ms β no change β
β QQuery 85 β 106.62 ms β 102.62 ms β no change β
β QQuery 86 β 27.43 ms β 25.14 ms β +1.09x faster β
β QQuery 87 β 62.86 ms β 62.57 ms β no change β
β QQuery 88 β 64.09 ms β 63.75 ms β no change β
β QQuery 89 β 35.46 ms β 35.04 ms β no change β
β QQuery 90 β 18.18 ms β 16.88 ms β +1.08x faster β
β QQuery 91 β 45.46 ms β 44.85 ms β no change β
β QQuery 92 β 28.69 ms β 29.46 ms β no change β
β QQuery 93 β 49.76 ms β 48.99 ms β no change β
β QQuery 94 β 37.72 ms β 37.66 ms β no change β
β QQuery 95 β 80.46 ms β 78.31 ms β no change β
β QQuery 96 β 24.53 ms β 23.84 ms β no change β
β QQuery 97 β 52.64 ms β 52.25 ms β no change β
β QQuery 98 β 42.37 ms β 42.68 ms β no change β
β QQuery 99 β 69.57 ms β 69.14 ms β no change β
βββββββββββββ΄βββββββββββββ΄βββββββββββββββββββββββββββββ΄ββββββββββββββββ
βββββββββββββββββββββββββββββββββββββββββββββ³ββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 9466.88ms β
β Total Time (perf_left-semi-incremental) β 9320.11ms β
β Average Time (HEAD) β 95.63ms β
β Average Time (perf_left-semi-incremental) β 94.14ms β
β Queries Faster β 14 β
β Queries Slower β 19 β
β Queries with No Change β 66 β
β Queries with Failure β 0 β
βββββββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββ
Distribution per query (min / mean Β±stddev / max):
Comparing HEAD and perf_left-semi-incremental
--------------------
Benchmark tpcds_sf1.json
--------------------
βββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β perf_left-semi-incremental β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 1 β 5.71 / 6.25 Β±0.90 / 8.05 ms β 5.61 / 6.16 Β±0.93 / 8.01 ms β no change β
β QQuery 2 β 80.79 / 81.04 Β±0.21 / 81.41 ms β 81.04 / 81.56 Β±0.39 / 82.21 ms β no change β
β QQuery 3 β 28.58 / 29.08 Β±0.31 / 29.48 ms β 29.30 / 29.50 Β±0.12 / 29.68 ms β no change β
β QQuery 4 β 478.10 / 481.66 Β±2.80 / 485.23 ms β 485.17 / 491.16 Β±5.87 / 500.54 ms β no change β
β QQuery 5 β 51.65 / 53.88 Β±3.54 / 60.92 ms β 51.40 / 52.08 Β±0.55 / 52.71 ms β no change β
β QQuery 6 β 36.26 / 36.60 Β±0.42 / 37.30 ms β 35.87 / 36.28 Β±0.42 / 37.02 ms β no change β
β QQuery 7 β 74.93 / 76.35 Β±0.87 / 77.16 ms β 75.09 / 75.47 Β±0.32 / 75.98 ms β no change β
β QQuery 8 β 36.36 / 36.69 Β±0.21 / 36.96 ms β 36.06 / 37.71 Β±2.68 / 43.05 ms β no change β
β QQuery 9 β 52.91 / 56.18 Β±2.76 / 61.23 ms β 51.25 / 53.62 Β±1.52 / 55.99 ms β no change β
β QQuery 10 β 63.00 / 63.57 Β±0.35 / 64.00 ms β 63.08 / 63.37 Β±0.24 / 63.78 ms β no change β
β QQuery 11 β 298.31 / 356.47 Β±29.55 / 378.84 ms β 301.56 / 307.18 Β±3.68 / 312.53 ms β +1.16x faster β
β QQuery 12 β 31.13 / 31.99 Β±0.63 / 33.01 ms β 28.46 / 28.70 Β±0.18 / 28.98 ms β +1.11x faster β
β QQuery 13 β 124.53 / 126.34 Β±1.17 / 127.44 ms β 117.36 / 118.71 Β±0.83 / 119.96 ms β +1.06x faster β
β QQuery 14 β 433.43 / 437.47 Β±3.21 / 443.21 ms β 415.44 / 418.66 Β±1.78 / 420.23 ms β no change β
β QQuery 15 β 68.82 / 70.71 Β±2.05 / 74.60 ms β 57.84 / 58.44 Β±0.50 / 59.21 ms β +1.21x faster β
β QQuery 16 β 7.71 / 7.80 Β±0.05 / 7.84 ms β 6.79 / 7.03 Β±0.28 / 7.56 ms β +1.11x faster β
β QQuery 17 β 84.30 / 85.82 Β±1.30 / 88.03 ms β 79.17 / 80.09 Β±1.24 / 82.52 ms β +1.07x faster β
β QQuery 18 β 115.27 / 117.22 Β±1.21 / 118.59 ms β 103.81 / 105.47 Β±1.19 / 106.91 ms β +1.11x faster β
β QQuery 19 β 43.73 / 44.34 Β±0.62 / 45.33 ms β 40.89 / 41.15 Β±0.20 / 41.41 ms β +1.08x faster β
β QQuery 20 β 40.52 / 40.78 Β±0.35 / 41.48 ms β 35.41 / 35.83 Β±0.47 / 36.75 ms β +1.14x faster β
β QQuery 21 β 18.50 / 18.80 Β±0.18 / 19.01 ms β 17.38 / 18.00 Β±0.76 / 19.49 ms β no change β
β QQuery 22 β 72.73 / 73.38 Β±0.51 / 74.08 ms β 63.70 / 64.05 Β±0.30 / 64.51 ms β +1.15x faster β
β QQuery 23 β 315.99 / 332.25 Β±17.59 / 360.78 ms β 309.37 / 312.68 Β±3.88 / 320.25 ms β +1.06x faster β
β QQuery 24 β 195.01 / 201.23 Β±5.80 / 210.68 ms β 195.90 / 198.32 Β±2.87 / 203.50 ms β no change β
β QQuery 25 β 113.37 / 114.98 Β±1.37 / 117.45 ms β 111.75 / 113.31 Β±2.32 / 117.85 ms β no change β
β QQuery 26 β 50.00 / 51.17 Β±0.60 / 51.59 ms β 52.26 / 52.75 Β±0.30 / 53.10 ms β no change β
β QQuery 27 β 6.26 / 6.42 Β±0.27 / 6.97 ms β 7.03 / 7.23 Β±0.19 / 7.49 ms β 1.12x slower β
β QQuery 28 β 60.50 / 60.85 Β±0.25 / 61.18 ms β 63.04 / 64.78 Β±2.89 / 70.55 ms β 1.06x slower β
β QQuery 29 β 97.16 / 99.77 Β±2.54 / 104.06 ms β 102.46 / 104.21 Β±1.64 / 106.87 ms β no change β
β QQuery 30 β 32.41 / 33.32 Β±0.97 / 35.15 ms β 35.74 / 36.05 Β±0.29 / 36.55 ms β 1.08x slower β
β QQuery 31 β 110.74 / 111.67 Β±0.93 / 113.42 ms β 117.15 / 119.17 Β±1.55 / 121.93 ms β 1.07x slower β
β QQuery 32 β 20.15 / 22.46 Β±3.61 / 29.65 ms β 22.66 / 22.88 Β±0.17 / 23.16 ms β no change β
β QQuery 33 β 37.69 / 38.30 Β±0.34 / 38.68 ms β 40.61 / 40.75 Β±0.16 / 41.03 ms β 1.06x slower β
β QQuery 34 β 9.95 / 10.46 Β±0.48 / 11.23 ms β 11.02 / 11.25 Β±0.20 / 11.58 ms β 1.07x slower β
β QQuery 35 β 73.69 / 74.20 Β±0.46 / 74.96 ms β 82.45 / 84.31 Β±1.41 / 86.70 ms β 1.14x slower β
β QQuery 36 β 6.03 / 6.15 Β±0.15 / 6.43 ms β 6.47 / 6.70 Β±0.20 / 7.00 ms β 1.09x slower β
β QQuery 37 β 6.93 / 7.00 Β±0.05 / 7.07 ms β 7.66 / 7.84 Β±0.09 / 7.89 ms β 1.12x slower β
β QQuery 38 β 61.58 / 62.76 Β±1.28 / 65.15 ms β 69.46 / 69.77 Β±0.36 / 70.48 ms β 1.11x slower β
β QQuery 39 β 90.24 / 92.09 Β±3.12 / 98.33 ms β 103.87 / 105.50 Β±2.05 / 109.50 ms β 1.15x slower β
β QQuery 40 β 26.24 / 26.77 Β±0.47 / 27.43 ms β 26.43 / 27.21 Β±0.45 / 27.69 ms β no change β
β QQuery 41 β 12.11 / 12.34 Β±0.27 / 12.71 ms β 12.42 / 12.54 Β±0.09 / 12.68 ms β no change β
β QQuery 42 β 24.59 / 25.11 Β±0.41 / 25.62 ms β 25.58 / 26.78 Β±1.41 / 29.46 ms β 1.07x slower β
β QQuery 43 β 5.86 / 5.94 Β±0.07 / 6.07 ms β 5.88 / 6.05 Β±0.11 / 6.17 ms β no change β
β QQuery 44 β 9.65 / 9.94 Β±0.32 / 10.50 ms β 10.47 / 10.64 Β±0.10 / 10.74 ms β 1.07x slower β
β QQuery 45 β 37.72 / 42.51 Β±3.74 / 47.48 ms β 47.53 / 48.72 Β±0.75 / 49.75 ms β 1.15x slower β
β QQuery 46 β 12.64 / 13.63 Β±0.83 / 15.03 ms β 12.91 / 13.09 Β±0.10 / 13.19 ms β no change β
β QQuery 47 β 226.43 / 262.26 Β±21.09 / 283.89 ms β 280.53 / 283.85 Β±3.21 / 288.30 ms β 1.08x slower β
β QQuery 48 β 95.19 / 96.34 Β±0.77 / 97.54 ms β 101.08 / 102.01 Β±0.60 / 102.79 ms β 1.06x slower β
β QQuery 49 β 74.41 / 76.82 Β±4.00 / 84.80 ms β 75.79 / 77.90 Β±2.27 / 82.27 ms β no change β
β QQuery 50 β 59.09 / 59.89 Β±0.52 / 60.41 ms β 62.32 / 63.81 Β±2.18 / 68.12 ms β 1.07x slower β
β QQuery 51 β 92.17 / 95.93 Β±3.25 / 101.17 ms β 90.97 / 93.77 Β±2.47 / 98.29 ms β no change β
β QQuery 52 β 24.32 / 24.69 Β±0.42 / 25.46 ms β 24.01 / 26.15 Β±3.04 / 32.18 ms β 1.06x slower β
β QQuery 53 β 29.16 / 29.54 Β±0.20 / 29.69 ms β 28.93 / 29.24 Β±0.24 / 29.61 ms β no change β
β QQuery 54 β 54.43 / 54.69 Β±0.19 / 54.91 ms β 54.01 / 54.35 Β±0.34 / 54.92 ms β no change β
β QQuery 55 β 23.00 / 23.19 Β±0.21 / 23.59 ms β 22.92 / 23.49 Β±0.45 / 24.15 ms β no change β
β QQuery 56 β 38.34 / 39.04 Β±0.59 / 40.09 ms β 39.44 / 40.32 Β±0.56 / 41.16 ms β no change β
β QQuery 57 β 175.75 / 178.62 Β±4.66 / 187.89 ms β 176.04 / 180.47 Β±3.95 / 187.06 ms β no change β
β QQuery 58 β 110.85 / 113.25 Β±2.62 / 118.34 ms β 111.70 / 113.52 Β±1.74 / 116.77 ms β no change β
β QQuery 59 β 117.35 / 119.20 Β±2.44 / 123.99 ms β 120.66 / 121.70 Β±1.17 / 123.86 ms β no change β
β QQuery 60 β 39.48 / 40.06 Β±0.68 / 41.37 ms β 40.91 / 41.13 Β±0.12 / 41.25 ms β no change β
β QQuery 61 β 12.37 / 12.76 Β±0.57 / 13.87 ms β 13.18 / 13.27 Β±0.09 / 13.44 ms β no change β
β QQuery 62 β 46.03 / 46.26 Β±0.14 / 46.46 ms β 46.41 / 47.60 Β±1.35 / 50.15 ms β no change β
β QQuery 63 β 29.36 / 29.82 Β±0.36 / 30.44 ms β 28.84 / 29.52 Β±0.46 / 30.15 ms β no change β
β QQuery 64 β 364.77 / 369.50 Β±3.83 / 374.07 ms β 366.42 / 370.15 Β±2.11 / 372.89 ms β no change β
β QQuery 65 β 120.58 / 124.34 Β±3.31 / 130.49 ms β 124.53 / 126.15 Β±1.27 / 128.22 ms β no change β
β QQuery 66 β 80.40 / 81.08 Β±0.46 / 81.84 ms β 81.54 / 84.18 Β±3.14 / 89.94 ms β no change β
β QQuery 67 β 269.42 / 279.06 Β±6.12 / 286.42 ms β 270.79 / 279.93 Β±6.50 / 288.32 ms β no change β
β QQuery 68 β 12.57 / 12.84 Β±0.19 / 13.10 ms β 11.83 / 12.02 Β±0.22 / 12.44 ms β +1.07x faster β
β QQuery 69 β 58.07 / 60.40 Β±3.27 / 66.69 ms β 59.29 / 59.84 Β±0.89 / 61.60 ms β no change β
β QQuery 70 β 106.19 / 108.41 Β±1.79 / 110.88 ms β 104.73 / 109.32 Β±4.06 / 115.05 ms β no change β
β QQuery 71 β 35.54 / 35.78 Β±0.19 / 36.01 ms β 35.16 / 35.61 Β±0.31 / 35.92 ms β no change β
β QQuery 72 β 1860.30 / 1898.12 Β±32.28 / 1955.99 ms β 1639.37 / 1787.59 Β±96.81 / 1889.78 ms β +1.06x faster β
β QQuery 73 β 9.93 / 10.53 Β±0.37 / 11.04 ms β 9.55 / 10.18 Β±0.78 / 11.67 ms β no change β
β QQuery 74 β 169.66 / 172.00 Β±1.99 / 175.59 ms β 167.85 / 178.73 Β±14.43 / 206.56 ms β no change β
β QQuery 75 β 145.45 / 147.61 Β±2.49 / 152.33 ms β 155.29 / 157.89 Β±2.04 / 160.52 ms β 1.07x slower β
β QQuery 76 β 34.60 / 35.19 Β±0.41 / 35.65 ms β 35.54 / 37.19 Β±1.07 / 38.36 ms β 1.06x slower β
β QQuery 77 β 61.02 / 61.48 Β±0.65 / 62.75 ms β 60.96 / 64.28 Β±2.83 / 69.20 ms β no change β
β QQuery 78 β 212.49 / 218.63 Β±4.31 / 223.74 ms β 215.92 / 222.86 Β±8.20 / 238.70 ms β no change β
β QQuery 79 β 66.15 / 68.66 Β±3.81 / 76.22 ms β 66.10 / 67.04 Β±0.88 / 68.66 ms β no change β
β QQuery 80 β 98.21 / 99.50 Β±1.05 / 101.29 ms β 97.56 / 100.07 Β±2.31 / 103.83 ms β no change β
β QQuery 81 β 25.54 / 25.79 Β±0.13 / 25.92 ms β 26.23 / 26.40 Β±0.23 / 26.87 ms β no change β
β QQuery 82 β 16.25 / 16.88 Β±0.98 / 18.81 ms β 16.26 / 16.56 Β±0.24 / 16.86 ms β no change β
β QQuery 83 β 33.47 / 33.64 Β±0.11 / 33.79 ms β 33.53 / 33.82 Β±0.19 / 34.11 ms β no change β
β QQuery 84 β 28.95 / 31.41 Β±3.28 / 37.74 ms β 28.96 / 29.86 Β±0.66 / 30.82 ms β no change β
β QQuery 85 β 106.62 / 109.96 Β±3.08 / 115.66 ms β 102.62 / 104.79 Β±2.14 / 108.51 ms β no change β
β QQuery 86 β 27.43 / 27.85 Β±0.32 / 28.40 ms β 25.14 / 25.45 Β±0.35 / 26.12 ms β +1.09x faster β
β QQuery 87 β 62.86 / 68.92 Β±3.17 / 71.44 ms β 62.57 / 63.51 Β±1.09 / 65.53 ms β +1.09x faster β
β QQuery 88 β 64.09 / 64.55 Β±0.30 / 64.99 ms β 63.75 / 64.23 Β±0.38 / 64.89 ms β no change β
β QQuery 89 β 35.46 / 36.62 Β±0.71 / 37.43 ms β 35.04 / 36.59 Β±2.04 / 40.64 ms β no change β
β QQuery 90 β 18.18 / 18.40 Β±0.21 / 18.77 ms β 16.88 / 17.13 Β±0.23 / 17.57 ms β +1.07x faster β
β QQuery 91 β 45.46 / 46.37 Β±0.95 / 48.02 ms β 44.85 / 45.06 Β±0.23 / 45.42 ms β no change β
β QQuery 92 β 28.69 / 31.82 Β±4.79 / 41.34 ms β 29.46 / 30.06 Β±0.60 / 31.11 ms β +1.06x faster β
β QQuery 93 β 49.76 / 51.27 Β±1.21 / 53.09 ms β 48.99 / 50.10 Β±1.07 / 51.93 ms β no change β
β QQuery 94 β 37.72 / 38.24 Β±0.47 / 38.90 ms β 37.66 / 38.12 Β±0.31 / 38.64 ms β no change β
β QQuery 95 β 80.46 / 82.81 Β±1.77 / 85.42 ms β 78.31 / 79.15 Β±0.74 / 80.16 ms β no change β
β QQuery 96 β 24.53 / 26.71 Β±3.45 / 33.60 ms β 23.84 / 24.00 Β±0.18 / 24.34 ms β +1.11x faster β
β QQuery 97 β 52.64 / 54.84 Β±2.51 / 59.69 ms β 52.25 / 53.40 Β±1.89 / 57.15 ms β no change β
β QQuery 98 β 42.37 / 43.04 Β±0.60 / 44.09 ms β 42.68 / 43.35 Β±0.46 / 43.98 ms β no change β
β QQuery 99 β 69.57 / 70.17 Β±0.47 / 70.98 ms β 69.14 / 70.20 Β±1.19 / 72.48 ms β no change β
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
βββββββββββββββββββββββββββββββββββββββββββββ³ββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 9758.51ms β
β Total Time (perf_left-semi-incremental) β 9609.62ms β
β Average Time (HEAD) β 98.57ms β
β Average Time (perf_left-semi-incremental) β 97.07ms β
β Queries Faster β 18 β
β Queries Slower β 20 β
β Queries with No Change β 61 β
β Queries with Failure β 0 β
βββββββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpcds β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 50.0s |
| Peak memory | 2.1 GiB |
| Avg memory | 1.4 GiB |
| CPU user | 213.9s |
| CPU sys | 5.8s |
| Peak spill | 0 B |
**tpcds β branch**
| Metric | Value |
|--------|-------|
| Wall time | 50.0s |
| Peak memory | 2.3 GiB |
| Avg memory | 1.6 GiB |
| CPU user | 201.3s |
| CPU sys | 5.7s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5471138814
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/24794#issuecomment-5471125244)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5471125244-2048-qhnhk 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/left-semi-incremental (9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a) to 61bf6b9 (merge-base) [diff](https://github.com/apache/datafusion/compare/61bf6b96cc07d6a0518014dec7fe86fbef76576e..9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a)
<details><summary>Run configuration</summary>
```yaml
run benchmark tpcds
```
</details>
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5471206216
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/24794#issuecomment-5471125244)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing perf/left-semi-incremental (9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a) to 61bf6b9 (merge-base) [diff](https://github.com/apache/datafusion/compare/61bf6b96cc07d6a0518014dec7fe86fbef76576e..9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a)
<details><summary>Run configuration</summary>
```yaml
run benchmark tpch
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and perf_left-semi-incremental
--------------------
Benchmark tpch_sf1.json
--------------------
βββββββββββββ³βββββββββββ³βββββββββββββββββββββββββββββ³ββββββββββββ
β Query β HEAD β perf_left-semi-incremental β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 1 β 38.92 ms β 38.73 ms β no change β
β QQuery 2 β 18.90 ms β 18.80 ms β no change β
β QQuery 3 β 28.95 ms β 28.79 ms β no change β
β QQuery 4 β 17.95 ms β 17.48 ms β no change β
β QQuery 5 β 36.45 ms β 36.43 ms β no change β
β QQuery 6 β 16.22 ms β 16.23 ms β no change β
β QQuery 7 β 42.21 ms β 41.33 ms β no change β
β QQuery 8 β 41.13 ms β 41.61 ms β no change β
β QQuery 9 β 50.27 ms β 50.97 ms β no change β
β QQuery 10 β 43.05 ms β 42.64 ms β no change β
β QQuery 11 β 13.61 ms β 13.72 ms β no change β
β QQuery 12 β 24.05 ms β 24.24 ms β no change β
β QQuery 13 β 40.80 ms β 39.67 ms β no change β
β QQuery 14 β 24.65 ms β 24.16 ms β no change β
β QQuery 15 β 31.35 ms β 31.19 ms β no change β
β QQuery 16 β 14.08 ms β 14.08 ms β no change β
β QQuery 17 β 72.67 ms β 71.29 ms β no change β
β QQuery 18 β 59.88 ms β 59.49 ms β no change β
β QQuery 19 β 32.79 ms β 32.69 ms β no change β
β QQuery 20 β 31.49 ms β 31.01 ms β no change β
β QQuery 21 β 56.56 ms β 54.86 ms β no change β
β QQuery 22 β 14.37 ms β 13.97 ms β no change β
βββββββββββββ΄βββββββββββ΄βββββββββββββββββββββββββββββ΄ββββββββββββ
βββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββ
β Benchmark Summary β β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 750.35ms β
β Total Time (perf_left-semi-incremental) β 743.38ms β
β Average Time (HEAD) β 34.11ms β
β Average Time (perf_left-semi-incremental) β 33.79ms β
β Queries Faster β 0 β
β Queries Slower β 0 β
β Queries with No Change β 22 β
β Queries with Failure β 0 β
βββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββ
Distribution per query (min / mean Β±stddev / max):
Comparing HEAD and perf_left-semi-incremental
--------------------
Benchmark tpch_sf1.json
--------------------
βββββββββββββ³βββββββββββββββββββββββββββββββββ³βββββββββββββββββββββββββββββββββ³βββββββββββββββ
β Query β HEAD β perf_left-semi-incremental β Change β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 1 β 38.92 / 40.24 Β±1.27 / 41.90 ms β 38.73 / 40.84 Β±2.30 / 45.11 ms β no change β
β QQuery 2 β 18.90 / 19.51 Β±0.92 / 21.34 ms β 18.80 / 19.76 Β±0.93 / 21.36 ms β no change β
β QQuery 3 β 28.95 / 29.31 Β±0.29 / 29.77 ms β 28.79 / 29.13 Β±0.31 / 29.68 ms β no change β
β QQuery 4 β 17.95 / 18.39 Β±0.65 / 19.67 ms β 17.48 / 17.98 Β±0.63 / 19.13 ms β no change β
β QQuery 5 β 36.45 / 37.38 Β±1.53 / 40.41 ms β 36.43 / 36.72 Β±0.20 / 36.97 ms β no change β
β QQuery 6 β 16.22 / 16.58 Β±0.45 / 17.36 ms β 16.23 / 16.89 Β±0.99 / 18.85 ms β no change β
β QQuery 7 β 42.21 / 42.88 Β±0.75 / 44.33 ms β 41.33 / 42.17 Β±0.61 / 42.98 ms β no change β
β QQuery 8 β 41.13 / 41.87 Β±0.92 / 43.67 ms β 41.61 / 42.81 Β±1.26 / 44.96 ms β no change β
β QQuery 9 β 50.27 / 50.81 Β±0.59 / 51.63 ms β 50.97 / 51.67 Β±0.58 / 52.41 ms β no change β
β QQuery 10 β 43.05 / 43.88 Β±0.87 / 45.47 ms β 42.64 / 43.64 Β±1.21 / 46.02 ms β no change β
β QQuery 11 β 13.61 / 13.81 Β±0.16 / 14.00 ms β 13.72 / 13.99 Β±0.24 / 14.40 ms β no change β
β QQuery 12 β 24.05 / 24.53 Β±0.24 / 24.68 ms β 24.24 / 24.80 Β±0.40 / 25.35 ms β no change β
β QQuery 13 β 40.80 / 41.00 Β±0.15 / 41.26 ms β 39.67 / 40.20 Β±0.78 / 41.73 ms β no change β
β QQuery 14 β 24.65 / 24.85 Β±0.17 / 25.05 ms β 24.16 / 26.78 Β±1.38 / 28.02 ms β 1.08x slower β
β QQuery 15 β 31.35 / 32.03 Β±0.73 / 33.23 ms β 31.19 / 32.95 Β±0.95 / 34.10 ms β no change β
β QQuery 16 β 14.08 / 14.22 Β±0.15 / 14.48 ms β 14.08 / 14.16 Β±0.08 / 14.32 ms β no change β
β QQuery 17 β 72.67 / 73.78 Β±1.67 / 77.10 ms β 71.29 / 73.10 Β±1.38 / 75.53 ms β no change β
β QQuery 18 β 59.88 / 61.29 Β±1.11 / 63.20 ms β 59.49 / 60.69 Β±0.77 / 61.72 ms β no change β
β QQuery 19 β 32.79 / 33.76 Β±1.17 / 35.98 ms β 32.69 / 33.66 Β±1.00 / 35.51 ms β no change β
β QQuery 20 β 31.49 / 32.11 Β±0.82 / 33.71 ms β 31.01 / 31.56 Β±0.45 / 32.31 ms β no change β
β QQuery 21 β 56.56 / 58.26 Β±0.95 / 59.40 ms β 54.86 / 55.55 Β±0.67 / 56.81 ms β no change β
β QQuery 22 β 14.37 / 14.55 Β±0.17 / 14.83 ms β 13.97 / 14.37 Β±0.23 / 14.68 ms β no change β
βββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββ
βββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββ
β Benchmark Summary β β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 765.02ms β
β Total Time (perf_left-semi-incremental) β 763.43ms β
β Average Time (HEAD) β 34.77ms β
β Average Time (perf_left-semi-incremental) β 34.70ms β
β Queries Faster β 0 β
β Queries Slower β 1 β
β Queries with No Change β 21 β
β Queries with Failure β 0 β
βββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**tpch β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 5.0s |
| Peak memory | 1.3 GiB |
| Avg memory | 503.9 MiB |
| CPU user | 21.6s |
| CPU sys | 1.6s |
| Peak spill | 0 B |
**tpch β branch**
| Metric | Value |
|--------|-------|
| Wall time | 5.0s |
| Peak memory | 1.2 GiB |
| Avg memory | 500.9 MiB |
| CPU user | 21.4s |
| CPU sys | 1.8s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-6512735887
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Emit LeftSemi hash join rows while probing [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #24794:
URL: https://github.com/apache/datafusion/pull/24794#issuecomment-5471134907
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/24794#issuecomment-5471125244)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5471125244-2047-dghwr 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/left-semi-incremental (9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a) to 61bf6b9 (merge-base) [diff](https://github.com/apache/datafusion/compare/61bf6b96cc07d6a0518014dec7fe86fbef76576e..9b9d1d7da3c3c6a37ab1f8a1b7e470b6d7207b5a)
<details><summary>Run configuration</summary>
```yaml
run benchmark tpch
```
</details>
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org