You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "askalt (via GitHub)" <gi...@apache.org> on 2026/03/12 10:40:59 UTC
[PR] fix: do not recompute hash join exec properties if not required [datafusion]
askalt opened a new pull request, #20900:
URL: https://github.com/apache/datafusion/pull/20900
## Rationale for this change
The check on same properties was accidentally deleted which led to `reset_plan_states` benchmark performance drop: from ~2us to ~3ms. This patch fixes it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#discussion_r2923862445
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -1508,7 +1508,7 @@ pub fn reset_plan_states(plan: Arc<dyn ExecutionPlan>) -> Result<Arc<dyn Executi
/// replace is requested.
/// The size of `children` must be equal to the size of `ExecutionPlan::children()`.
pub fn has_same_children_properties(
- plan: &Arc<impl ExecutionPlan>,
+ plan: &impl ExecutionPlan,
Review Comment:
this is needed because the builder doesn't have an Arc<ExecutionPlan>, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#issuecomment-4046022795
🤖: Benchmark completed
<details><summary>Details</summary>
<p>
```
group askalt_fix-hash-join-with-new-children-perf main
----- ------------------------------------------- ----
query1 1.00 2.6±0.01µs ? ?/sec 1.00 2.6±0.02µs ? ?/sec
query2 1.00 3.3±0.01µs ? ?/sec 1469.86 4.8±0.01ms ? ?/sec
query3 1.01 987.7±13.93ns ? ?/sec 1.00 980.8±3.79ns ? ?/sec
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#issuecomment-4045848897
🤖 `./gh_compare_branch_bench.sh` [compare_branch_bench.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/compare_branch_bench.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing askalt/fix-hash-join-with-new-children-perf (e0cb11a0cd590113c73fe40286548fe916b1059f) to 8d9b080882179b618a2057e042fc32865f6484b4 [diff](https://github.com/apache/datafusion/compare/8d9b080882179b618a2057e042fc32865f6484b4..e0cb11a0cd590113c73fe40286548fe916b1059f)
BENCH_NAME=reset_plan_states
BENCH_COMMAND=cargo bench --features=parquet --bench reset_plan_states
BENCH_FILTER=
BENCH_BRANCH_NAME=askalt_fix-hash-join-with-new-children-perf
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#discussion_r2923922865
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -1508,7 +1508,7 @@ pub fn reset_plan_states(plan: Arc<dyn ExecutionPlan>) -> Result<Arc<dyn Executi
/// replace is requested.
/// The size of `children` must be equal to the size of `ExecutionPlan::children()`.
pub fn has_same_children_properties(
- plan: &Arc<impl ExecutionPlan>,
+ plan: &impl ExecutionPlan,
Review Comment:
Agreed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#issuecomment-4045848339
run benchmark reset_plan_states
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "askalt (via GitHub)" <gi...@apache.org>.
askalt commented on code in PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#discussion_r2923885392
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -1508,7 +1508,7 @@ pub fn reset_plan_states(plan: Arc<dyn ExecutionPlan>) -> Result<Arc<dyn Executi
/// replace is requested.
/// The size of `children` must be equal to the size of `ExecutionPlan::children()`.
pub fn has_same_children_properties(
- plan: &Arc<impl ExecutionPlan>,
+ plan: &impl ExecutionPlan,
Review Comment:
Yes, it is so. It looks ok to change this argument type as this API was not released yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20900:
URL: https://github.com/apache/datafusion/pull/20900#issuecomment-4046335992
> query2 1.00 3.3±0.01µs ? ?/sec 1469.86 4.8±0.01ms ? ?/sec
🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: do not recompute hash join exec properties if not required [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #20900:
URL: https://github.com/apache/datafusion/pull/20900
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org