You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "adriangb (via GitHub)" <gi...@apache.org> on 2026/03/10 05:20:48 UTC
[PR] Optimize logical optimizer: skip map_subqueries + in-place rewriting [datafusion]
adriangb opened a new pull request, #20837:
URL: https://github.com/apache/datafusion/pull/20837
## Summary
- **map_subqueries short-circuit**: skip expression tree walks when no subquery expressions exist (biggest contributor)
- **plan_has_subqueries per-pass check**: bypass `rewrite_with_subqueries` entirely when plan has no subqueries
- **rewrite_plan_in_place with Arc::make_mut**: avoid `Arc::unwrap_or_clone` + `Arc::new` cycle during tree traversal
- Adds optimizer-only benchmarks that isolate optimizer perf from SQL parsing/analysis
## Benchmark Results (optimizer-only)
| Benchmark | Baseline | Optimized | Change |
|-----------|----------|-----------|--------|
| optimizer_select_one_from_700 | 196 µs | 201 µs | +2.7% (noise) |
| optimizer_select_all_from_1000 | 4.84 ms | 4.25 ms | **-12%** |
| optimizer_join_chain_4 | 150 µs | 136 µs | **-9%** |
| optimizer_join_chain_8 | 462 µs | 426 µs | **-8%** |
| optimizer_wide_filter_200 | 4.95 ms | 3.41 ms | **-31%** |
| optimizer_wide_aggregate_100 | 1.98 ms | 1.49 ms | **-25%** |
| optimizer_join_4_with_agg_filter | 429 µs | 358 µs | **-17%** |
| optimizer_tpch_all | 13.96 ms | 11.54 ms | **-17%** |
| optimizer_tpcds_all | 255.9 ms | 213.1 ms | **-17%** |
## Test plan
- [x] All 642 optimizer unit tests pass
- [x] Benchmarks confirmed across 2 independent runs
- [ ] Run full CI
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize logical optimizer: skip map_subqueries + in-place rewriting [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20837:
URL: https://github.com/apache/datafusion/pull/20837#issuecomment-4414221386
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize logical optimizer: skip map_subqueries + in-place rewriting [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20837: Optimize logical optimizer: skip map_subqueries + in-place rewriting
URL: https://github.com/apache/datafusion/pull/20837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize logical optimizer: skip map_subqueries + in-place rewriting [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20837:
URL: https://github.com/apache/datafusion/pull/20837#issuecomment-4469817912
Revived and superseded by #22298 — rebased onto current `main` and re-benchmarked. GitHub does not allow reopening a closed PR after its branch has been force-pushed, hence a fresh PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org