You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "va24nsh (via GitHub)" <gi...@apache.org> on 2025/11/09 18:51:39 UTC
[PR] feat: Track individual expr's execution time in ProjectionExec metrics [datafusion]
va24nsh opened a new pull request, #18570:
URL: https://github.com/apache/datafusion/pull/18570
## Which issue does this PR close?
- First Part of #18456
## Rationale for this change
Currently, `EXPLAIN ANALYZE` for `ProjectionExec` only shows aggregated metrics like total elapsed compute time, but doesn't provide visibility into how much time individual expressions within the projection consume. This makes it difficult to identify performance bottlenecks when debugging slow queries with complex projection expressions.
## What changes are included in this PR?
**1. Enhanced `ProjectionExec` with per-expression timing metrics:**
- Added per-expression Time metrics collection in `ProjectionStream`
- Each projection expression gets its own timer using the expression alias as the metric name
- Metrics are properly registered with the execution plan's metrics set
**2. Extended metrics display in `EXPLAIN ANALYZE`:**
- Modified `display.rs` to detect ProjectionExec plans
- Added `get_projection_expr_metrics()` helper function to extract per-expression timing data
- Enhanced metrics output to include per-expression times alongside standard metrics
**3. Added intelligent time formatting:**
- Implemented proper time unit formatting (ns, μs, ms, s) based on duration magnitude
- Maintains consistency with DataFusion's existing time formatting standards
## Are these changes tested?
**Yes** - The tests have been added to check the proper functionality of latest functions.
## Are there any user-facing changes?
**Yes** - This is an enhancement to the existing `EXPLAIN ANALYZE` functionality
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Track individual expr's execution time in ProjectionExec metrics [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on PR #18570:
URL: https://github.com/apache/datafusion/pull/18570#issuecomment-3508977814
Thanks for the efforts, but I'll skip follow-up due to https://datafusion.apache.org/contributor-guide/index.html#why-fully-ai-generated-prs-without-understanding-are-not-helpful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Track individual expr's execution time in ProjectionExec metrics [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 closed pull request #18570: feat: Track individual expr's execution time in ProjectionExec metrics
URL: https://github.com/apache/datafusion/pull/18570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org