You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/10/14 13:55:32 UTC
[I] Extract a shared metrics-preserving exec wrapper from CometFilterExec and CometProjectionExec [datafusion-comet]
andygrove opened a new issue, #6975:
URL: https://github.com/apache/datafusion-comet/issues/6975
### What is the problem the feature request solves?
`CometFilterExec` and `CometProjectionExec` are renamed copies of the same wrapper:
- [operators/src/filter.rs#L67-L238](https://github.com/apache/datafusion-comet/blob/7d8d9b1544b734c85a93cca2cc9d5380ad70accd/native/operators/src/filter.rs#L67-L238)
- [operators/src/projection.rs#L63-L237](https://github.com/apache/datafusion-comet/blob/7d8d9b1544b734c85a93cca2cc9d5380ad70accd/native/operators/src/projection.rs#L63-L237)
Each holds a DataFusion exec plus a stable `ExecutionPlanMetricsSet`, with identical `replace_input`, `with_execution_input`, `replace_children`, `execute` (which registers fresh metrics on the stable set) and about 15 pure delegating `ExecutionPlan` methods. Apart from type names and doc comments, the only differences are the filter-only `predicate()`/`has_projection()` accessors and that the projection wrapper forwards `benefits_from_input_partitioning` while the filter wrapper does not.
That last difference shows how the copies drift: whenever DataFusion adds or changes an `ExecutionPlan` method, each wrapper has to be updated by hand, and the next operator that needs stable metrics will copy this a third time. The metric-identity logic in `execute` is subtle and should live in one place.
### Describe the potential solution
- Introduce a generic wrapper in `native/operators` (e.g. `MetricOwningExec<T: ExecutionPlan>`, or a small macro) that implements `replace_input`, `with_execution_input`, `replace_children`, `execute` and `metrics` once and delegates the remaining `ExecutionPlan` methods.
- Make `CometFilterExec`/`CometProjectionExec` thin newtypes or aliases that only add their specific accessors.
- Decide explicitly whether `benefits_from_input_partitioning` should be forwarded for filter too.
### Additional context
Line references are pinned to commit 7d8d9b1544b7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Extract a shared metrics-preserving exec wrapper from CometFilterExec and CometProjectionExec [datafusion-comet]
Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on issue #6975:
URL: https://github.com/apache/datafusion-comet/issues/6975#issuecomment-6218693486
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Extract a shared metrics-preserving exec wrapper from CometFilterExec and CometProjectionExec [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #6975: Extract a shared metrics-preserving exec wrapper from CometFilterExec and CometProjectionExec
URL: https://github.com/apache/datafusion-comet/issues/6975
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org