You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "geoffreyclaude (via GitHub)" <gi...@apache.org> on 2026/05/27 11:16:59 UTC
[PR] Support transparent ExecutionPlan downcasts [datafusion]
geoffreyclaude opened a new pull request, #22559:
URL: https://github.com/apache/datafusion/pull/22559
## Summary
This draft PR proposes one possible fix for [apache/datafusion#22557](https://github.com/apache/datafusion/issues/22557): add an explicit `ExecutionPlan::transparent_child()` hook for wrapper nodes that want their wrapped plan type to remain visible through `ExecutionPlan` downcasting.
The implementation keeps the new DataFusion 54 `Any`-based downcast helpers, but changes `dyn ExecutionPlan::is::<T>()` and `downcast_ref::<T>()` to:
- check the concrete plan first, so wrappers can still identify as themselves;
- then follow `transparent_child()` when the concrete type does not match;
- stop if no transparent child is provided.
This restores an opt-in version of the transparent-wrapper behavior that was previously possible by overriding `ExecutionPlan::as_any`.
## API naming
I used `transparent_child()` because it is short and lines up with the existing “transparent node” wording around execution plans. The docs call out that this hook is only for type introspection and is independent from `children()`, so it should not be used for traversal or optimizer rewrites.
This is only a suggested implementation for the linked issue. Other solution proposals, including different API names or a different shape for the hook, are very welcome.
## Tests
Added coverage for:
- downcasting through a single transparent wrapper;
- preserving the wrapper’s own concrete type identity;
- downcasting through nested transparent wrappers.
## Validation
- `cargo test -p datafusion-physical-plan execution_plan_downcast`
- `cargo test -p datafusion-physical-plan --lib`
- `cargo fmt --all -- --check`
- `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3312914303
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
> Should we revert the new `:Any` instead? I think it was just supposed to be a cleanup but we didn't realize it would have downstream impacts 🤔
@alamb Honestly I prefer the new API with `downcast_delegate`: it's explicit and clearer.
> Also I wonder if there is some way to avoid using as_any / delegating logic at all ... But to do that we would have to remove all uses of as_any / downcasting I suspect
That would be a pretty big breaking change as well, as the `is a` check is quite widespread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #22559:
URL: https://github.com/apache/datafusion/pull/22559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3310945973
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +813,44 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan is a transparent wrapper, also checks its
+ /// [`ExecutionPlan::transparent_child`].
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ if (self as &dyn Any).is::<T>() {
+ return true;
+ }
+
+ let Some(child) = self.transparent_child() else {
+ return false;
+ };
+ if std::ptr::eq(self, child) {
+ return false;
+ }
Review Comment:
Oh, this was from the prior commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3312934561
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate instead
Review Comment:
There's the unit test with `DowncastDelegatingExec` which already shows a possible implementation. Would you also like an inline comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3310983901
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +813,44 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan is a transparent wrapper, also checks its
+ /// [`ExecutionPlan::transparent_child`].
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ if (self as &dyn Any).is::<T>() {
+ return true;
+ }
+
+ let Some(child) = self.transparent_child() else {
+ return false;
+ };
+ if std::ptr::eq(self, child) {
+ return false;
+ }
Review Comment:
None, I removed this weirdness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4558064089
Looks good to me -- thank you everyone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4557280538
> Should we revert the hew `:Any` instead? I think it was just supposed to be a cleanup but we didn't realize it would have downstream impacts 🤔
@alamb Honestly I prefer the new API with `downcast_delegate`: it's explicit and clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3310962699
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate before
Review Comment:
nit: replace "before" with "instead of" because I don't believe we are checking the wrapper
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4554222459
I'll review as soon as it's marked ready, but at a high level looks good so far
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4554660112
> I'll review as soon as it's marked ready, but at a high level looks good so far
@timsaucer Should be ready for review now. I just renamed the API to `downcast_delegate` which seemed clearer to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4555055097
Do we need to also open a PR to target `branch-54`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3311046896
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
It seems like the earlier PR would still work for all use cases, right? Because you're always choosing to downcast to a specific type, it seems like it would enable downcasting to either (or many, depending on levels) type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3310998612
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
Yes, and this is intentional to mimic the old `as_any` behavior. I've updated the PR description to be clear about this choice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3312944844
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate instead
Review Comment:
Yeah, I was thinking an inline comment so that if someone else is reading (just) the ExecutionPlan docs they can more quickly understand if they need to worry about implementing this method or not (probably not)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3313031104
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate instead
Review Comment:
done: I amended the commit to be clearer in the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3310600076
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +813,44 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan is a transparent wrapper, also checks its
+ /// [`ExecutionPlan::transparent_child`].
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ if (self as &dyn Any).is::<T>() {
+ return true;
+ }
+
+ let Some(child) = self.transparent_child() else {
+ return false;
+ };
+ if std::ptr::eq(self, child) {
+ return false;
+ }
Review Comment:
What's the corner case here?
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
Won't this prevent you from downcasting to the wrapper type?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3312514274
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate instead
Review Comment:
It might be good to add a specific example here (like a wrapper node that wants to observe changes but still be treated as the underlying node when DataFusion checks for patterns using `downcast` 🤔
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
Should we revert the hew `:Any` instead? I think it was just supposed to be a cleanup but we didn't realize it would have downstream impacts 🤔
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
Also I wonder if there is some way to avoid using `as_any` / delegating logic at all ... But to do that we would have to remove all uses of `as_any` / `downcasting` I suspect
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3311078311
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
The previous PR would stop on the first match, rather than go all the way to the leaf child. I agree this is being pedantic, as "real world" cases would probably never have a case where you'd need to match the leaf child but get interrupted by an intermediate node in a "wrapper chain". However, I still find the current implementation clearer and easier to reason about: the leaf node is what is returned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#issuecomment-4555160618
> Do we need to also open a PR to target `branch-54`?
@timsaucer I opened https://github.com/apache/datafusion/pull/22565, which is a clean cherry-pick of this PR's single commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3312914303
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
> Should we revert the hew `:Any` instead? I think it was just supposed to be a cleanup but we didn't realize it would have downstream impacts 🤔
@alamb Honestly I prefer the new API with `downcast_delegate`: it's explicit and clearer.
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -718,20 +738,32 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
impl dyn ExecutionPlan {
/// Returns `true` if the plan is of type `T`.
///
+ /// If this plan provides a [`ExecutionPlan::downcast_delegate`], delegates
+ /// to it.
+ ///
/// Prefer this over `downcast_ref::<T>().is_some()`. Works correctly when
/// called on `Arc<dyn ExecutionPlan>` via auto-deref.
pub fn is<T: ExecutionPlan>(&self) -> bool {
- (self as &dyn Any).is::<T>()
+ match self.downcast_delegate() {
+ Some(delegate) => delegate.is::<T>(),
+ None => (self as &dyn Any).is::<T>(),
+ }
Review Comment:
> Should we revert the new `:Any` instead? I think it was just supposed to be a cleanup but we didn't realize it would have downstream impacts 🤔
@alamb Honestly I prefer the new API with `downcast_delegate`: it's explicit and clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support transparent ExecutionPlan downcasts [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #22559:
URL: https://github.com/apache/datafusion/pull/22559#discussion_r3311017647
##########
datafusion/physical-plan/src/execution_plan.rs:
##########
@@ -115,6 +115,26 @@ pub trait ExecutionPlan: Any + Debug + DisplayAs + Send + Sync {
}
}
+ /// Returns the plan that provides this plan's public
+ /// [`ExecutionPlan`] downcast identity.
+ ///
+ /// This hook is for wrapper nodes that delegate their public downcast
+ /// identity to another plan while adding cross-cutting behavior such as
+ /// instrumentation. The default implementation returns `None`, meaning this
+ /// plan's concrete type is used for type introspection.
+ ///
+ /// The `is` and `downcast_ref` helpers follow the returned delegate before
Review Comment:
done: I amended the commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org