You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ga...@apache.org on 2026/03/05 07:47:29 UTC
(datafusion) branch wire-up-with-new-state-data-source created (now 365e3e742f)
This is an automated email from the ASF dual-hosted git repository.
gabotechs pushed a change to branch wire-up-with-new-state-data-source
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 365e3e742f Wire up with_new_state with DataSource
This branch includes the following new commits:
new 365e3e742f Wire up with_new_state with DataSource
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: Wire up with_new_state with DataSource
Posted by ga...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
gabotechs pushed a commit to branch wire-up-with-new-state-data-source
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 365e3e742f5217310f2d319e1778a4285b44fa66
Author: Gabriel Musat Mestre <ga...@datadoghq.com>
AuthorDate: Thu Mar 5 08:47:18 2026 +0100
Wire up with_new_state with DataSource
---
datafusion/datasource/src/source.rs | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/datafusion/datasource/src/source.rs b/datafusion/datasource/src/source.rs
index bdb45e8e3b..deea108139 100644
--- a/datafusion/datasource/src/source.rs
+++ b/datafusion/datasource/src/source.rs
@@ -228,6 +228,22 @@ pub trait DataSource: Send + Sync + Debug {
&self,
f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion>;
+
+ /// Injects arbitrary run-time state into this DataSource, returning a new instance
+ /// that incorporates that state *if* it is relevant to the concrete DataSource implementation.
+ ///
+ /// This is a generic entry point: the `state` can be any type wrapped in
+ /// `Arc<dyn Any + Send + Sync>`. A data source that cares about the state should
+ /// down-cast it to the concrete type it expects and, if successful, return a
+ /// modified copy of itself that captures the provided value. If the state is
+ /// not applicable, the default behaviour is to return `None` so that parent
+ /// nodes can continue propagating the attempt further down the plan tree.
+ fn with_new_state(
+ &self,
+ _state: Arc<dyn Any + Send + Sync>,
+ ) -> Option<Arc<dyn DataSource>> {
+ None
+ }
}
/// [`ExecutionPlan`] that reads one or more files
@@ -432,6 +448,18 @@ impl ExecutionPlan for DataSourceExec {
as Arc<dyn ExecutionPlan>
})
}
+
+ fn with_new_state(
+ &self,
+ state: Arc<dyn Any + Send + Sync>,
+ ) -> Option<Arc<dyn ExecutionPlan>> {
+ self.data_source
+ .with_new_state(state)
+ .map(|new_data_source| {
+ Arc::new(self.clone().with_data_source(new_data_source))
+ as Arc<dyn ExecutionPlan>
+ })
+ }
}
impl DataSourceExec {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org