[GitHub] [arrow-datafusion] alamb opened a new issue #642: Allow extension nodes to correctly plan physical expressions with relations

[GitBox <gi...@apache.org>]

alamb opened a new issue #642:
URL: https://github.com/apache/arrow-datafusion/issues/642


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   In IOx we have several  extension nodes with `Expr`s which we want to compile to  `PhysicalExpr` as part of a plan
   
   This was previously annoying as the signature of the extension planner is:
   
   ```rust
       fn plan_extension(
           &self,
           node: &dyn UserDefinedLogicalNode,
           inputs: &[Arc<dyn ExecutionPlan>],
           ctx_state: &ExecutionContextState,
       ) -> Result<Option<Arc<dyn ExecutionPlan>>> {
   ```
   
   Which offers no way to plan physical expressions, so I had to do something like the following workaround
   
   ```rust
               // create physical expressions in extension planner)
               let input_schema = inputs[0].schema();
               let physical_planner = DefaultPhysicalPlanner::default();
               let split_expr = physical_planner.create_physical_expr(
                   stream_split.split_expr(),
                   &input_schema,
                   ctx_state,
               )?;
   ```
   
   However, with the introduction of qualified names, now I have to provide a `DFSchema` (which should have come from the input LogicalPlan which I do not have access to at this point).
   
   ```rust
               // create physical expressions in extension planner)
               let input_schema = inputs[0].schema();
               let input_df_schema = DFSchema::try_from_qualified_schema("foo", &input_schema).unwrap();
               let physical_planner = DefaultPhysicalPlanner::default();
               let split_expr = physical_planner.create_physical_expr(
                   stream_split.split_expr(),
                   &input_df_schema,
                   &input_schema,
                   ctx_state,
               )?;
   ```
   
   (note the "foo" needs to be the correct table name"
   
   **Describe the solution you'd like**
   1. Add `create_physical_expr` to the `PhysicalPlanner` trait 
   2. Pass an instance of the planner to the plan extension (rather than a `ctx_state`)
   3. Pass the input logical schema to each "plan extension" call as well
   
   Basically as the internals of DataFusion evolve we also need to evolve the interfaces along with it.
   
   **Describe alternatives you've considered**
   A clear and concise description of any alternative solutions or features you've considered.
   
   **Additional context**
   Add any other context or screenshots about the feature request here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb closed issue #642: Allow extension nodes to correctly plan physical expressions with relations

[GitBox <gi...@apache.org>]

alamb closed issue #642:
URL: https://github.com/apache/arrow-datafusion/issues/642


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org