[GitHub] [arrow-datafusion] thinkharderdev opened a new issue, #5424: Filter Pushdown no longer works for extension plans

["thinkharderdev (via GitHub)" <gi...@apache.org>]

thinkharderdev opened a new issue, #5424:
URL: https://github.com/apache/arrow-datafusion/issues/5424

   **Describe the bug**
   A clear and concise description of what the bug is.
   
   The filter pushdown optimizer will not push down predicates through extension plans even if the extension plan allows it.
   
   **To Reproduce**
   Steps to reproduce the behavior:
   
   ```
       #[derive(Debug)]
       struct NoopPlan {
           input: Vec<LogicalPlan>,
           schema: DFSchemaRef,
       }
   
       impl UserDefinedLogicalNode for NoopPlan {
           fn as_any(&self) -> &dyn Any {
               self
           }
   
           fn inputs(&self) -> Vec<&LogicalPlan> {
               self.input.iter().collect()
           }
   
           fn schema(&self) -> &DFSchemaRef {
               &self.schema
           }
   
           fn expressions(&self) -> Vec<Expr> {
               self.input
                   .iter()
                   .flat_map(|child| child.expressions())
                   .collect()
           }
   
           fn prevent_predicate_push_down_columns(&self) -> HashSet<String> {
               HashSet::from_iter(vec!["c".to_string()])
           }
   
           fn fmt_for_explain(&self, f: &mut Formatter) -> std::fmt::Result {
               write!(f, "NoopPlan")
           }
   
           fn from_template(
               &self,
               _exprs: &[Expr],
               inputs: &[LogicalPlan],
           ) -> Arc<dyn UserDefinedLogicalNode> {
               Arc::new(Self {
                   input: inputs.iter().cloned().collect(),
                   schema: self.schema.clone(),
               })
           }
       }
   
       #[test]
       fn user_defined_plan() -> Result<()> {
           let table_scan = test_table_scan()?;
   
           let custom_plan = LogicalPlan::Extension(Extension {
               node: Arc::new(NoopPlan {
                   input: vec![table_scan.clone()],
                   schema: table_scan.schema().clone(),
               }),
           });
           let plan = LogicalPlanBuilder::from(custom_plan)
               .filter(col("a").eq(lit(1i64)))?
               .build()?;
   
           // Push filter below NoopPlan
           let expected = "\
               NoopPlan\
               \n  Filter: test.a = Int64(1)\
               \n    TableScan: test";
           assert_optimized_plan_eq(&plan, expected)?;
      }
   ```
   
   **Expected behavior**
   A clear and concise description of what you expected to happen.
   
   If a predicate does not contain any columns in `UserDefinedLogicalNode::prevent_predicate_push_down_columns` it should get pushed down
   
   **Additional context**
   Add any other context about the problem here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] thinkharderdev closed issue #5424: Filter Pushdown no longer works for extension plans

["thinkharderdev (via GitHub)" <gi...@apache.org>]

thinkharderdev closed issue #5424: Filter Pushdown no longer works for extension plans
URL: https://github.com/apache/arrow-datafusion/issues/5424


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org