You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "rgbuilds (via GitHub)" <gi...@apache.org> on 2026/10/03 18:07:13 UTC

Re: [PR] feat: derive per-column pruning guarantees from tuple IN lists [datafusion]

rgbuilds commented on code in PR #25469:
URL: https://github.com/apache/datafusion/pull/25469#discussion_r4174252842


##########
datafusion/physical-expr/src/utils/guarantee.rs:
##########
@@ -377,6 +388,85 @@ impl<'a> GuaranteeBuilder<'a> {
     }
 }
 
+/// Project necessary per-column guarantees; the original predicate retains tuple correlation.
+fn project_struct_in_list(
+    inlist: &crate::expressions::InListExpr,
+) -> Option<Vec<(&crate::expressions::Column, Vec<ScalarValue>)>> {
+    if inlist.negated() || inlist.is_empty() {
+        return None;
+    }
+    let expr = inlist.expr().downcast_ref::<crate::ScalarFunctionExpr>()?;
+    let literal_args = expr
+        .args()
+        .iter()
+        .map(|arg| {
+            arg.downcast_ref::<crate::expressions::Literal>()
+                .map(|lit| lit.value().clone())
+        })
+        .collect::<Vec<_>>();
+    let mapping = expr.fun().struct_field_mapping(&literal_args)?;
+    if mapping.field_accessor.signature().volatility != Volatility::Immutable {
+        return None;
+    }
+    let tuples = inlist
+        .list()
+        .iter()
+        .map(|value| {
+            let literal = value.downcast_ref::<crate::expressions::Literal>()?;
+            let ScalarValue::Struct(array) = literal.value() else {
+                return None;
+            };
+            (array.len() == 1).then_some(literal.value())
+        })
+        .collect::<Option<Vec<_>>>()?;
+    // Null tuples cannot make a positive IN predicate true.
+    let tuples = ScalarValue::iter_to_array(
+        tuples.into_iter().filter(|tuple| !tuple.is_null()).cloned(),
+    )
+    .ok()?;
+    let batch = RecordBatch::try_from_iter([("tuple", tuples)]).ok()?;
+
+    let mut projected = Vec::new();
+    for (accessor_args, source_index) in mapping.fields {

Review Comment:
   Thanks. Returning no mapping when a field name is repeated prevents the ambiguous inference, and the focused regression test covers the case I raised. This resolves my concern.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org