You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/03 12:25:02 UTC

(datafusion) branch main updated: fix: literal-on-left equality no longer collapses the filter to false (#24763)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git


The following commit(s) were added to refs/heads/main by this push:
     new 20d1c56761 fix: literal-on-left equality no longer collapses the filter to false (#24763)
20d1c56761 is described below

commit 20d1c5676118e0bca955ccf92c0d2cf665384d0d
Author: Bharadwaj Pendyala <bh...@gmail.com>
AuthorDate: Thu Sep 3 12:15:10 2026 +0000

    fix: literal-on-left equality no longer collapses the filter to false (#24763)
    
    ## Which issue does this PR close?
    
    No separate issue. I found this while reading `simplify_predicates`.
    
    ## Rationale for this change
    
    `WHERE s = 'a' AND 'a' = s` returns no rows, where one row is expected:
    
    ```sql
    > CREATE TABLE t(s VARCHAR) AS VALUES ('a'), ('b');
    > SELECT * FROM t WHERE s = 'a' AND 'a' = s;
    0 row(s) fetched.
    ```
    
    Either half on its own returns `a`, and the same query against an INT
    column returns the row.
    
    On `main` (4d3e79e), `EXPLAIN VERBOSE` shows the filter turning into a
    constant between two rules:
    
    ```
    logical_plan after simplify_expressions   Filter: t.s = Utf8View("a") AND Utf8View("a") = t.s
    logical_plan after push_down_filter       Filter: Boolean(false)
    logical_plan after eliminate_filter       EmptyRelation: rows=0
    ```
    
    `PushDownFilter` splits the conjuncts and calls `simplify_predicates`.
    It accepts both `<col> <op> <literal>` and `<literal> <op> <col>`, but
    `simplify_column_predicates` compares whole `Expr`s. `t.s =
    Utf8View("a")` and `Utf8View("a") = t.s` aren't structurally equal, so
    the two equalities read as a contradiction and the conjunction becomes
    `false`.
    
    The INT version survives because the `Canonicalizer` reorders it first.
    It can't do that here: it runs once at `expr_simplifier.rs:203`, ahead
    of the const-evaluation loop, so it sees `CAST(Utf8("a") AS Utf8View)`
    rather than a `Literal` and its `(Literal, Column)` arm doesn't match.
    The cast folds to a literal afterwards. Canonicalization is skipped
    entirely for `Join` (`simplify_exprs.rs:130`), so `simplify_predicates`
    can't assume canonical input either way.
    
    The same gap costs a strict bound. Given `a >= 5` and `5 < a`,
    `find_most_restrictive_predicate` breaks the tie on `op == Gt`, doesn't
    count `Lt` with the literal on the left as strict, keeps `a >= 5`, and
    lets `a = 5` through.
    
    ## What changes are included in this PR?
    
    `simplify_predicates` now normalizes the literal to the right with
    `op.swap()`, at the point where it already distinguishes the two
    orientations. `simplify_column_predicates` can then match on the
    operator alone. No signature changes.
    
    ## Are these changes tested?
    
    Two unit tests in `simplify_predicates.rs` and four cases in
    `simplify_predicates.slt`. All six fail before the fix. With only
    `simplify_predicates.rs` reverted the SLT reports `EmptyRelation:
    rows=0` where `Filter: test_data.str_col = Utf8View("apple")` is
    expected, and the `apple` row goes missing.
    
    `datafusion-optimizer` is green (765 lib, 26 integration, 5 doc) and
    clippy with `-D warnings` is clean. The full `sqllogictests` run passes
    except `window_limits.slt`, which fails identically on an unmodified
    `main`.
    
    `SELECT * FROM t WHERE s = 'a' AND 'b' = s` stays `EmptyRelation:
    rows=0` before and after, and that's pinned in the SLT.
    
    ## Are there any user-facing changes?
    
    Affected queries return the right rows instead of none.
    
    Predicates reaching `simplify_predicates` with the literal on the left
    now come back with it on the right, so a plan can show `a > 5` where it
    used to show `5 < a`. Nothing in the test suite depended on that, but
    the function is public.
    
    Equalities whose literals are equal in value but differ in `ScalarValue`
    representation still collapse to `false`. On `main`, `[a = 5i32, a =
    5i64]` in the same orientation already returns `Boolean(false)`, so that
    predates this change and isn't orientation related.
    
    This PR was written with AI assistance.
---
 .../simplify_expressions/simplify_predicates.rs    | 92 ++++++++++++++--------
 .../test_files/simplify_predicates.slt             | 26 ++++++
 2 files changed, 87 insertions(+), 31 deletions(-)

diff --git a/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs b/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
index 356f2711b7..e7edc34cfe 100644
--- a/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
+++ b/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
@@ -52,27 +52,45 @@ pub fn simplify_predicates(predicates: Vec<Expr>) -> Result<Vec<Expr>> {
     let mut other_predicates = Vec::new();
 
     for pred in predicates {
-        match &pred {
-            Expr::BinaryExpr(BinaryExpr {
-                left,
-                op:
+        match pred {
+            Expr::BinaryExpr(BinaryExpr { left, op, right })
+                if matches!(
+                    op,
                     Operator::Gt
-                    | Operator::GtEq
-                    | Operator::Lt
-                    | Operator::LtEq
-                    | Operator::Eq,
-                right,
-            }) => {
+                        | Operator::GtEq
+                        | Operator::Lt
+                        | Operator::LtEq
+                        | Operator::Eq
+                ) =>
+            {
                 if let (Some(col), Some(_)) =
-                    (extract_column_from_expr(left), right.as_literal())
-                {
-                    column_predicates.entry(col).or_default().push(pred);
-                } else if let (Some(_), Some(col)) =
-                    (left.as_literal(), extract_column_from_expr(right))
+                    (extract_column_from_expr(&left), right.as_literal())
                 {
-                    column_predicates.entry(col).or_default().push(pred);
+                    column_predicates
+                        .entry(col)
+                        .or_default()
+                        .push(Expr::BinaryExpr(BinaryExpr { left, op, right }));
+                } else if let (Some(_), Some(col), Some(swapped_op)) = (
+                    left.as_literal(),
+                    extract_column_from_expr(&right),
+                    op.swap(),
+                ) {
+                    // Put the literal on the right so that predicates differing only in
+                    // operand order compare equal below
+                    column_predicates
+                        .entry(col)
+                        .or_default()
+                        .push(Expr::BinaryExpr(BinaryExpr {
+                            left: right,
+                            op: swapped_op,
+                            right: left,
+                        }));
                 } else {
-                    other_predicates.push(pred);
+                    other_predicates.push(Expr::BinaryExpr(BinaryExpr {
+                        left,
+                        op,
+                        right,
+                    }));
                 }
             }
             _ => other_predicates.push(pred),
@@ -114,20 +132,12 @@ fn simplify_column_predicates(predicates: Vec<Expr>) -> Result<Vec<Expr>> {
 
     for pred in predicates {
         match &pred {
-            Expr::BinaryExpr(BinaryExpr { left: _, op, right }) => {
-                match (op, right.as_literal().is_some()) {
-                    (Operator::Gt, true)
-                    | (Operator::Lt, false)
-                    | (Operator::GtEq, true)
-                    | (Operator::LtEq, false) => greater_predicates.push(pred),
-                    (Operator::Lt, true)
-                    | (Operator::Gt, false)
-                    | (Operator::LtEq, true)
-                    | (Operator::GtEq, false) => less_predicates.push(pred),
-                    (Operator::Eq, _) => eq_predicates.push(pred),
-                    _ => unreachable!("Unexpected operator: {}", op),
-                }
-            }
+            Expr::BinaryExpr(BinaryExpr { op, .. }) => match op {
+                Operator::Gt | Operator::GtEq => greater_predicates.push(pred),
+                Operator::Lt | Operator::LtEq => less_predicates.push(pred),
+                Operator::Eq => eq_predicates.push(pred),
+                _ => unreachable!("Unexpected operator: {}", op),
+            },
             _ => unreachable!("Unexpected predicate {}", pred.to_string()),
         }
     }
@@ -305,6 +315,26 @@ mod tests {
         assert_eq!(extract_column_from_expr(&col_expr), Some(Column::from("a")));
     }
 
+    #[test]
+    fn test_eq_predicates_are_matched_regardless_of_operand_order() {
+        // a = 5 AND 5 = a is a single condition, not a contradiction
+        let predicates = vec![col("a").eq(lit(5i32)), lit(5i32).eq(col("a"))];
+
+        let result = simplify_predicates(predicates).unwrap();
+
+        assert_eq!(result, vec![col("a").eq(lit(5i32))]);
+    }
+
+    #[test]
+    fn test_strict_bound_wins_when_literal_is_on_the_left() {
+        // a >= 5 AND 5 < a is a > 5; keeping a >= 5 would let a = 5 through
+        let predicates = vec![col("a").gt_eq(lit(5i32)), lit(5i32).lt(col("a"))];
+
+        let result = simplify_predicates(predicates).unwrap();
+
+        assert_eq!(result, vec![col("a").gt(lit(5i32))]);
+    }
+
     #[test]
     fn test_simplify_predicates_direct_columns_only() {
         // Test that only predicates on direct columns are simplified together
diff --git a/datafusion/sqllogictest/test_files/simplify_predicates.slt b/datafusion/sqllogictest/test_files/simplify_predicates.slt
index 44fdedc9c8..c36b0c864d 100644
--- a/datafusion/sqllogictest/test_files/simplify_predicates.slt
+++ b/datafusion/sqllogictest/test_files/simplify_predicates.slt
@@ -242,5 +242,31 @@ logical_plan
 02)--TableScan: test_data projection=[int_col, float_col, str_col, date_col, bool_col]
 
 
+# x = 'apple' AND 'apple' = x is one condition, not a contradiction
+query TT
+EXPLAIN SELECT * FROM test_data WHERE str_col = 'apple' AND 'apple' = str_col;
+----
+logical_plan
+01)Filter: test_data.str_col = Utf8View("apple")
+02)--TableScan: test_data projection=[int_col, float_col, str_col, date_col, bool_col]
+
+# x = 'apple' AND 'banana' = x is still impossible
+query TT
+EXPLAIN SELECT * FROM test_data WHERE str_col = 'apple' AND 'banana' = str_col;
+----
+logical_plan EmptyRelation: rows=0
+
 statement ok
 set datafusion.explain.logical_plan_only=false;
+
+statement ok
+CREATE TABLE fruit (name VARCHAR) AS VALUES ('apple'), ('banana');
+
+query T
+SELECT name FROM fruit WHERE name = 'apple' AND 'apple' = name;
+----
+apple
+
+query T
+SELECT name FROM fruit WHERE name = 'apple' AND 'banana' = name;
+----


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org