You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/03 12:25:02 UTC
(datafusion) branch main updated: fix: literal-on-left equality no longer collapses the filter to false (#24763)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new 20d1c56761 fix: literal-on-left equality no longer collapses the filter to false (#24763)
20d1c56761 is described below
commit 20d1c5676118e0bca955ccf92c0d2cf665384d0d
Author: Bharadwaj Pendyala <bh...@gmail.com>
AuthorDate: Thu Sep 3 12:15:10 2026 +0000
fix: literal-on-left equality no longer collapses the filter to false (#24763)
## Which issue does this PR close?
No separate issue. I found this while reading `simplify_predicates`.
## Rationale for this change
`WHERE s = 'a' AND 'a' = s` returns no rows, where one row is expected:
```sql
> CREATE TABLE t(s VARCHAR) AS VALUES ('a'), ('b');
> SELECT * FROM t WHERE s = 'a' AND 'a' = s;
0 row(s) fetched.
```
Either half on its own returns `a`, and the same query against an INT
column returns the row.
On `main` (4d3e79e), `EXPLAIN VERBOSE` shows the filter turning into a
constant between two rules:
```
logical_plan after simplify_expressions Filter: t.s = Utf8View("a") AND Utf8View("a") = t.s
logical_plan after push_down_filter Filter: Boolean(false)
logical_plan after eliminate_filter EmptyRelation: rows=0
```
`PushDownFilter` splits the conjuncts and calls `simplify_predicates`.
It accepts both `<col> <op> <literal>` and `<literal> <op> <col>`, but
`simplify_column_predicates` compares whole `Expr`s. `t.s =
Utf8View("a")` and `Utf8View("a") = t.s` aren't structurally equal, so
the two equalities read as a contradiction and the conjunction becomes
`false`.
The INT version survives because the `Canonicalizer` reorders it first.
It can't do that here: it runs once at `expr_simplifier.rs:203`, ahead
of the const-evaluation loop, so it sees `CAST(Utf8("a") AS Utf8View)`
rather than a `Literal` and its `(Literal, Column)` arm doesn't match.
The cast folds to a literal afterwards. Canonicalization is skipped
entirely for `Join` (`simplify_exprs.rs:130`), so `simplify_predicates`
can't assume canonical input either way.
The same gap costs a strict bound. Given `a >= 5` and `5 < a`,
`find_most_restrictive_predicate` breaks the tie on `op == Gt`, doesn't
count `Lt` with the literal on the left as strict, keeps `a >= 5`, and
lets `a = 5` through.
## What changes are included in this PR?
`simplify_predicates` now normalizes the literal to the right with
`op.swap()`, at the point where it already distinguishes the two
orientations. `simplify_column_predicates` can then match on the
operator alone. No signature changes.
## Are these changes tested?
Two unit tests in `simplify_predicates.rs` and four cases in
`simplify_predicates.slt`. All six fail before the fix. With only
`simplify_predicates.rs` reverted the SLT reports `EmptyRelation:
rows=0` where `Filter: test_data.str_col = Utf8View("apple")` is
expected, and the `apple` row goes missing.
`datafusion-optimizer` is green (765 lib, 26 integration, 5 doc) and
clippy with `-D warnings` is clean. The full `sqllogictests` run passes
except `window_limits.slt`, which fails identically on an unmodified
`main`.
`SELECT * FROM t WHERE s = 'a' AND 'b' = s` stays `EmptyRelation:
rows=0` before and after, and that's pinned in the SLT.
## Are there any user-facing changes?
Affected queries return the right rows instead of none.
Predicates reaching `simplify_predicates` with the literal on the left
now come back with it on the right, so a plan can show `a > 5` where it
used to show `5 < a`. Nothing in the test suite depended on that, but
the function is public.
Equalities whose literals are equal in value but differ in `ScalarValue`
representation still collapse to `false`. On `main`, `[a = 5i32, a =
5i64]` in the same orientation already returns `Boolean(false)`, so that
predates this change and isn't orientation related.
This PR was written with AI assistance.
---
.../simplify_expressions/simplify_predicates.rs | 92 ++++++++++++++--------
.../test_files/simplify_predicates.slt | 26 ++++++
2 files changed, 87 insertions(+), 31 deletions(-)
diff --git a/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs b/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
index 356f2711b7..e7edc34cfe 100644
--- a/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
+++ b/datafusion/optimizer/src/simplify_expressions/simplify_predicates.rs
@@ -52,27 +52,45 @@ pub fn simplify_predicates(predicates: Vec<Expr>) -> Result<Vec<Expr>> {
let mut other_predicates = Vec::new();
for pred in predicates {
- match &pred {
- Expr::BinaryExpr(BinaryExpr {
- left,
- op:
+ match pred {
+ Expr::BinaryExpr(BinaryExpr { left, op, right })
+ if matches!(
+ op,
Operator::Gt
- | Operator::GtEq
- | Operator::Lt
- | Operator::LtEq
- | Operator::Eq,
- right,
- }) => {
+ | Operator::GtEq
+ | Operator::Lt
+ | Operator::LtEq
+ | Operator::Eq
+ ) =>
+ {
if let (Some(col), Some(_)) =
- (extract_column_from_expr(left), right.as_literal())
- {
- column_predicates.entry(col).or_default().push(pred);
- } else if let (Some(_), Some(col)) =
- (left.as_literal(), extract_column_from_expr(right))
+ (extract_column_from_expr(&left), right.as_literal())
{
- column_predicates.entry(col).or_default().push(pred);
+ column_predicates
+ .entry(col)
+ .or_default()
+ .push(Expr::BinaryExpr(BinaryExpr { left, op, right }));
+ } else if let (Some(_), Some(col), Some(swapped_op)) = (
+ left.as_literal(),
+ extract_column_from_expr(&right),
+ op.swap(),
+ ) {
+ // Put the literal on the right so that predicates differing only in
+ // operand order compare equal below
+ column_predicates
+ .entry(col)
+ .or_default()
+ .push(Expr::BinaryExpr(BinaryExpr {
+ left: right,
+ op: swapped_op,
+ right: left,
+ }));
} else {
- other_predicates.push(pred);
+ other_predicates.push(Expr::BinaryExpr(BinaryExpr {
+ left,
+ op,
+ right,
+ }));
}
}
_ => other_predicates.push(pred),
@@ -114,20 +132,12 @@ fn simplify_column_predicates(predicates: Vec<Expr>) -> Result<Vec<Expr>> {
for pred in predicates {
match &pred {
- Expr::BinaryExpr(BinaryExpr { left: _, op, right }) => {
- match (op, right.as_literal().is_some()) {
- (Operator::Gt, true)
- | (Operator::Lt, false)
- | (Operator::GtEq, true)
- | (Operator::LtEq, false) => greater_predicates.push(pred),
- (Operator::Lt, true)
- | (Operator::Gt, false)
- | (Operator::LtEq, true)
- | (Operator::GtEq, false) => less_predicates.push(pred),
- (Operator::Eq, _) => eq_predicates.push(pred),
- _ => unreachable!("Unexpected operator: {}", op),
- }
- }
+ Expr::BinaryExpr(BinaryExpr { op, .. }) => match op {
+ Operator::Gt | Operator::GtEq => greater_predicates.push(pred),
+ Operator::Lt | Operator::LtEq => less_predicates.push(pred),
+ Operator::Eq => eq_predicates.push(pred),
+ _ => unreachable!("Unexpected operator: {}", op),
+ },
_ => unreachable!("Unexpected predicate {}", pred.to_string()),
}
}
@@ -305,6 +315,26 @@ mod tests {
assert_eq!(extract_column_from_expr(&col_expr), Some(Column::from("a")));
}
+ #[test]
+ fn test_eq_predicates_are_matched_regardless_of_operand_order() {
+ // a = 5 AND 5 = a is a single condition, not a contradiction
+ let predicates = vec![col("a").eq(lit(5i32)), lit(5i32).eq(col("a"))];
+
+ let result = simplify_predicates(predicates).unwrap();
+
+ assert_eq!(result, vec![col("a").eq(lit(5i32))]);
+ }
+
+ #[test]
+ fn test_strict_bound_wins_when_literal_is_on_the_left() {
+ // a >= 5 AND 5 < a is a > 5; keeping a >= 5 would let a = 5 through
+ let predicates = vec![col("a").gt_eq(lit(5i32)), lit(5i32).lt(col("a"))];
+
+ let result = simplify_predicates(predicates).unwrap();
+
+ assert_eq!(result, vec![col("a").gt(lit(5i32))]);
+ }
+
#[test]
fn test_simplify_predicates_direct_columns_only() {
// Test that only predicates on direct columns are simplified together
diff --git a/datafusion/sqllogictest/test_files/simplify_predicates.slt b/datafusion/sqllogictest/test_files/simplify_predicates.slt
index 44fdedc9c8..c36b0c864d 100644
--- a/datafusion/sqllogictest/test_files/simplify_predicates.slt
+++ b/datafusion/sqllogictest/test_files/simplify_predicates.slt
@@ -242,5 +242,31 @@ logical_plan
02)--TableScan: test_data projection=[int_col, float_col, str_col, date_col, bool_col]
+# x = 'apple' AND 'apple' = x is one condition, not a contradiction
+query TT
+EXPLAIN SELECT * FROM test_data WHERE str_col = 'apple' AND 'apple' = str_col;
+----
+logical_plan
+01)Filter: test_data.str_col = Utf8View("apple")
+02)--TableScan: test_data projection=[int_col, float_col, str_col, date_col, bool_col]
+
+# x = 'apple' AND 'banana' = x is still impossible
+query TT
+EXPLAIN SELECT * FROM test_data WHERE str_col = 'apple' AND 'banana' = str_col;
+----
+logical_plan EmptyRelation: rows=0
+
statement ok
set datafusion.explain.logical_plan_only=false;
+
+statement ok
+CREATE TABLE fruit (name VARCHAR) AS VALUES ('apple'), ('banana');
+
+query T
+SELECT name FROM fruit WHERE name = 'apple' AND 'apple' = name;
+----
+apple
+
+query T
+SELECT name FROM fruit WHERE name = 'apple' AND 'banana' = name;
+----
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org