You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tornike Gurgenidze (Jira)" <ji...@apache.org> on 2025/09/17 08:05:00 UTC

[jira] [Created] (SPARK-53618) Turn a FULL JOIN with false condition to a UNION ALL

Tornike Gurgenidze created SPARK-53618:
------------------------------------------

             Summary: Turn a FULL JOIN with false condition to a UNION ALL
                 Key: SPARK-53618
                 URL: https://issues.apache.org/jira/browse/SPARK-53618
             Project: Spark
          Issue Type: Improvement
          Components: Optimizer
    Affects Versions: 4.0.1
            Reporter: Tornike Gurgenidze


Catalyst should translate full joins that have static always false conditions (like 1 = 0) to a simple union of tables padded with nulls where necessary. Example join looks like this:
{code:sql}
SELECT *
FROM A
FULL JOIN B ON 1 = 0
{code}
Instead of turning the query to it's simpler format, right now spark executes a BroadcastNestedLoop join as 1 = 0 is not an equi join condition.

While this is often trivial enough that the users can do it themselves w/o involving an optimizer, there's a corner case of 1 = 0 condition being used in a MERGE statement that can't really be optimized by a user by rewriting a query.

This issue is a follow-up on an another [PR|https://github.com/apache/spark/pull/52185] of mine that paves the way for a query like this to be used for a replaceWhere-like operation from delta lake which is not available in iceberg:
{code:sql}
MERGE INTO target_table as t
USING source as s
ON 1 = 0
WHEN NOT MATCHED THEN INSERT *
WHEN NOT MATCHED BY SOURCE AND [replaceWhereCondition] THEN DELETE
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org