[jira] [Created] (SPARK-40999) Hints on subqueries are not properly propagated

["Fredrik Klauß (Jira)" <ji...@apache.org>]

Fredrik Klauß created SPARK-40999:
-------------------------------------

             Summary: Hints on subqueries are not properly propagated
                 Key: SPARK-40999
                 URL: https://issues.apache.org/jira/browse/SPARK-40999
             Project: Spark
          Issue Type: Bug
          Components: Optimizer, Spark Core
    Affects Versions: 3.3.1, 3.2.2, 3.3.0, 3.2.1, 3.1.3, 3.2.0, 3.1.2, 3.1.1, 3.1.0, 3.0.3, 3.0.2, 3.0.1, 3.0.0
            Reporter: Fredrik Klauß
             Fix For: 3.4.0


Currently, if a user tries to specify a query like the following, the hints on the subquery will be lost. 
{code:java}
SELECT * FROM target t WHERE EXISTS
(SELECT /*+ BROADCAST */ * FROM source s WHERE s.key = t.key){code}
This happens as hints are removed from the plan and pulled into joins in the beginning of the optimization stage, but subqueries are only turned into joins during optimization. As we remove any hints that are not below a join, we end up removing hints that are below a subquery. 

 

To resolve this, we add a hint field to SubqueryExpression that any hints inside a subquery's plan can be pulled into during EliminateResolvedHint, and then pass this hint on when the subquery is turned into a join.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org