[jira] [Created] (SPARK-29552) Fix the flaky test failed in AdaptiveQueryExecSuite # multiple joins

["Ke Jia (Jira)" <ji...@apache.org>]

Ke Jia created SPARK-29552:
------------------------------

             Summary: Fix the flaky test failed in AdaptiveQueryExecSuite # multiple joins
                 Key: SPARK-29552
                 URL: https://issues.apache.org/jira/browse/SPARK-29552
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Ke Jia


AQE will optimize the logical plan once there is query stage finished. So for inner join, when two join side is all small to be the build side. The planner of converting logical plan to physical plan will select the build side as BuildRight if right side finished firstly or BuildLeft if left side finished firstly. In some case, when BuildRight or BuildLeft may introduce additioanl exchange to the parent node. The revert approach in OptimizeLocalShuffleReader rule may be too conservative, which revert all the local shuffle reader when introduce additional exchange not  revert the local shuffle reader introduced shuffle.  It may be expense to only revert the local shuffle reader introduced shuffle. The workaround is to apply the OptimizeLocalShuffleReader rule again when creating new query stage to further optimize the sub tree shuffle reader to local shuffle reader.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org