You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "XiDuo You (Jira)" <ji...@apache.org> on 2023/01/17 10:10:00 UTC

[jira] [Created] (SPARK-42101) Wrap InMemoryTableScanExec + AQE with QueryStage

XiDuo You created SPARK-42101:
---------------------------------

             Summary: Wrap InMemoryTableScanExec + AQE with QueryStage
                 Key: SPARK-42101
                 URL: https://issues.apache.org/jira/browse/SPARK-42101
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.4.0
            Reporter: XiDuo You


The first access to the cached plan which is enable AQE is tricky. Currently, we can not preverse it's output partitioning and ordering.

The whole query plan also missed lots of optimization in AQE framework. Wrap 
InMemoryTableScanExec + AQE to query stage can resolve all these issues.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org