[jira] [Created] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

["gaoyajun02 (Jira)" <ji...@apache.org>]

gaoyajun02 created SPARK-36630:
----------------------------------

             Summary: Add the option to use physical statistics to avoid large tables being broadcast
                 Key: SPARK-36630
                 URL: https://issues.apache.org/jira/browse/SPARK-36630
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: gaoyajun02


Currently AQE is turned on, when queryStage is not materialized, it uses the stats of the logical plan to estimate whether the plan can be converted to BHJ, and in some scenarios the estimated value is several orders of magnitude larger than the actual broadcast data, which can lead to large tables being broadcast



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org