[jira] [Created] (SPARK-57041) Fix deadlock between waitForSubqueries and lazy val initialization in subquery thread

["Eric Yang (Jira)" <ji...@apache.org>]

Eric Yang created SPARK-57041:
---------------------------------

             Summary: Fix deadlock between waitForSubqueries and lazy val initialization in subquery thread
                 Key: SPARK-57041
                 URL: https://issues.apache.org/jira/browse/SPARK-57041
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 4.1.1
            Reporter: Eric Yang


A deadlock can occur between a broadcast-exchange thread and a subquery thread:
- Thread A (broadcast-exchange): enters SparkPlan.waitForSubqueries(), which acquires this.synchronized, then blocks waiting for a subquery future.
- Thread B (subquery-0): while executing the subquery under AQE, onUpdatePlan calls SparkPlanInfo.fromSparkPlan → FileSourceScanLike.metadata (a Scala lazy val). Scala compiles lazy val to this.synchronized, so Thread B blocks trying to acquire the same lock Thread A holds.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org