[jira] [Created] (SPARK-43157) TreeNode tags can become corrupted and hang driver when the dataset is cached

["Rob Reeves (Jira)" <ji...@apache.org>]

Rob Reeves created SPARK-43157:
----------------------------------

             Summary: TreeNode tags can become corrupted and hang driver when the dataset is cached
                 Key: SPARK-43157
                 URL: https://issues.apache.org/jira/browse/SPARK-43157
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.0, 3.5.0
            Reporter: Rob Reeves


If a cached dataset is used by multiple other datasets materialized in separate threads it can corrupt the TreeNode.tags map in any of the cached plan nodes. This will hang the driver forever. This happens because TreeNode.tags is not thread-safe. How this happens:
 # Multiple datasets are materialized at the same time in different threads that reference the same cached dataset
 # AdaptiveSparkPlanExec.onUpdatePlan will call ExplainMode.fromString
 # ExplainUtils uses the TreeNode.tags map to store the operator Id for every node in the plan. This is usually okay because the plan is cloned. When there is an InMemoryScanExec the InMemoryRelation.cachedPlan is not cloned so multiple threads can set the operator Id.

Making the TreeNode.tags field thread-safe does not solve this problem because there is still a correctness issue. The threads may be overwriting each other's operator Ids, which could be different.

Example stack trace of the infinite loop:
{code:java}
scala.collection.mutable.HashTable.resize(HashTable.scala:265)
scala.collection.mutable.HashTable.addEntry0(HashTable.scala:158)
scala.collection.mutable.HashTable.findOrAddEntry(HashTable.scala:170)
scala.collection.mutable.HashTable.findOrAddEntry$(HashTable.scala:167)
scala.collection.mutable.HashMap.findOrAddEntry(HashMap.scala:44)
scala.collection.mutable.HashMap.put(HashMap.scala:126)
scala.collection.mutable.HashMap.update(HashMap.scala:131)
org.apache.spark.sql.catalyst.trees.TreeNode.setTagValue(TreeNode.scala:108)
org.apache.spark.sql.execution.ExplainUtils$.setOpId$1(ExplainUtils.scala:134)
…org.apache.spark.sql.execution.QueryExecution.explainString(QueryExecution.scala:175)
org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.onUpdatePlan(AdaptiveSparkPlanExec.scala:662){code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org