[jira] [Created] (SPARK-32165) SessionState leaks SparkListener with multiple SparkSession

["Xianjin YE (Jira)" <ji...@apache.org>]

Xianjin YE created SPARK-32165:
----------------------------------

             Summary: SessionState leaks SparkListener with multiple SparkSession
                 Key: SPARK-32165
                 URL: https://issues.apache.org/jira/browse/SPARK-32165
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Xianjin YE


Copied from [https://github.com/apache/spark/pull/28128#issuecomment-653102770]

 
{code:java}
  test("SPARK-31354: SparkContext only register one SparkSession ApplicationEnd listener") {
    val conf = new SparkConf()
      .setMaster("local")
      .setAppName("test-app-SPARK-31354-1")
    val context = new SparkContext(conf)
    SparkSession
      .builder()
      .sparkContext(context)
      .master("local")
      .getOrCreate()
      .sessionState // this touches the sessionState
    val postFirstCreation = context.listenerBus.listeners.size()
    SparkSession.clearActiveSession()
    SparkSession.clearDefaultSession()

    SparkSession
      .builder()
      .sparkContext(context)
      .master("local")
      .getOrCreate()
      .sessionState // this touches the sessionState
    val postSecondCreation = context.listenerBus.listeners.size()
    SparkSession.clearActiveSession()
    SparkSession.clearDefaultSession()
    assert(postFirstCreation == postSecondCreation)
  }
{code}
The problem can be reproduced by the above code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org