You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/03/16 22:55:04 UTC

[jira] [Updated] (SPARK-29972) Manage/trace all spark sessions

     [ https://issues.apache.org/jira/browse/SPARK-29972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun updated SPARK-29972:
----------------------------------
    Affects Version/s:     (was: 3.0.0)
                       3.1.0

> Manage/trace all spark sessions
> -------------------------------
>
>                 Key: SPARK-29972
>                 URL: https://issues.apache.org/jira/browse/SPARK-29972
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Lantao Jin
>            Priority: Major
>
> Current there is no place to trace all live spark sessions in a non-thrift-server application. For example spark session can be stored in thread local variable and be garbage collection when user's thread exit. So it may has some problems like memory leak in an application. Also, seems we don't have such an action called "close session" in Spark SQL. SparkSession.close closes the SparkContext and thus closes all sessions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org