You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Lantao Jin (JIRA)" <ji...@apache.org> on 2019/04/11 02:48:00 UTC
[jira] [Comment Edited] (SPARK-27337) QueryExecutionListener never
cleans up listeners from the bus after SparkSession is cleared
[ https://issues.apache.org/jira/browse/SPARK-27337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815028#comment-16815028 ]
Lantao Jin edited comment on SPARK-27337 at 4/11/19 2:47 AM:
-------------------------------------------------------------
[~vinooganesh] Could you point me the subject of mail thread in dev list?
was (Author: cltlfcjin):
[~vinooganesh] Could you point me the title of mail thread in dev list?
> QueryExecutionListener never cleans up listeners from the bus after SparkSession is cleared
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-27337
> URL: https://issues.apache.org/jira/browse/SPARK-27337
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Vinoo Ganesh
> Priority: Critical
> Attachments: image001-1.png
>
>
> As a result of [https://github.com/apache/spark/commit/9690eba16efe6d25261934d8b73a221972b684f3], it looks like there is a memory leak (specifically [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L131).]
> Because the Listener Bus on the context still has a reference to the listener (even after the SparkSession is cleared), they are never cleaned up. This means that if you close and remake spark sessions fairly frequently, you're leaking every single time.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org