You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Juliusz Sompolski (Jira)" <ji...@apache.org> on 2023/08/08 23:51:00 UTC
[jira] [Created] (SPARK-44730) Spark Connect: Cleaner thread not stopped when SparkSession stops
Juliusz Sompolski created SPARK-44730:
-----------------------------------------
Summary: Spark Connect: Cleaner thread not stopped when SparkSession stops
Key: SPARK-44730
URL: https://issues.apache.org/jira/browse/SPARK-44730
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 3.5.0, 4.0.0
Reporter: Juliusz Sompolski
Spark Connect scala client SparkSession has a cleaner, which starts a daemon thread to clean up Closeable objects after GC. This daemon thread is never stopped, and every SparkSession creates a new one.
Cleaner implements a stop() function, but no-one ever calls it. Possibly because even after SparkSession.stop(), the cleaner may still be needed when remaining references are GCed... For this reason it seems that the Cleaner should rather be a global singleton than within a session.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org