You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/02/03 05:02:00 UTC

[jira] [Updated] (SPARK-55326) Add an optional connect session release in pyspark

     [ https://issues.apache.org/jira/browse/SPARK-55326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-55326:
-----------------------------------
    Labels: pull-request-available  (was: )

> Add an optional connect session release in pyspark
> --------------------------------------------------
>
>                 Key: SPARK-55326
>                 URL: https://issues.apache.org/jira/browse/SPARK-55326
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect, PySpark
>    Affects Versions: 4.2
>            Reporter: Bobby Wang
>            Priority: Major
>              Labels: pull-request-available
>
> In Spark Connect, if a user’s code does not include {{{}spark.stop(){}}}—an omission that is quite common—the session will persist on the Spark Connect Server until the idle timeout elapses, which can lead to unnecessary resource waste. For this reason, I propose adding an environment variable in PySpark to implement session release logic within the PySpark process’s shutdown hook. When this environment variable is enabled, PySpark will attempt to release the session as best it can before the Python process exits.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org