You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/12/15 01:16:00 UTC

[jira] [Assigned] (SPARK-33783) Unload State Store Provider after configured keep alive time

     [ https://issues.apache.org/jira/browse/SPARK-33783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-33783:
------------------------------------

    Assignee: Apache Spark  (was: L. C. Hsieh)

> Unload State Store Provider after configured keep alive time
> ------------------------------------------------------------
>
>                 Key: SPARK-33783
>                 URL: https://issues.apache.org/jira/browse/SPARK-33783
>             Project: Spark
>          Issue Type: New Feature
>          Components: Structured Streaming
>    Affects Versions: 3.2.0
>            Reporter: L. C. Hsieh
>            Assignee: Apache Spark
>            Priority: Major
>
> Currently Spark unloads an inactive state store provider in an maintenance task which is run periodically. So it is said one state store provider might be unloaded immediately after it becomes inactive because the maintenance task is asynchronous. It is inefficient as the state store provider could be reused in next batches and we should be able to have more control of unloading behavior. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org