You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/02/24 09:50:00 UTC

[jira] [Assigned] (SPARK-30927) StreamingQueryManager should avoid keeping reference to terminated StreamingQuery

     [ https://issues.apache.org/jira/browse/SPARK-30927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon reassigned SPARK-30927:
------------------------------------

    Assignee: Shixiong Zhu

> StreamingQueryManager should avoid keeping reference to terminated StreamingQuery
> ---------------------------------------------------------------------------------
>
>                 Key: SPARK-30927
>                 URL: https://issues.apache.org/jira/browse/SPARK-30927
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 3.0.0
>            Reporter: Shixiong Zhu
>            Assignee: Shixiong Zhu
>            Priority: Major
>
> Right now StreamingQueryManager will keep the last terminated query until "resetTerminated" is called. When the last terminated query has lots of states (a large sql plan, cached RDDs, etc.), it will waste these memory unnecessarily.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org