You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiao Li (Jira)" <ji...@apache.org> on 2020/01/03 01:43:00 UTC
[jira] [Resolved] (SPARK-29568) Add flag to stop existing stream
when new copy starts
[ https://issues.apache.org/jira/browse/SPARK-29568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li resolved SPARK-29568.
-----------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
> Add flag to stop existing stream when new copy starts
> -----------------------------------------------------
>
> Key: SPARK-29568
> URL: https://issues.apache.org/jira/browse/SPARK-29568
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Burak Yavuz
> Assignee: Burak Yavuz
> Priority: Major
> Fix For: 3.0.0
>
>
> In multi-tenant environments where you have multiple SparkSessions, you can accidentally start multiple copies of the same stream (i.e. streams using the same checkpoint location). This will cause all new instantiations of the new stream to fail. However, sometimes you may want to turn off the old stream, as the old stream may have turned into a zombie (you no longer have access to the query handle or SparkSession).
> It would be nice to have a SQL flag that allows the stopping of the old stream for such zombie cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org