Spark Streaming Listener to Kill Stages?

[suchenzang <su...@gmail.com>]

Hello,

I was wondering if there's a way to kill stages in spark streaming via a
StreamingListener. Sometimes stages will hang, and simply killing it via the
UI:

<http://apache-spark-user-list.1001560.n3.nabble.com/file/n24478/Screen_Shot_2015-08-27_at_11.png> 

is enough to let the streaming job continue. Ideally we wouldn't have
hanging issues, but we would rather troubleshoot later via logs instead of
terminating the entire streaming job.

Is there any way to access something that is equivalent to
SparkContext.cancelStage, or is this generally not advised?

Thanks!



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Listener-to-Kill-Stages-tp24478.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org