You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by David Zhou <zh...@gmail.com> on 2019/09/06 21:07:19 UTC
Question on streaming job wait and re-run
Hi,
My streaming job consumes data from kafka and writes them into Cassandra.
Current status:
Cassandra is not stable. Streaming job crashed when it can't write data
into Cassandra. Streaming job has check point. Usually, the Cassandra
cluster will come back in 4 hours. Finally, I start the streaming job
again, and it will start from check point and run.
Expectation:
Streaming job can enter a status like thread wait, not crash, and it will
be notified automatically when Cassandra comes back.
Does Streaming job support this feature? I searched by Google, can't find
related solutions.
Thanks.
David