You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by David Zhou <zh...@gmail.com> on 2019/09/06 21:07:19 UTC

Question on streaming job wait and re-run

Hi,
My streaming job consumes data from kafka and writes them into Cassandra.

Current status:
Cassandra is not stable. Streaming job crashed when it can't write data
into Cassandra. Streaming job has check point. Usually, the Cassandra
cluster will come back in 4 hours. Finally, I start the streaming job
again, and it will start from check point and run.

Expectation:
Streaming job can enter a status like thread wait, not crash, and it will
be notified automatically when Cassandra comes back.

Does Streaming job support this feature? I searched by Google, can't find
related solutions.

Thanks.
David