spark streaming client program needs to be restarted after few hours of idle time. how can I fix it?

[kant kodali <ka...@gmail.com>]

Hi Guys,

My Spark Streaming Client program works fine as the long as the receiver
receives the data  but say my receiver has no more data to receive for few
hours like (4-5 hours) and then its starts receiving the data again at that
point spark client program doesn't seem to process any data. It needs to be
restarted in which case everything seem to work fine again. I am using
spark standalone mode and my client program has following lines in the end
for it to run forever. any ideas what can go wrong? I have some potential
suspects and I will share them after a bit of experimentation from my end.

Thanks!

ssc.start();
ssc.awaitTermination();