Streaming : WAL ignored

[Walid Lezzar <wa...@gmail.com>]

Hi,

I have a spark streaming application running on yarn that consumes from a jms source. I have the checkpointing and WAL enabled to ensure zero data loss. 
However, When I suddenly kill my application and restarts it, sometimes it recovers the data from the WAL but sometimes it doesn’t !! In all the cases, I can see the WAL written correctly on HDFS. 

Can someone explains me why my WAL is sometimes ignored on restart ? What are the conditions for spark to decide to recover or not from the WAL ?

Thanks,
Walid.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org