[Spark Streaming] "Could not compute split, block input-0-1452563923800 not found” when trying to recover from checkpoint data

[Collin Shi <sh...@aliyun.com>]

Hi

I was doing a simple updateByKey transformation and print on the data received from socket, and spark version is 1.4.0. The first submit went all right, but after I kill (CTRL + C) the job and submit again. Apparently spark was trying to recover from the checkpoint data , but then the exception occured. So I'm wondering why this happen and how to fix it. 

Answers I found on google is "Set the storage level of input DStream to MEMORY_AND_DISK" or "set spark.streaming.stopGracefullyOnShutdown to true", but neither works. 

I was using nc -lk 9999 as data server, notice that this problem will always happen in which case I send a few message and shutdown the job immediately. 

The code and driver logs is in the attachment.  



Thanks

Collin