[jira] [Created] (SPARK-18225) job will miss when driver removed by master in spark streaming

["liujianhui (JIRA)" <ji...@apache.org>]

liujianhui created SPARK-18225:
----------------------------------

             Summary: job will miss when driver removed by master in spark streaming 
                 Key: SPARK-18225
                 URL: https://issues.apache.org/jira/browse/SPARK-18225
             Project: Spark
          Issue Type: Bug
          Components: DStreams, Scheduler
    Affects Versions: 1.6.2, 1.6.1
            Reporter: liujianhui


kill the application on spark ui, the master will send an ApplicationRemoved to driver, driver will abort the all pending job,and then the job finish with exception "Master removed our application:Killed",and then Jobscheduler will remove the job from jobsets, but the jobgenerator still docheckpoint without the job which removed before, and then driver stop；when recover  from the check point file，it miss all jobs which aborted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org