You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:23:35 UTC

[jira] [Updated] (SPARK-13222) checkpoint latest stateful RDD on graceful shutdown

     [ https://issues.apache.org/jira/browse/SPARK-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-13222:
---------------------------------
    Labels: bulk-closed  (was: )

> checkpoint latest stateful RDD on graceful shutdown
> ---------------------------------------------------
>
>                 Key: SPARK-13222
>                 URL: https://issues.apache.org/jira/browse/SPARK-13222
>             Project: Spark
>          Issue Type: Bug
>          Components: DStreams
>            Reporter: Mao, Wei
>            Priority: Major
>              Labels: bulk-closed
>
> The CheckpointInterval of DStream could be customized to multiple of BatchInteveal. So when user call shutdown it could be in the middle of two RDD checkpoint. In case the input source is not repeatable and user don't want to enable WAL because of its extra cost, the application could be unrecoverable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org