You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/09/22 08:08:20 UTC
[jira] [Commented] (SPARK-17632) make console sink and other sinks
work with 'recoverFromCheckpointLocation' option enabled
[ https://issues.apache.org/jira/browse/SPARK-17632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512552#comment-15512552 ]
Apache Spark commented on SPARK-17632:
--------------------------------------
User 'chuanlei' has created a pull request for this issue:
https://github.com/apache/spark/pull/15195
> make console sink and other sinks work with 'recoverFromCheckpointLocation' option enabled
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-17632
> URL: https://issues.apache.org/jira/browse/SPARK-17632
> Project: Spark
> Issue Type: Bug
> Components: SQL, Streaming
> Affects Versions: 2.0.0
> Reporter: Chuanlei Ni
> Priority: Trivial
>
> val (useTempCheckpointLocation, recoverFromCheckpointLocation) =
> if (source == "console") {
> (true, false)
> } else {
> (false, true)
> }
> I think 'source' sink should work with 'recoverFromCheckpoint' option enabled. If user specified checkpointLocation, we may lost the processed data, however it is acceptable since this sink used for test mainly.
> And other sink can be work with 'useTempCheckpointLocation' enabled, which means replaying all the data in kafka. If user mean it, it is ok.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org