You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/09/21 19:49:04 UTC
[jira] [Commented] (SPARK-10732) Starting spark streaming from a
specific point in time.
[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901065#comment-14901065 ]
Sean Owen commented on SPARK-10732:
-----------------------------------
You can set the initial offsets to begin from in, for example, a Kafka input stream. This is because Kafka has a notion of 'time' (offsets). Spark Streaming does not in general aside from wall-clock time. It's not clear how this would work from, say, input from a network or a directory? but it's already accomplishable where it's possible.
> Starting spark streaming from a specific point in time.
> -------------------------------------------------------
>
> Key: SPARK-10732
> URL: https://issues.apache.org/jira/browse/SPARK-10732
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.5.0
> Reporter: Bijay Singh Bisht
>
> Currently, spark streaming either starts from current time or from the latest checkpoint. It would be extremely useful to start from any arbitrary point. This would be useful in replay scenarios or in running regression tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org