You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2014/12/27 08:31:21 UTC
[jira] [Updated] (SPARK-3505) Augmenting SparkStreaming
updateStateByKey API with timestamp
[ https://issues.apache.org/jira/browse/SPARK-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell updated SPARK-3505:
-----------------------------------
Fix Version/s: (was: 1.2.0)
1.3.0
> Augmenting SparkStreaming updateStateByKey API with timestamp
> -------------------------------------------------------------
>
> Key: SPARK-3505
> URL: https://issues.apache.org/jira/browse/SPARK-3505
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.0.0
> Reporter: Xi Liu
> Priority: Minor
> Fix For: 1.3.0
>
>
> The current updateStateByKey API in Spark Streaming does not expose timestamp to the application.
> In our use case, the application need to know the batch timestamp to decide whether to keep the state or not. And we do not want to use real system time because we want to decouple the two (because the same code base is used for streaming and offline processing).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org