[jira] [Resolved] (FLINK-2393) Add a "stateless at-least-once" mode for streaming

["Stephan Ewen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephan Ewen resolved FLINK-2393.
---------------------------------
       Resolution: Implemented
         Assignee: Stephan Ewen
    Fix Version/s: 0.10

All subtasks done

> Add a "stateless at-least-once" mode for streaming
> --------------------------------------------------
>
>                 Key: FLINK-2393
>                 URL: https://issues.apache.org/jira/browse/FLINK-2393
>             Project: Flink
>          Issue Type: New Feature
>          Components: Streaming
>    Affects Versions: 0.10
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>             Fix For: 0.10
>
>
> Currently, the checkpointing mechanism provides "exactly once" guarantees. Part of that is the step that temporarily "aligns" the data streams. This step increases the tuple latency temporarily.
> By offering a version that does not provide "exactly-once", but only "at-least-once", we can avoid the latency increase. For super-low-latency applications, that tolerate duplicates, this may be an interesting option.
> To realize that, we would use a slightly modified version of the checkpointing algorithm. Effectively, the streams would not be aligned, but tasks would only count the received barriers and emit their own barrier as soon as the saw a barrier from all inputs.
> My feeling is that it makes not sense to implement state backups, when being concerned with this super low latency. The mode would hence be a purely stateless at-least-once mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)