[jira] [Commented] (SAMZA-1494) Flush operator state on end of stream

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261025#comment-16261025 ] 

ASF GitHub Bot commented on SAMZA-1494:
---------------------------------------

GitHub user vjagadish1989 opened a pull request:

    https://github.com/apache/samza/pull/366

    SAMZA-1494: Flush operator state at end of stream

    - Propagate operator messages at endOfStream to all down-stream operators.
    - Emit all pending windows when endOfStream is reached.
    - Flush all state on endOfStream irrespective of auto-commit behavior.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/vjagadish1989/samza eos-flush

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/366.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #366
    
----
commit db67ce84197d589c72c11a8eaa9af9cda1d5f05c
Author: Jagadish <jv...@linkedin.com>
Date:   2017-11-21T16:38:02Z

    SAMZA-1494: Flush operator state at end of stream

----


> Flush operator state on end of stream
> -------------------------------------
>
>                 Key: SAMZA-1494
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1494
>             Project: Samza
>          Issue Type: Sub-task
>            Reporter: Prateek Maheshwari
>            Assignee: Jagadish
>             Fix For: 0.14.0
>
>
> Stateful operators should flush all buffered state when end of stream is reached. Otherwise the container may shut down with open windows and lead to message loss.
> As of 0.14, only window operator needs to do this. When we add outer/left/right joins they'll need to do it as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)