[jira] [Commented] (BEAM-1589) Add OnWindowExpiration method to Stateful DoFn

["Reuven Lax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011071#comment-17011071 ] 

Reuven Lax commented on BEAM-1589:
----------------------------------

output timestamp support is now in (Flink support pending).

I think we should just use the same timer that's used for garbage collection to fire OnWindowExpiration callbacks. In order to make this work, we need to modify this timer to have an output timestamp inside the window (maybe the end of the window?).

> Add OnWindowExpiration method to Stateful DoFn
> ----------------------------------------------
>
>                 Key: BEAM-1589
>                 URL: https://issues.apache.org/jira/browse/BEAM-1589
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-core, sdk-java-core
>            Reporter: Jingsong Lee
>            Priority: Major
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> See BEAM-1517
> This allows the user to do some work before the state's garbage collection.
> It seems kind of annoying, but on the other hand forgetting to set a final timer to flush state is probably data loss most of the time.
> FlinkRunner does this work very simply, but other runners, such as DirectRunner, need to traverse all the states to do this, and maybe it's a little hard.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)