[jira] [Commented] (FLINK-3178) Make Closing Behavior of Window Operators Configurable

["Aljoscha Krettek (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105118#comment-15105118 ] 

Aljoscha Krettek commented on FLINK-3178:
-----------------------------------------

I can't come up with cases where emitting the unfinished windows would actually be the desirable behavior. As I see it there are two cases that involve closing the window operator:
1. The job crashes, in that case the job will hopefully restart and I don't want the unfinished windows emitted before
2. The job is gracefully stopped in order to be restarted later (possibly with save points), in that case I also don't want the unfinished windows in my output

So I would suggest simply changing the behavior to not emit windows on closing. Most people I talked to that are developing real-world jobs would actually expect non-emission to be the default behavior.

What do you think?

> Make Closing Behavior of Window Operators Configurable
> ------------------------------------------------------
>
>                 Key: FLINK-3178
>                 URL: https://issues.apache.org/jira/browse/FLINK-3178
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 1.0.0
>            Reporter: Aljoscha Krettek
>            Priority: Blocker
>
> Right now, all window operators emit (incomplete) in-flight windows when they are closing down. This happens for two reasons:
> 1. sources emit a final Long.MAX_VALUE watermark to flush out windows when they close.
> 2. window operators emit in-flight windows in their close() method
> For some users this might not be the desired behavior so we should either change it or make it configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)