[jira] [Assigned] (BEAM-654) When and how can merging windows "shrink" or "grow"?

["Kenneth Knowles (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles reassigned BEAM-654:
------------------------------------

    Assignee:     (was: Kenneth Knowles)

> When and how can merging windows "shrink" or "grow"?
> ----------------------------------------------------
>
>                 Key: BEAM-654
>                 URL: https://issues.apache.org/jira/browse/BEAM-654
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Kenneth Knowles
>
> The primary example of a merging window today is {{Sessions}} by gap duration, in which the merged window is the interval enclosure / span of the windows being merged.
> However, another reasonable abstract use case is a session identified by id with an explicit end event. We might consider that the session ends with no gap duration after the end event. In this case, the merged window may be smaller than the enclosure of the sub-windows. Sometimes this has been referred to as "merging shrinks the window".
> Perhaps the only requirement is that the merged window contains the timestamps of the data therein, but we should document this clearly. The current spec is ["Does whatever merging is necessary"|https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/WindowFn.java#L106]
> There are repercussions for triggers, some documented in the [trigger design doc|https://s.apache.org/beam-triggers]: With nonzero allowed lateness, {{Sessions}} by gap duration can switch a trigger from ON_TIME or LATE behavior back to speculative behavior, or cause another ON_TIME firing. Conversely, sessions with abrupt termination/shrinking may have that behavior _as well as_ an ON_TIME and subsequent LATE firings due only to the merging (this already works properly).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)