You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2021/07/08 17:21:01 UTC

[jira] [Commented] (BEAM-11084) [Go SDK] Validate Windowing

    [ https://issues.apache.org/jira/browse/BEAM-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17377519#comment-17377519 ] 

Beam JIRA Bot commented on BEAM-11084:
--------------------------------------

This issue is assigned but has not received an update in 30 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.

> [Go SDK] Validate Windowing
> ---------------------------
>
>                 Key: BEAM-11084
>                 URL: https://issues.apache.org/jira/browse/BEAM-11084
>             Project: Beam
>          Issue Type: Task
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Robert Burke
>            Priority: P3
>              Labels: stale-assigned
>
> While Windowing basics are  implemented in the Go SDK, there is precious little validation that it's correct. Much work has been done largely for the benefit of Global Windowing.
> DoFn Authors can specify event times for elements by returning a `beam.EventTime` along with their element, allowing windowing transforms by the runner to take effect. They can observe windows in their DoFns by having a `[]beam.Window` parameter, and extract results.
> Pipeline authors can insert `beam.WindowInto` transforms to have elements be batched into the appropriate window slices.
> But very little of this has been validated end to end.  This umbrella task is to test and validate specific aspects of the window implementation to ensure correctness.
> The tricky part is that windows are selected by runners, meaning many of these must be integration tests.
>  * Windows are correctly observed in DoFns.
>  * Validate windows are aggregated correctly in GBKs and Lifted Combines
>  ** In particular sliding windows where a single element could contribute to multiple aggregates
>  * Validate Side Input behavior WRT windows (implicit observation of windows)
>  ** In particular, Side Inputs are valid per window.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)