You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Jack McCluskey (Jira)" <ji...@apache.org> on 2021/10/20 19:17:00 UTC
[jira] [Updated] (BEAM-11087) [Go SDK] Validate Side Input behavior
WRT windows
[ https://issues.apache.org/jira/browse/BEAM-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jack McCluskey updated BEAM-11087:
----------------------------------
Fix Version/s: 2.35.0
Resolution: Fixed
Status: Resolved (was: Open)
> [Go SDK] Validate Side Input behavior WRT windows
> --------------------------------------------------
>
> Key: BEAM-11087
> URL: https://issues.apache.org/jira/browse/BEAM-11087
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Jack McCluskey
> Priority: P3
> Fix For: 2.35.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> DoFns with Side inputs implicitly observe windows, as Side Inputs are scoped to a current window, a powerful feature of beam.
> The ideal would be to you could try to unit test this in the exec package, creating a fake side input adapter (or use the real one), to more directly target the implementation.
> [https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/sideinput.go#L34] and exercising the ParDo code for Side Input handling directly.
> [https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/pardo.go#L38]
> Then it should be possible to bolster that with appropriate mock windows to ensure that side inputs are configured correctly.
> Altnernatively this behavior could be tested and validated with an integration test against real runners due to the complexity around Side Inputs making unit testing a challenge. (While side input code could be tested in that fashion, it's likely dramatically simpler to do the integration test.)
> Some light tests with Side Inputs already exist, but they're purely in the Global Window. Add tests for non-global windows to ensure that Side Inputs are scoped correctly.
> Integration test directory: [https://github.com/apache/beam/tree/master/sdks/go/test/integration/primitives]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)