You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/02/08 15:30:35 UTC
[GitHub] [beam] jrmccluskey opened a new pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
jrmccluskey opened a new pull request #16775:
URL: https://github.com/apache/beam/pull/16775
Adds support for native iterable side inputs to the Go SDK, removing the fixed key + multimap infrastructure that was previously in place and replacing it with a Fn API iterable side input request instead. Also includes some refactors to clarify what functions are doing.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/badge/icon?subject=Portable">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJava_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJava_Dataflow/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_JavaUsingPython_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_JavaUsingPython_Dataflow/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r802714417
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput_test.go
##########
@@ -0,0 +1,201 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exec
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "reflect"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/coder"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/window"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/typex"
+)
+
+func makeWindowedCoder() *coder.Coder {
+ vCoder := coder.NewDouble()
+ return coder.NewW(vCoder, coder.NewGlobalWindow())
+}
+
+func makeWindowedKVCoder() *coder.Coder {
+ kCoder := coder.NewString()
+ vCoder := coder.NewDouble()
+ kvCoder := coder.NewKV([]*coder.Coder{kCoder, vCoder})
+ return coder.NewW(kvCoder, coder.NewGlobalWindow())
+}
+
+func TestNewSideInputAdapter(t *testing.T) {
+ tests := []struct {
+ name string
+ sid StreamID
+ sideInputID string
+ c *coder.Coder
+ kc ElementEncoder
+ ec ElementDecoder
+ }{
+ {
+ name: "KV coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedKVCoder(),
+ kc: &stringEncoder{},
+ ec: &doubleDecoder{},
+ },
+ {
+ name: "V coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedCoder(),
+ kc: nil,
+ ec: &doubleDecoder{},
+ },
+ }
+ for _, test := range tests {
+ adapter := NewSideInputAdapter(test.sid, test.sideInputID, test.c, nil)
+ adapterStruct, ok := adapter.(*sideInputAdapter)
+ if !ok {
+ t.Errorf("failed to convert interface to sideInputAdapter struct in test %v", test)
+ }
+ if got, want := adapterStruct.sid, test.sid; got != want {
+ t.Errorf("got SID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.sideInputID, test.sideInputID; got != want {
+ t.Errorf("got sideInputID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.c, test.c; got != want {
+ t.Errorf("got coder %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.kc), reflect.TypeOf(test.kc); got != want {
+ t.Errorf("got ElementEncoder type %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.ec), reflect.TypeOf(test.ec); got != want {
+ t.Errorf("got ElementDecoder type %v, want %v", got, want)
+ }
+ }
+}
+
+func TestNewKeyedIterable_Unkeyed(t *testing.T) {
+ adapter := NewSideInputAdapter(StreamID{}, "", makeWindowedCoder(), nil)
+ rs, err := adapter.NewKeyedIterable(context.Background(), nil, nil, nil)
+ if err == nil {
+ t.Error("NewKeyedIterable() succeeded when it should have failed")
+ }
+ if rs != nil {
+ t.Errorf("NewKeyedIterable() returned a ReStream when it should not have, got %v", rs)
Review comment:
We can always beef up the testing here later since that's a focus moving forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f8edbaf) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.14%`.
> The diff coverage is `21.56%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.64% +0.14%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9319 +48
+ Misses 9679 9671 -8
+ Partials 990 989 -1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `23.45% <31.25%> (+12.74%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...f8edbaf](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c95c993) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.08%`.
> The diff coverage is `1.96%`.
> :exclamation: Current head c95c993 differs from pull request most recent head ba56c74. Consider uploading reports for the commit ba56c74 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.57% +0.08%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9306 +35
- Misses 9679 9685 +6
+ Partials 990 988 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `7.40% <0.00%> (-3.31%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...ba56c74](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r802712399
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput_test.go
##########
@@ -0,0 +1,201 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exec
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "reflect"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/coder"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/window"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/typex"
+)
+
+func makeWindowedCoder() *coder.Coder {
+ vCoder := coder.NewDouble()
+ return coder.NewW(vCoder, coder.NewGlobalWindow())
+}
+
+func makeWindowedKVCoder() *coder.Coder {
+ kCoder := coder.NewString()
+ vCoder := coder.NewDouble()
+ kvCoder := coder.NewKV([]*coder.Coder{kCoder, vCoder})
+ return coder.NewW(kvCoder, coder.NewGlobalWindow())
+}
+
+func TestNewSideInputAdapter(t *testing.T) {
+ tests := []struct {
+ name string
+ sid StreamID
+ sideInputID string
+ c *coder.Coder
+ kc ElementEncoder
+ ec ElementDecoder
+ }{
+ {
+ name: "KV coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedKVCoder(),
+ kc: &stringEncoder{},
+ ec: &doubleDecoder{},
+ },
+ {
+ name: "V coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedCoder(),
+ kc: nil,
+ ec: &doubleDecoder{},
+ },
+ }
+ for _, test := range tests {
+ adapter := NewSideInputAdapter(test.sid, test.sideInputID, test.c, nil)
+ adapterStruct, ok := adapter.(*sideInputAdapter)
+ if !ok {
+ t.Errorf("failed to convert interface to sideInputAdapter struct in test %v", test)
+ }
+ if got, want := adapterStruct.sid, test.sid; got != want {
+ t.Errorf("got SID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.sideInputID, test.sideInputID; got != want {
+ t.Errorf("got sideInputID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.c, test.c; got != want {
+ t.Errorf("got coder %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.kc), reflect.TypeOf(test.kc); got != want {
+ t.Errorf("got ElementEncoder type %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.ec), reflect.TypeOf(test.ec); got != want {
+ t.Errorf("got ElementDecoder type %v, want %v", got, want)
+ }
Review comment:
You're totally right, I dropped in the name field so it could be piped through and forgot to put it into the error messages. t.Run() looks helpful for that pattern, too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e1f916) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.24%`.
> The diff coverage is `41.17%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.74% +0.24%
==========================================
Files 202 203 +1
Lines 19940 20020 +80
==========================================
+ Hits 9271 9358 +87
+ Misses 9679 9661 -18
- Partials 990 1001 +11
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `41.97% <62.50%> (+31.26%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/io/xlang/schemaio/schemaio.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby94bGFuZy9zY2hlbWFpby9zY2hlbWFpby5nbw==) | `43.90% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.84% <0.00%> (+1.07%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...4e1f916](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032795812
R: @lostluck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f7270e8) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.21%`.
> The diff coverage is `41.17%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.71% +0.21%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9333 +62
+ Misses 9679 9653 -26
- Partials 990 993 +3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `37.03% <62.50%> (+26.32%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.30% <0.00%> (+0.53%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...f7270e8](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r802034197
##########
File path: sdks/go/pkg/beam/core/runtime/graphx/translate.go
##########
@@ -418,7 +379,7 @@ func (m *marshaller) addMultiEdge(edge NamedEdge) ([]string, error) {
si[fmt.Sprintf("i%v", i)] = &pipepb.SideInput{
AccessPattern: &pipepb.FunctionSpec{
- Urn: URNMultimapSideInput,
+ Urn: URNIterableSideInput,
},
ViewFn: &pipepb.FunctionSpec{
Urn: "foo",
Review comment:
ViewFns are not currently implemented in the Go SDK but the proto requires the field to be non-empty. So we put a dummy placeholder URN in place for now. The WindowMappingFn field had the same set-up until I implemented the basic window mapping functionality we use now.
"Required" fields in protos are a legacy thing that are discouraged/outright unsupported now since they're a headache to deal with, generally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
lostluck commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r802187655
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput_test.go
##########
@@ -0,0 +1,201 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exec
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "reflect"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/coder"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/window"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/typex"
+)
+
+func makeWindowedCoder() *coder.Coder {
+ vCoder := coder.NewDouble()
+ return coder.NewW(vCoder, coder.NewGlobalWindow())
+}
+
+func makeWindowedKVCoder() *coder.Coder {
+ kCoder := coder.NewString()
+ vCoder := coder.NewDouble()
+ kvCoder := coder.NewKV([]*coder.Coder{kCoder, vCoder})
+ return coder.NewW(kvCoder, coder.NewGlobalWindow())
+}
+
+func TestNewSideInputAdapter(t *testing.T) {
+ tests := []struct {
+ name string
+ sid StreamID
+ sideInputID string
+ c *coder.Coder
+ kc ElementEncoder
+ ec ElementDecoder
+ }{
+ {
+ name: "KV coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedKVCoder(),
+ kc: &stringEncoder{},
+ ec: &doubleDecoder{},
+ },
+ {
+ name: "V coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedCoder(),
+ kc: nil,
+ ec: &doubleDecoder{},
+ },
+ }
+ for _, test := range tests {
+ adapter := NewSideInputAdapter(test.sid, test.sideInputID, test.c, nil)
+ adapterStruct, ok := adapter.(*sideInputAdapter)
+ if !ok {
+ t.Errorf("failed to convert interface to sideInputAdapter struct in test %v", test)
+ }
+ if got, want := adapterStruct.sid, test.sid; got != want {
+ t.Errorf("got SID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.sideInputID, test.sideInputID; got != want {
+ t.Errorf("got sideInputID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.c, test.c; got != want {
+ t.Errorf("got coder %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.kc), reflect.TypeOf(test.kc); got != want {
+ t.Errorf("got ElementEncoder type %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.ec), reflect.TypeOf(test.ec); got != want {
+ t.Errorf("got ElementDecoder type %v, want %v", got, want)
+ }
Review comment:
Consider wrapping this in t.Run so that failures will be more identifiable to the test case. As it stands, if something failed in two cases, it might not be easy to idenfity where they came from. Further, see https://github.com/golang/go/wiki/TestComments#got-before-want and https://github.com/golang/go/wiki/TestComments#table-driven-tests-vs-multiple-test-functions
```suggestion
t.Run(test.name, func(t *testing.T) {
adapter := NewSideInputAdapter(test.sid, test.sideInputID, test.c, nil)
adapterStruct, ok := adapter.(*sideInputAdapter)
if !ok {
t.Errorf("failed to convert interface to sideInputAdapter struct in test %v", test)
}
if got, want := adapterStruct.sid, test.sid; got != want {
t.Errorf("got SID %v, want %v", got, want)
}
if got, want := adapterStruct.sideInputID, test.sideInputID; got != want {
t.Errorf("got sideInputID %v, want %v", got, want)
}
if got, want := adapterStruct.c, test.c; got != want {
t.Errorf("got coder %v, want %v", got, want)
}
if got, want := reflect.TypeOf(adapterStruct.kc), reflect.TypeOf(test.kc); got != want {
t.Errorf("got ElementEncoder type %v, want %v", got, want)
}
if got, want := reflect.TypeOf(adapterStruct.ec), reflect.TypeOf(test.ec); got != want {
t.Errorf("got ElementDecoder type %v, want %v", got, want)
}
})
```
##########
File path: sdks/go/pkg/beam/core/runtime/graphx/translate.go
##########
@@ -418,7 +379,7 @@ func (m *marshaller) addMultiEdge(edge NamedEdge) ([]string, error) {
si[fmt.Sprintf("i%v", i)] = &pipepb.SideInput{
AccessPattern: &pipepb.FunctionSpec{
- Urn: URNMultimapSideInput,
+ Urn: URNIterableSideInput,
},
ViewFn: &pipepb.FunctionSpec{
Urn: "foo",
Review comment:
In this case, it's "required" by some runner implementations to be non-empty, if not actually required by explicit proto semantics.
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput_test.go
##########
@@ -0,0 +1,201 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exec
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "reflect"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/coder"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/window"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/typex"
+)
+
+func makeWindowedCoder() *coder.Coder {
+ vCoder := coder.NewDouble()
+ return coder.NewW(vCoder, coder.NewGlobalWindow())
+}
+
+func makeWindowedKVCoder() *coder.Coder {
+ kCoder := coder.NewString()
+ vCoder := coder.NewDouble()
+ kvCoder := coder.NewKV([]*coder.Coder{kCoder, vCoder})
+ return coder.NewW(kvCoder, coder.NewGlobalWindow())
+}
+
+func TestNewSideInputAdapter(t *testing.T) {
+ tests := []struct {
+ name string
+ sid StreamID
+ sideInputID string
+ c *coder.Coder
+ kc ElementEncoder
+ ec ElementDecoder
+ }{
+ {
+ name: "KV coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedKVCoder(),
+ kc: &stringEncoder{},
+ ec: &doubleDecoder{},
+ },
+ {
+ name: "V coder",
+ sid: StreamID{Port: Port{URL: "localhost:8099"}, PtransformID: "n0"},
+ sideInputID: "i0",
+ c: makeWindowedCoder(),
+ kc: nil,
+ ec: &doubleDecoder{},
+ },
+ }
+ for _, test := range tests {
+ adapter := NewSideInputAdapter(test.sid, test.sideInputID, test.c, nil)
+ adapterStruct, ok := adapter.(*sideInputAdapter)
+ if !ok {
+ t.Errorf("failed to convert interface to sideInputAdapter struct in test %v", test)
+ }
+ if got, want := adapterStruct.sid, test.sid; got != want {
+ t.Errorf("got SID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.sideInputID, test.sideInputID; got != want {
+ t.Errorf("got sideInputID %v, want %v", got, want)
+ }
+ if got, want := adapterStruct.c, test.c; got != want {
+ t.Errorf("got coder %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.kc), reflect.TypeOf(test.kc); got != want {
+ t.Errorf("got ElementEncoder type %v, want %v", got, want)
+ }
+ if got, want := reflect.TypeOf(adapterStruct.ec), reflect.TypeOf(test.ec); got != want {
+ t.Errorf("got ElementDecoder type %v, want %v", got, want)
+ }
+ }
+}
+
+func TestNewKeyedIterable_Unkeyed(t *testing.T) {
+ adapter := NewSideInputAdapter(StreamID{}, "", makeWindowedCoder(), nil)
+ rs, err := adapter.NewKeyedIterable(context.Background(), nil, nil, nil)
+ if err == nil {
+ t.Error("NewKeyedIterable() succeeded when it should have failed")
+ }
+ if rs != nil {
+ t.Errorf("NewKeyedIterable() returned a ReStream when it should not have, got %v", rs)
Review comment:
I don't love a plain constructor only test. Ideally we would be faking out the cache or state reader behavior, but that feels like a bit much. We'd probably want to have some additional test infrastructure to get easier to use Test ReStreams and a Test io.ReadCloser to validate the other half of things. Probably better to punt to another PR. (we have some existing utilities in a another test file in the exec package.)
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput.go
##########
@@ -43,26 +43,71 @@ type sideInputAdapter struct {
kc ElementEncoder
ec ElementDecoder
wm WindowMapper
+ c *coder.Coder
}
// NewSideInputAdapter returns a side input adapter for the given StreamID and coder.
-// It expects a W<KV<K,V>> coder, because the protocol supports MultiSet access only.
+// It expects a W<V> or W<KV<K,V>> coder, because the protocol requires windowing information.
func NewSideInputAdapter(sid StreamID, sideInputID string, c *coder.Coder, wm WindowMapper) SideInputAdapter {
- if !coder.IsW(c) || !coder.IsKV(coder.SkipW(c)) {
- panic(fmt.Sprintf("expected WKV coder for side input %v: %v", sid, c))
+ if !coder.IsW(c) {
+ panic(fmt.Sprintf("expected WV coder for side input %v: %v", sid, c))
}
wc := MakeWindowEncoder(c.Window)
- kc := MakeElementEncoder(coder.SkipW(c).Components[0])
- ec := MakeElementDecoder(coder.SkipW(c).Components[1])
- return &sideInputAdapter{sid: sid, sideInputID: sideInputID, wc: wc, kc: kc, ec: ec, wm: wm}
+ var kc ElementEncoder
+ var ec ElementDecoder
+ if coder.IsKV(coder.SkipW(c)) {
+ kc = MakeElementEncoder(coder.SkipW(c).Components[0])
+ ec = MakeElementDecoder(coder.SkipW(c).Components[1])
+ } else {
+ ec = MakeElementDecoder(coder.SkipW(c))
Review comment:
Good explanation!
In particular, for MultiMap side inputs, a user requests the values for a given Key, which we then encode with the window, and we return out the unwindowed values out as the user iterates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c95c993) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.08%`.
> The diff coverage is `1.96%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.57% +0.08%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9306 +35
- Misses 9679 9685 +6
+ Partials 990 988 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `7.40% <0.00%> (-3.31%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...c95c993](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f8edbaf) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.14%`.
> The diff coverage is `21.56%`.
> :exclamation: Current head f8edbaf differs from pull request most recent head f7270e8. Consider uploading reports for the commit f7270e8 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.64% +0.14%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9319 +48
+ Misses 9679 9671 -8
+ Partials 990 989 -1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `23.45% <31.25%> (+12.74%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...f7270e8](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck merged pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
lostluck merged pull request #16775:
URL: https://github.com/apache/beam/pull/16775
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c95c993) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.08%`.
> The diff coverage is `1.96%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.57% +0.08%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9306 +35
- Misses 9679 9685 +6
+ Partials 990 988 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `7.40% <0.00%> (-3.31%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...c95c993](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r802032193
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput.go
##########
@@ -43,26 +43,71 @@ type sideInputAdapter struct {
kc ElementEncoder
ec ElementDecoder
wm WindowMapper
+ c *coder.Coder
}
// NewSideInputAdapter returns a side input adapter for the given StreamID and coder.
-// It expects a W<KV<K,V>> coder, because the protocol supports MultiSet access only.
+// It expects a W<V> or W<KV<K,V>> coder, because the protocol requires windowing information.
func NewSideInputAdapter(sid StreamID, sideInputID string, c *coder.Coder, wm WindowMapper) SideInputAdapter {
- if !coder.IsW(c) || !coder.IsKV(coder.SkipW(c)) {
- panic(fmt.Sprintf("expected WKV coder for side input %v: %v", sid, c))
+ if !coder.IsW(c) {
+ panic(fmt.Sprintf("expected WV coder for side input %v: %v", sid, c))
}
wc := MakeWindowEncoder(c.Window)
- kc := MakeElementEncoder(coder.SkipW(c).Components[0])
- ec := MakeElementDecoder(coder.SkipW(c).Components[1])
- return &sideInputAdapter{sid: sid, sideInputID: sideInputID, wc: wc, kc: kc, ec: ec, wm: wm}
+ var kc ElementEncoder
+ var ec ElementDecoder
+ if coder.IsKV(coder.SkipW(c)) {
+ kc = MakeElementEncoder(coder.SkipW(c).Components[0])
+ ec = MakeElementDecoder(coder.SkipW(c).Components[1])
+ } else {
+ ec = MakeElementDecoder(coder.SkipW(c))
Review comment:
In this particular context, we need coders when we're communicating with the runner over the Fn API. The key we are querying for the side input in a keyed context needs to be encoded when put into the proto, and then we need to decode the elements we get out from the runner so we can pass them back in the correct form.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b6ec5e1) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.25%`.
> The diff coverage is `41.17%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.74% +0.25%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9340 +69
+ Misses 9679 9644 -35
- Partials 990 995 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `41.97% <62.50%> (+31.26%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.84% <0.00%> (+1.07%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...b6ec5e1](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f7270e8) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.21%`.
> The diff coverage is `41.17%`.
> :exclamation: Current head f7270e8 differs from pull request most recent head b6ec5e1. Consider uploading reports for the commit b6ec5e1 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.71% +0.21%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9333 +62
+ Misses 9679 9653 -26
- Partials 990 993 +3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `37.03% <62.50%> (+26.32%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.30% <0.00%> (+0.53%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...b6ec5e1](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a change in pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
damccorm commented on a change in pull request #16775:
URL: https://github.com/apache/beam/pull/16775#discussion_r801914322
##########
File path: sdks/go/pkg/beam/core/runtime/exec/sideinput.go
##########
@@ -43,26 +43,71 @@ type sideInputAdapter struct {
kc ElementEncoder
ec ElementDecoder
wm WindowMapper
+ c *coder.Coder
}
// NewSideInputAdapter returns a side input adapter for the given StreamID and coder.
-// It expects a W<KV<K,V>> coder, because the protocol supports MultiSet access only.
+// It expects a W<V> or W<KV<K,V>> coder, because the protocol requires windowing information.
func NewSideInputAdapter(sid StreamID, sideInputID string, c *coder.Coder, wm WindowMapper) SideInputAdapter {
- if !coder.IsW(c) || !coder.IsKV(coder.SkipW(c)) {
- panic(fmt.Sprintf("expected WKV coder for side input %v: %v", sid, c))
+ if !coder.IsW(c) {
+ panic(fmt.Sprintf("expected WV coder for side input %v: %v", sid, c))
}
wc := MakeWindowEncoder(c.Window)
- kc := MakeElementEncoder(coder.SkipW(c).Components[0])
- ec := MakeElementDecoder(coder.SkipW(c).Components[1])
- return &sideInputAdapter{sid: sid, sideInputID: sideInputID, wc: wc, kc: kc, ec: ec, wm: wm}
+ var kc ElementEncoder
+ var ec ElementDecoder
+ if coder.IsKV(coder.SkipW(c)) {
+ kc = MakeElementEncoder(coder.SkipW(c).Components[0])
+ ec = MakeElementDecoder(coder.SkipW(c).Components[1])
+ } else {
+ ec = MakeElementDecoder(coder.SkipW(c))
Review comment:
Could you help me understand why we need an encoder and decoder in the KV case, and just a decoder in this case (aka, please help Danny understand how we use Coders 😅)
##########
File path: sdks/go/pkg/beam/core/runtime/graphx/translate.go
##########
@@ -418,7 +379,7 @@ func (m *marshaller) addMultiEdge(edge NamedEdge) ([]string, error) {
si[fmt.Sprintf("i%v", i)] = &pipepb.SideInput{
AccessPattern: &pipepb.FunctionSpec{
- Urn: URNMultimapSideInput,
+ Urn: URNIterableSideInput,
},
ViewFn: &pipepb.FunctionSpec{
Urn: "foo",
Review comment:
I know this is coming from the MultiMap version of this, but what does `Urn: "foo"` actually do here? Is this doing something different than just leaving the ViewFn empty would?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b6ec5e1) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.25%`.
> The diff coverage is `41.17%`.
> :exclamation: Current head b6ec5e1 differs from pull request most recent head 4e1f916. Consider uploading reports for the commit 4e1f916 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.74% +0.25%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9340 +69
+ Misses 9679 9644 -35
- Partials 990 995 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `41.97% <62.50%> (+31.26%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.84% <0.00%> (+1.07%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...4e1f916](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ba56c74) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.13%`.
> The diff coverage is `17.64%`.
> :exclamation: Current head ba56c74 differs from pull request most recent head f8edbaf. Consider uploading reports for the commit f8edbaf to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.62% +0.13%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9316 +45
+ Misses 9679 9674 -5
+ Partials 990 989 -1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `19.75% <25.00%> (+9.03%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...f8edbaf](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032736875
Run Go Flink ValidatesRunner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032737283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ba56c74) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.13%`.
> The diff coverage is `17.64%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.62% +0.13%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9316 +45
+ Misses 9679 9674 -5
+ Partials 990 989 -1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `19.75% <25.00%> (+9.03%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...ba56c74](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #16775: [BEAM-13803] Add support for native iterable side inputs to the Go SDK
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #16775:
URL: https://github.com/apache/beam/pull/16775#issuecomment-1032743197
# [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#16775](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f7270e8) into [master](https://codecov.io/gh/apache/beam/commit/bb15d1896a2972bb6b95eaa28408345ac8c27b50?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bb15d18) will **increase** coverage by `0.21%`.
> The diff coverage is `41.17%`.
[](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #16775 +/- ##
==========================================
+ Coverage 46.49% 46.71% +0.21%
==========================================
Files 202 202
Lines 19940 19979 +39
==========================================
+ Hits 9271 9333 +62
+ Misses 9679 9653 -26
- Partials 990 993 +3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/runtime/exec/data.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9kYXRhLmdv) | `0.00% <ø> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/harness/statemgr.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9zdGF0ZW1nci5nbw==) | `51.01% <0.00%> (-3.78%)` | :arrow_down: |
| [sdks/go/pkg/beam/core/runtime/exec/sideinput.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zaWRlaW5wdXQuZ28=) | `37.03% <62.50%> (+26.32%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/translate.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L3RyYW5zbGF0ZS5nbw==) | `31.53% <100.00%> (+3.15%)` | :arrow_up: |
| [sdks/go/pkg/beam/artifact/materialize.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9hcnRpZmFjdC9tYXRlcmlhbGl6ZS5nbw==) | `48.94% <0.00%> (ø)` | |
| [.../go/pkg/beam/runners/dataflow/dataflowlib/stage.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL3N0YWdlLmdv) | `0.00% <0.00%> (ø)` | |
| [...o/pkg/beam/runners/dataflow/dataflowlib/execute.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL2RhdGFmbG93L2RhdGFmbG93bGliL2V4ZWN1dGUuZ28=) | `0.00% <0.00%> (ø)` | |
| [sdks/go/pkg/beam/core/runtime/exec/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9jb2Rlci5nbw==) | `31.30% <0.00%> (+0.53%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/graphx/coder.go](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZ3JhcGh4L2NvZGVyLmdv) | `57.94% <0.00%> (+4.41%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/beam/pull/16775/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bb15d18...f7270e8](https://codecov.io/gh/apache/beam/pull/16775?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org