You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by "lostluck (via GitHub)" <gi...@apache.org> on 2023/07/27 02:44:25 UTC

[GitHub] [beam] lostluck opened a new pull request, #27703: [#24789] Make Prism the default Go SDK runner.

lostluck opened a new pull request, #27703:
URL: https://github.com/apache/beam/pull/27703

   Final PR that sets the prism runner as the default runner for the Go SDK.
   
   Large PR with all changes, and passing tests: see 27550. This PR should pass everything once all parts of that PR have been submitted.
   
   See https://github.com/apache/beam/issues/24789
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] jrmccluskey commented on a diff in pull request #27703: [#24789] Make Prism the default Go SDK runner.

Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.

jrmccluskey commented on code in PR #27703:
URL: https://github.com/apache/beam/pull/27703#discussion_r1281115878


##########
sdks/go/test/integration/integration.go:
##########
@@ -136,6 +136,39 @@ var portableFilters = []string{
 	"TestSetStateClear",
 }
 
+// TODO(lostluck): set up a specific run for these.
+var prismFilters = []string{
+	// The portable runner does not support the TestStream primitive

Review Comment:
   ```suggestion
   	// The prism runner does not support the TestStream primitive
   ```



##########
sdks/go/test/integration/integration.go:
##########
@@ -136,6 +136,39 @@ var portableFilters = []string{
 	"TestSetStateClear",
 }
 
+// TODO(lostluck): set up a specific run for these.
+var prismFilters = []string{
+	// The portable runner does not support the TestStream primitive
+	"TestTestStream.*",
+	// The trigger and pane tests uses TestStream
+	"TestTrigger.*",
+	"TestPanes",
+	// TODO(https://github.com/apache/beam/issues/21058): Python portable runner times out on Kafka reads.
+	"TestKafkaIO.*",
+	// TODO(BEAM-13215): GCP IOs currently do not work in non-Dataflow portable runners.
+	"TestBigQueryIO.*",
+	"TestSpannerIO.*",
+	// The portable runner does not support self-checkpointing
+	"TestCheckpointing",
+	// The portable runner does not support pipeline drain for SDF.
+	"TestDrain",
+	// FhirIO currently only supports Dataflow runner
+	"TestFhirIO.*",
+	// OOMs currently only lead to heap dumps on Dataflow runner
+	"TestOomParDo",
+	// The portable runner does not support user state.

Review Comment:
   ```suggestion
   	// The prism runner does not support user state.
   ```



##########
sdks/go/test/integration/integration.go:
##########
@@ -136,6 +136,39 @@ var portableFilters = []string{
 	"TestSetStateClear",
 }
 
+// TODO(lostluck): set up a specific run for these.
+var prismFilters = []string{
+	// The portable runner does not support the TestStream primitive
+	"TestTestStream.*",
+	// The trigger and pane tests uses TestStream
+	"TestTrigger.*",
+	"TestPanes",
+	// TODO(https://github.com/apache/beam/issues/21058): Python portable runner times out on Kafka reads.
+	"TestKafkaIO.*",
+	// TODO(BEAM-13215): GCP IOs currently do not work in non-Dataflow portable runners.
+	"TestBigQueryIO.*",
+	"TestSpannerIO.*",
+	// The portable runner does not support self-checkpointing
+	"TestCheckpointing",
+	// The portable runner does not support pipeline drain for SDF.

Review Comment:
   ```suggestion
   	// The prism runner does not support pipeline drain for SDF.
   ```



##########
sdks/go/test/integration/integration.go:
##########
@@ -136,6 +136,39 @@ var portableFilters = []string{
 	"TestSetStateClear",
 }
 
+// TODO(lostluck): set up a specific run for these.
+var prismFilters = []string{
+	// The portable runner does not support the TestStream primitive
+	"TestTestStream.*",
+	// The trigger and pane tests uses TestStream
+	"TestTrigger.*",
+	"TestPanes",
+	// TODO(https://github.com/apache/beam/issues/21058): Python portable runner times out on Kafka reads.

Review Comment:
   If this issue is present for prism you'll want a more specific error message



##########
sdks/go/test/integration/integration.go:
##########
@@ -136,6 +136,39 @@ var portableFilters = []string{
 	"TestSetStateClear",
 }
 
+// TODO(lostluck): set up a specific run for these.
+var prismFilters = []string{
+	// The portable runner does not support the TestStream primitive
+	"TestTestStream.*",
+	// The trigger and pane tests uses TestStream
+	"TestTrigger.*",
+	"TestPanes",
+	// TODO(https://github.com/apache/beam/issues/21058): Python portable runner times out on Kafka reads.
+	"TestKafkaIO.*",
+	// TODO(BEAM-13215): GCP IOs currently do not work in non-Dataflow portable runners.
+	"TestBigQueryIO.*",
+	"TestSpannerIO.*",
+	// The portable runner does not support self-checkpointing

Review Comment:
   ```suggestion
   	// The prism runner does not support self-checkpointing
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org