[GitHub] [beam] youngoli commented on a diff in pull request #21979: Add Search transform to Go FhirIO

[GitBox <gi...@apache.org>]

youngoli commented on code in PR #21979:
URL: https://github.com/apache/beam/pull/21979#discussion_r911540966


##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -177,6 +178,22 @@ func extractResourcePathFrom(resourceLocationUrl string) (string, error) {
 	return resourceLocationUrl[startIdx:endIdx], nil
 }
 
+// Useful to prevent flaky results.
+func runWithBackoffRetries(t *testing.T, p *beam.Pipeline) error {

Review Comment:
   I'm still hesitating with this one only because rerunning a pipeline on Dataflow is _slow_. This isn't a problem on the Direct runner, but on Dataflow rerunning a pipeline is around a 5 minute affair even for a really simple pipeline.
   
   So this function will work. In the sense that 5 minutes is definitely going to be long enough to fix the flakiness.  But waiting a guaranteed 30 seconds before running the pipeline is probably still better overall.
   
   By the way, I should probably mention that tests within a package in Go run in sequence by default. So the reason I'm being very conscious of every additional integration test in this file is because each one will make the entire test suite take ~5 minutes longer to finish (on Dataflow at least).
   
   Anyway, I'm still gonna merge this because this isn't bad enough to block anything. But if you can think of any way to fix the flakiness without rerunning the whole pipeline it would probably be much better for performance on Dataflow.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org