You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/13 22:02:31 UTC
[GitHub] [beam] lnogueir opened a new pull request, #21840: Add ExecuteBundles transform to Go FhirIO
lnogueir opened a new pull request, #21840:
URL: https://github.com/apache/beam/pull/21840
Porting [ExecuteBundles](https://github.com/apache/beam/blob/3c623abd631b493c8a71d05fe591065754c75d0c/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java#L1368) transform from Java to Go FhirIO.
Also, extracted some shared code between transforms to common.go.
Fixes https://github.com/apache/beam/issues/21726
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154485201
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
youngoli commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154767762
Run Go PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154483161
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154483160
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154485814
# [Codecov](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#21840](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ca8e99e) into [master](https://codecov.io/gh/apache/beam/commit/8a779c0516ff904de923ce451762843f28ba0f44?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8a779c0) will **decrease** coverage by `0.01%`.
> The diff coverage is `64.22%`.
```diff
@@ Coverage Diff @@
## master #21840 +/- ##
==========================================
- Coverage 74.15% 74.14% -0.02%
==========================================
Files 698 699 +1
Lines 92417 92503 +86
==========================================
+ Hits 68530 68582 +52
- Misses 22636 22669 +33
- Partials 1251 1252 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| go | `50.95% <64.22%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/io/fhirio/execute\_bundles.go](https://codecov.io/gh/apache/beam/pull/21840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby9maGlyaW8vZXhlY3V0ZV9idW5kbGVzLmdv) | `56.71% <56.71%> (ø)` | |
| [sdks/go/pkg/beam/io/fhirio/common.go](https://codecov.io/gh/apache/beam/pull/21840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby9maGlyaW8vY29tbW9uLmdv) | `59.45% <73.33%> (+59.45%)` | :arrow_up: |
| [sdks/go/pkg/beam/io/fhirio/read.go](https://codecov.io/gh/apache/beam/pull/21840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby9maGlyaW8vcmVhZC5nbw==) | `82.75% <83.33%> (+2.75%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8a779c0...ca8e99e](https://codecov.io/gh/apache/beam/pull/21840?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r897346064
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -192,12 +204,29 @@ func TestFhirIO_InvalidRead(t *testing.T) {
passert.Count(s, failedReads, "", 1)
passert.Empty(s, resources)
passert.True(s, failedReads, func(errorMsg string) bool {
- return strings.Contains(errorMsg, "bad status [404]")
+ return strings.Contains(errorMsg, "404")
Review Comment:
Nice, I wasn't aware of these constants.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] DanKotowski commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
DanKotowski commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r901910056
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -75,13 +84,16 @@ func setupFhirStore(t *testing.T) (string, []string, func()) {
healthcareDataset := fmt.Sprintf(datasetPathFmt, *gcpopts.Project, *gcpopts.Region)
Review Comment:
If we don't have immediate need I think this should be ok as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
youngoli commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154768078
Will merge after confirming the integration test passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154483164
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] DanKotowski commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
DanKotowski commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r901912263
##########
sdks/go/pkg/beam/io/fhirio/execute_bundles_test.go:
##########
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fhirio
+
+import (
+ "bytes"
+ "net/http"
+ "strings"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/passert"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest"
+)
+
+func TestExecuteBundles(t *testing.T) {
Review Comment:
OK SGTM.
##########
sdks/go/pkg/beam/io/fhirio/read_test.go:
##########
@@ -75,23 +57,10 @@ func TestRead(t *testing.T) {
return strings.Contains(errorMsg, testCase.containedError)
})
pipelineResult := ptest.RunAndValidate(t, p)
- counterResults := pipelineResult.Metrics().AllMetrics().Counters()
-
- if len(counterResults) != 1 {
- t.Fatalf("counterResults got length %v, expected %v", len(counterResults), 1)
- }
- counterResult := counterResults[0]
-
- expectedCounterName := "fhirio/read_resource_error_count"
- if counterResult.Name() != expectedCounterName {
- t.Fatalf("counterResult.Name() is '%v', expected '%v'", counterResult.Name(), expectedCounterName)
+ err := validateResourceErrorCounter(pipelineResult, len(testResourcePaths))
+ if err != nil {
+ t.Fatal(err.Error())
Review Comment:
This should have t.Fatalf and give some test context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154483167
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154484213
R: @youngoli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli merged pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
youngoli merged PR #21840:
URL: https://github.com/apache/beam/pull/21840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154507478
R: @DanKotowski
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
youngoli commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r896415866
##########
sdks/go/pkg/beam/io/fhirio/common.go:
##########
@@ -32,8 +38,54 @@ const (
baseMetricPrefix = "fhirio/"
)
+func executeRequestAndRecordLatency(ctx context.Context, latencyMs *beam.Distribution, requestSupplier func() (*http.Response, error)) (*http.Response, error) {
Review Comment:
Good thinking 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r896212908
##########
sdks/go/pkg/beam/io/fhirio/common.go:
##########
@@ -32,8 +38,54 @@ const (
baseMetricPrefix = "fhirio/"
)
+func executeRequestAndRecordLatency(ctx context.Context, latencyMs *beam.Distribution, requestSupplier func() (*http.Response, error)) (*http.Response, error) {
Review Comment:
@youngoli, follow up to [this comment](https://github.com/lnogueir/beam/pull/24#discussion_r896060946), I got rid of the generic return value here because I double checked that the response would always be a `*http.Response`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1154483169
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] DanKotowski commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
DanKotowski commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r897278363
##########
sdks/go/pkg/beam/io/fhirio/utils_test.go:
##########
@@ -0,0 +1,105 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fhirio
+
+import (
+ "errors"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam"
+ "net/http"
+ "testing"
+)
+
+var (
+ fakeRequestReturnErrorMessage = "internal error"
+ requestReturnErrorFakeClient = &fakeFhirStoreClient{
+ fakeReadResources: func(resource string) (*http.Response, error) {
+ return nil, errors.New(fakeRequestReturnErrorMessage)
+ },
+ fakeExecuteBundles: func(storePath string, bundle []byte) (*http.Response, error) {
+ return nil, errors.New(fakeRequestReturnErrorMessage)
+ },
+ }
+
+ fakeBadStatus = "403 Forbidden"
+ badStatusFakeResponse = &http.Response{Status: fakeBadStatus}
+ badStatusFakeClient = &fakeFhirStoreClient{
+ fakeReadResources: func(resource string) (*http.Response, error) {
+ return badStatusFakeResponse, nil
+ },
+ fakeExecuteBundles: func(storePath string, bundle []byte) (*http.Response, error) {
+ return badStatusFakeResponse, nil
+ },
+ }
+
+ fakeBodyReaderErrorMessage = "ReadAll fail"
+ bodyReaderErrorFakeResponse = &http.Response{
+ Body: &fakeReaderCloser{
+ fakeRead: func([]byte) (int, error) {
+ return 0, errors.New(fakeBodyReaderErrorMessage)
+ },
+ }, Status: "200 Ok"}
+ bodyReaderErrorFakeClient = &fakeFhirStoreClient{
+ fakeReadResources: func(resource string) (*http.Response, error) {
+ return bodyReaderErrorFakeResponse, nil
+ },
+ fakeExecuteBundles: func(storePath string, bundle []byte) (*http.Response, error) {
+ return bodyReaderErrorFakeResponse, nil
+ },
+ }
+)
+
+type fakeFhirStoreClient struct {
+ fakeReadResources func(string) (*http.Response, error)
+ fakeExecuteBundles func(storePath string, bundle []byte) (*http.Response, error)
+}
+
+func (c *fakeFhirStoreClient) executeBundle(storePath string, bundle []byte) (*http.Response, error) {
+ return c.fakeExecuteBundles(storePath, bundle)
+}
+
+func (c *fakeFhirStoreClient) readResource(resourcePath string) (*http.Response, error) {
+ return c.fakeReadResources(resourcePath)
+}
+
+// Useful to fake the Body of a http.Response.
+type fakeReaderCloser struct {
+ fakeRead func([]byte) (int, error)
+}
+
+func (*fakeReaderCloser) Close() error {
+ return nil
+}
+
+func (m *fakeReaderCloser) Read(b []byte) (int, error) {
+ return m.fakeRead(b)
+}
+
+func validateResourceErrorCounter(t *testing.T, pipelineResult beam.PipelineResult, expectedCount int) {
Review Comment:
This method should not performa the t.Fatalf but return an error back to the testing function that can then check for error and display appropriate message within the tests context. Additionally if you were to keep this helper util you would likely want to call t.Helper().
##########
sdks/go/pkg/beam/io/fhirio/execute_bundles.go:
##########
@@ -0,0 +1,146 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fhirio provides an API for reading and writing resources to Google
+// Cloud Healthcare Fhir stores.
+// Experimental.
+package fhirio
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "net/http"
+ "strings"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/register"
+)
+
+const (
+ bundleResponseTypeBatch = "batch-response"
+ bundleResponseTypeTransaction = "transaction-response"
+)
+
+func init() {
+ register.DoFn4x0[context.Context, []byte, func(string), func(string)]((*executeBundlesFn)(nil))
+ register.Emitter1[string]()
+}
+
+type executeBundlesFn struct {
Review Comment:
nit: Should this be executeBundlesFn or just executeBundleFn, I think each item in the PCollection is an individual FHIR bundle?
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -192,12 +204,29 @@ func TestFhirIO_InvalidRead(t *testing.T) {
passert.Count(s, failedReads, "", 1)
passert.Empty(s, resources)
passert.True(s, failedReads, func(errorMsg string) bool {
- return strings.Contains(errorMsg, "bad status [404]")
+ return strings.Contains(errorMsg, "404")
Review Comment:
Could we use the net/http constants here instead of hard coding "404"
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -192,12 +204,29 @@ func TestFhirIO_InvalidRead(t *testing.T) {
passert.Count(s, failedReads, "", 1)
passert.Empty(s, resources)
passert.True(s, failedReads, func(errorMsg string) bool {
- return strings.Contains(errorMsg, "bad status [404]")
+ return strings.Contains(errorMsg, "404")
})
ptest.RunAndValidate(t, p)
}
+func TestFhirIO_ExecuteBundles(t *testing.T) {
+ integration.CheckFilters(t)
+ checkFlags(t)
+
+ fhirStorePath, teardownFhirStore := setupEmptyFhirStore(t)
+ defer teardownFhirStore()
+
+ p, s, bundles := ptest.CreateList(readPrettyBundles())
+ successBodies, failures := fhirio.ExecuteBundles(s, fhirStorePath, bundles)
+ passert.Count(s, successBodies, "", 2)
+ passert.Count(s, failures, "", 2)
+ passert.True(s, failures, func(errorMsg string) bool {
+ return strings.Contains(errorMsg, "400")
Review Comment:
Same comment as above.
##########
sdks/go/pkg/beam/io/fhirio/execute_bundles_test.go:
##########
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fhirio
+
+import (
+ "bytes"
+ "net/http"
+ "strings"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/passert"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest"
+)
+
+func TestExecuteBundles(t *testing.T) {
Review Comment:
Could we add another test cases for ExecuteBundles that is successful
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -75,13 +84,16 @@ func setupFhirStore(t *testing.T) (string, []string, func()) {
healthcareDataset := fmt.Sprintf(datasetPathFmt, *gcpopts.Project, *gcpopts.Region)
Review Comment:
Is this the standard way to define a dataset for Apache beam? Would we ever want to potentially have different datasets for different tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on a diff in pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on code in PR #21840:
URL: https://github.com/apache/beam/pull/21840#discussion_r897345897
##########
sdks/go/pkg/beam/io/fhirio/execute_bundles.go:
##########
@@ -0,0 +1,146 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fhirio provides an API for reading and writing resources to Google
+// Cloud Healthcare Fhir stores.
+// Experimental.
+package fhirio
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "net/http"
+ "strings"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/register"
+)
+
+const (
+ bundleResponseTypeBatch = "batch-response"
+ bundleResponseTypeTransaction = "transaction-response"
+)
+
+func init() {
+ register.DoFn4x0[context.Context, []byte, func(string), func(string)]((*executeBundlesFn)(nil))
+ register.Emitter1[string]()
+}
+
+type executeBundlesFn struct {
Review Comment:
Good point. Although the equivalent class in the Java SDK is called `ExecuteBundlesFn`, I think that `executeBundleFn` would be more appropriate in this case, I guess it won't harm to make this naming improvement.
##########
sdks/go/pkg/beam/io/fhirio/execute_bundles_test.go:
##########
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fhirio
+
+import (
+ "bytes"
+ "net/http"
+ "strings"
+ "testing"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/passert"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest"
+)
+
+func TestExecuteBundles(t *testing.T) {
Review Comment:
The successful scenario is being tested as part of the integration test. Milena had suggested to only unit test the error scenarios since we'd have to mock too many things to be able to bypass all error checks and reach success. That's what we do in the Java SDK.
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -75,13 +84,16 @@ func setupFhirStore(t *testing.T) (string, []string, func()) {
healthcareDataset := fmt.Sprintf(datasetPathFmt, *gcpopts.Project, *gcpopts.Region)
Review Comment:
I don't think it would ever be necessary to have a different dataset for different tests since we don't really care about the datasets, only the stores themselves matter here.
Did you have a particular way to define it in mind?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lnogueir commented on pull request #21840: Add ExecuteBundles transform to Go FhirIO
Posted by GitBox <gi...@apache.org>.
lnogueir commented on PR #21840:
URL: https://github.com/apache/beam/pull/21840#issuecomment-1155745488
Run Go PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org