You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "khaledh (via GitHub)" <gi...@apache.org> on 2026/09/21 16:18:06 UTC
[PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
khaledh opened a new pull request, #40204:
URL: https://github.com/apache/beam/pull/40204
fixes #40189
When `allowMissingRequiredFields` is true, conversion already skips the missing-required check, but both build sites still call `build()`, which re-validates the whole tree and can throw. This makes the flag consistent at build time:
```java
return allowMissingRequiredFields ? builder.buildPartial() : builder.build();
```
- `false` path unchanged; a nested test pins the exact `SchemaMissingRequiredFieldException` diagnostic.
- In-tree callers passing `true` already use all-optional descriptors: identical output, one redundant `isInitialized()` walk removed.
- Behavior change, declared in `CHANGES.md`: an external caller passing `true` with a strict descriptor now gets a partial message instead of `UninitializedMessageException`. A one-time descriptor check can be added if preferred.
- Dataflow, production workload: −12% CPU/record alone; −34% CPU and +46% throughput combined with #40188.
Tests: on/off parity for all-optional descriptors including unknown fields, strict-path failure unchanged, nested diagnostic chain, partial messages for map and positional rows. GCP IO suite green on JDK 11 and 17.
Related: #40188 (independent; either order).
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [x] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "khaledh (via GitHub)" <gi...@apache.org>.
khaledh commented on code in PR #40204:
URL: https://github.com/apache/beam/pull/40204#discussion_r4339699407
##########
CHANGES.md:
##########
@@ -72,6 +72,7 @@
## Breaking Changes
+* (Java) BigQuery Storage Write API `TableRow` conversion now honors `allowMissingRequiredFields` during protocol-buffer construction. Callers that pass `true` with required descriptors receive partial messages instead of conversion-time `UninitializedMessageException` failures ([#40189](https://github.com/apache/beam/issues/40189)).
Review Comment:
Moved under Bugfixes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40204:
URL: https://github.com/apache/beam/pull/40204#issuecomment-5798010033
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40204:
URL: https://github.com/apache/beam/pull/40204#issuecomment-5765673046
Assigning reviewers:
R: @kennknowles for label java.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #40204:
URL: https://github.com/apache/beam/pull/40204#issuecomment-6353880960
Thanks @khaledh!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #40204:
URL: https://github.com/apache/beam/pull/40204#discussion_r4331554916
##########
CHANGES.md:
##########
@@ -72,6 +72,7 @@
## Breaking Changes
+* (Java) BigQuery Storage Write API `TableRow` conversion now honors `allowMissingRequiredFields` during protocol-buffer construction. Callers that pass `true` with required descriptors receive partial messages instead of conversion-time `UninitializedMessageException` failures ([#40189](https://github.com/apache/beam/issues/40189)).
Review Comment:
This isn't really a breaking change right? Can you add it under `## Bugfixes` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #40204:
URL: https://github.com/apache/beam/pull/40204#issuecomment-5798000586
R: @ahmedabu98
Sorry to send so many to you, but am I right that you are the current expert in the storage write API paths?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java][BigQueryIO] Honor allowMissingRequiredFields when building Storage Write API messages [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 merged PR #40204:
URL: https://github.com/apache/beam/pull/40204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org