You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "bvolpato (via GitHub)" <gi...@apache.org> on 2026/09/03 19:14:46 UTC
[PR] [Java] Preserve BigQuery failed rows across retries [beam]
bvolpato opened a new pull request, #39999:
URL: https://github.com/apache/beam/pull/39999
Consecutive BigQuery Storage Write API row errors can duplicate a previously rejected record in Beam's failure output and omit the newly rejected record. After a serialization error, both Java writers compact the retry payload and timestamps but retain the original `failsafeTableRows` list. A later error index is then applied to the wrong version of that list.
For a write using `withFormatRecordOnFailureFunction`:
| Append request | Simulated service response | Expected failed original row | Current Beam output |
| --- | --- | --- | --- |
| `[A, B, C]` | Reject index 0 | A | A |
| `[B, C]` | Reject index 0 | B | A |
| `[C]` | Success | — | — |
C is written successfully. B is neither written nor recoverable from the failure output. The [Storage Write API contract](https://docs.cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1#rowerror) defines error indexes relative to the current request. This is a controlled reproduction with simulated responses, not a reproduced live BigQuery incident.
Filter failsafe rows alongside payloads and timestamps in both the sharded and unsharded writers. This preserves their positional correspondence through successive retries, including nullable entries used for the protobuf-to-TableRow fallback.
## Reproduction and testing
The new `StorageApiWriteRetryTest` runs public `BigQueryIO` pipelines with the DirectRunner and a fake append service. It uses a failure formatter that returns `original_id` records distinct from the serialized rows, rejects one row per append, then delegates the successful append to `FakeDatasetService`. Assertions cover the failed records, successful output, persisted rows, and the three shrinking append sizes.
```sh
./gradlew :sdks:java:io:google-cloud-platform:test \
--tests org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteRetryTest
```
The same regression fails in all three modes against unchanged upstream writer classes and passes with this fix: batch `STORAGE_WRITE_API`, streaming `STORAGE_WRITE_API` with a fixed shard, and `STORAGE_API_AT_LEAST_ONCE`.
The branch passes the normal Gradle compiler checks and a combined test run with the two existing `BigQueryIOWriteTest` cases `testStorageWriteReturnsAppendSerializationError` and `testStorageWriteWithMultipleAppendsPerStream`: 8 passed, 5 inapplicable parameter combinations skipped. `spotlessJavaCheck` and `git diff --check` also pass. The baseline comparison separately compiles the unchanged writer classes with the same regression against released Beam 2.76 dependencies.
## Downsides
Each serialization-error retry allocates one additional list of references to the surviving failsafe rows. The normal append path is unchanged; the error-path allocation is proportional to the batch being retried and does not copy the row objects.
------------------------
- [x] Describe the bug and include a reproducible regression test.
- [x] Update `CHANGES.md` with the behavior change.
- [ ] Apache Individual Contributor License Agreement, if required for this contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java] Preserve BigQuery failed rows across retries [beam]
Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax merged PR #39999:
URL: https://github.com/apache/beam/pull/39999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java] Preserve BigQuery failed rows across retries [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39999:
URL: https://github.com/apache/beam/pull/39999#issuecomment-6374865411
Assigning reviewers:
R: @kennknowles for label java.
Backup reviewer: ahmedabu98 (if the main reviewer is unresponsive, you can escalate to them by tagging their username)
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java] Preserve BigQuery failed rows across retries [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39999:
URL: https://github.com/apache/beam/pull/39999#issuecomment-6367012554
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java] Preserve BigQuery failed rows across retries [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39999:
URL: https://github.com/apache/beam/pull/39999#issuecomment-6449137555
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Java] Preserve BigQuery failed rows across retries [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #39999:
URL: https://github.com/apache/beam/pull/39999#issuecomment-6449112622
R: @reuvenlax
I think you know this stuff best
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org