You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by re...@apache.org on 2023/01/20 07:53:41 UTC
[beam] 01/01: Merge pull request #24145: Handle updates to table schema when using Storage API writes.
This is an automated email from the ASF dual-hosted git repository.
reuvenlax pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit f5020e7ac2bdd3363619aafbf257f2ebf8e3fe2b
Merge: 428ec97e30c 7ad44c84585
Author: Reuven Lax <re...@google.com>
AuthorDate: Thu Jan 19 23:53:33 2023 -0800
Merge pull request #24145: Handle updates to table schema when using Storage API writes.
.../beam/sdk/io/gcp/bigquery/AppendClientInfo.java | 115 +++++++++--
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 18 +-
.../beam/sdk/io/gcp/bigquery/BigQueryServices.java | 8 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 11 ++
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 29 ---
.../sdk/io/gcp/bigquery/SplittingIterable.java | 61 +++++-
.../bigquery/StorageApiDynamicDestinations.java | 2 +
.../StorageApiDynamicDestinationsBeamRow.java | 8 +-
.../StorageApiDynamicDestinationsTableRow.java | 24 ++-
.../beam/sdk/io/gcp/bigquery/StorageApiLoads.java | 20 +-
.../io/gcp/bigquery/StorageApiWritePayload.java | 25 ++-
.../StorageApiWriteRecordsInconsistent.java | 12 +-
.../bigquery/StorageApiWriteUnshardedRecords.java | 146 +++++++++++---
.../bigquery/StorageApiWritesShardedRecords.java | 173 +++++++++++------
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 110 +++++++++--
.../sdk/io/gcp/testing/FakeDatasetService.java | 63 +++++-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 216 +++++++++++++++++++++
.../bigquery/TableRowToStorageApiProtoTest.java | 95 ++++++++-
18 files changed, 958 insertions(+), 178 deletions(-)