You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ri...@apache.org on 2023/05/25 14:50:44 UTC
[beam] branch cp-26880 created (now 00413bd9d75)
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a change to branch cp-26880
in repository https://gitbox.apache.org/repos/asf/beam.git
at 00413bd9d75 Fix failing BQ tests on XVR GoUsingJava tests (#26880)
This branch includes the following new commits:
new 00413bd9d75 Fix failing BQ tests on XVR GoUsingJava tests (#26880)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[beam] 01/01: Fix failing BQ tests on XVR GoUsingJava tests (#26880)
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a commit to branch cp-26880
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 00413bd9d75176899cfbb54f6d5f03e55a556778
Author: Ahmed Abualsaud <65...@users.noreply.github.com>
AuthorDate: Thu May 25 17:36:04 2023 +0300
Fix failing BQ tests on XVR GoUsingJava tests (#26880)
* only set triggering frequency and autosharding with unbounded input
* spotless
* use unbounded
---
.../beam/sdk/io/gcp/bigquery/BigQuerySchemaIOProvider.java | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySchemaIOProvider.java b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySchemaIOProvider.java
index 00b740c7f47..662fac3d767 100644
--- a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySchemaIOProvider.java
+++ b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySchemaIOProvider.java
@@ -197,12 +197,15 @@ public class BigQuerySchemaIOProvider implements SchemaIOProvider {
BigQueryIO.<Row>write()
.useBeamSchema()
.withMethod(BigQueryIO.Write.Method.STORAGE_WRITE_API)
- .withTriggeringFrequency(Duration.standardSeconds(5))
.withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND);
final Boolean autoSharding = config.getBoolean("autoSharding");
- if (autoSharding != null && autoSharding) {
- write = write.withAutoSharding();
+ // use default value true for autoSharding if not configured for STORAGE_WRITE_API
+ if (input.isBounded() == PCollection.IsBounded.UNBOUNDED) {
+ write = write.withTriggeringFrequency(Duration.standardSeconds(5));
+ if (autoSharding == null || autoSharding) {
+ write = write.withAutoSharding();
+ }
}
final Boolean useTestingBigQueryServices =