[GitHub] [beam] johnjcasey commented on a diff in pull request #28264: [Java BQ] Storage API streaming load test

["johnjcasey (via GitHub)" <gi...@apache.org>]

johnjcasey commented on code in PR #28264:
URL: https://github.com/apache/beam/pull/28264#discussion_r1333420835


##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:
##########
@@ -3266,6 +3268,8 @@ && getStorageApiTriggeringFrequency(bqOptions) != null) {
         checkArgument(
             !getAutoSchemaUpdate(),
             "withAutoSchemaUpdate only supported when using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE.");
+      } else if (getWriteDisposition() == WriteDisposition.WRITE_TRUNCATE) {

Review Comment:
   What happens if the user misconfigures this? Will it fail?



##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritesShardedRecords.java:
##########
@@ -247,7 +251,13 @@ public PCollectionTuple expand(
     PCollectionTuple writeRecordsResult =
         input.apply(
             "Write Records",
-            ParDo.of(new WriteRecordsDoFn(operationName, streamIdleTime, splitSize, maxRequestSize))
+            ParDo.of(

Review Comment:
   Is there some way we can do crash interval via mocking instead of including it in the main code? In general, I'd prefer to avoid including test config in our running code



##########
it/google-cloud-platform/build.gradle:
##########
@@ -73,9 +74,12 @@ dependencies {
     testImplementation project(path: ":sdks:java:io:google-cloud-platform")
     testImplementation project(path: ":sdks:java:io:synthetic")
     testImplementation library.java.mockito_inline
-    testRuntimeOnly project(path: ":runners:direct-java", configuration: "shadowTest")
+    testImplementation project(path: ":sdks:java:core", configuration: "shadowTest")

Review Comment:
   Why do we need these additional dependencies?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org