[GitHub] [beam] igorbernstein2 commented on a diff in pull request #17823: [BEAM-14554] Create setting to report Bigtable client throttling time to Dataflow

[GitBox <gi...@apache.org>]

igorbernstein2 commented on code in PR #17823:
URL: https://github.com/apache/beam/pull/17823#discussion_r899279150


##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -844,6 +874,22 @@ public void processElement(ProcessContext c, BoundedWindow window) throws Except
                   failures.add(new BigtableWriteException(c.element(), exception));
                 }
               });
+      if (config.getDataflowThrottleReporting()) {
+        long delta = 0;
+        ResourceLimiterStats stats =
+            ResourceLimiterStats.getInstance(
+                new BigtableInstanceName(
+                    config.getProjectId().get(), config.getInstanceId().get()));

Review Comment:
   Please cache this in the constructor



##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -739,6 +754,16 @@ public Write withEmulator(String emulatorHost) {
       return toBuilder().setBigtableConfig(config.withEmulator(emulatorHost)).build();
     }
 
+    /* This is an experimental feature that may get changed in the future
+     *
+     * Returns a new {@link BigtableIO.Write} that will report amount of time throttling to Dataflow
+     */
+    @Experimental
+    public Write withDataflowThrottleReporting() {

Review Comment:
   I would add a bool param here. It will allow you to transition this to be opt out later



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org