(beam) branch master updated: Document dynamic destination key should be unique over the pipeline (#32338)

[yh...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new 815a049b1bd Document dynamic destination key should be unique over the pipeline (#32338)
815a049b1bd is described below

commit 815a049b1bd184e6b8393dfd545f123f77496317
Author: Yi Hu <ya...@google.com>
AuthorDate: Wed Aug 28 11:27:02 2024 -0400

    Document dynamic destination key should be unique over the pipeline (#32338)
    
    * Document dynamic destination key should be unique over the pipeline
    
    * address comments
---
 .../main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java  | 7 ++++++-
 .../org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinations.java   | 4 ++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
index 71302339af1..ea3976946ff 100644
--- a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
+++ b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
@@ -3995,7 +3995,12 @@ public class BigQueryIO {
     }
   }
 
-  /** Clear the cached map of created tables. Used for testing. */
+  /**
+   * Clear the cached map of created tables. Used for testing only.
+   *
+   * <p>Should not be used in any PTransform as cache is a static member shared by different
+   * BigQueryIO.write.
+   */
   @VisibleForTesting
   static void clearStaticCaches() throws ExecutionException, InterruptedException {
     CreateTables.clearCreatedTables();
diff --git a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinations.java b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinations.java
index e5cf82d7c2e..105da60c75b 100644
--- a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinations.java
+++ b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinations.java
@@ -132,6 +132,10 @@ public abstract class DynamicDestinations<T, DestinationT> implements Serializab
   /**
    * Returns an object that represents at a high level which table is being written to. May not
    * return null.
+   *
+   * <p>The method must return a unique object for different destination tables involved over all
+   * BigQueryIO write transforms in the same pipeline. See
+   * https://github.com/apache/beam/issues/32335 for details.
    */
   public abstract DestinationT getDestination(@Nullable ValueInSingleWindow<T> element);