You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2024/03/11 20:08:53 UTC
(beam) branch users/damccorm/v1-override created (now 0035692accd)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a change to branch users/damccorm/v1-override
in repository https://gitbox.apache.org/repos/asf/beam.git
at 0035692accd Skip replacing v1 transforms for runner v2 pipelines
This branch includes the following new commits:
new 0035692accd Skip replacing v1 transforms for runner v2 pipelines
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(beam) 01/01: Skip replacing v1 transforms for runner v2 pipelines
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/v1-override
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 0035692accdc3c4b8068c65e25a9ccfea9e1dae1
Author: Danny McCormick <da...@google.com>
AuthorDate: Mon Mar 11 16:08:47 2024 -0400
Skip replacing v1 transforms for runner v2 pipelines
---
.../main/java/org/apache/beam/runners/dataflow/DataflowRunner.java | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
index d7f9ba61442..af26adc61bc 100644
--- a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
+++ b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
@@ -1225,9 +1225,11 @@ public class DataflowRunner extends PipelineRunner<DataflowPipelineJob> {
DataflowPackage stagedPipeline =
options.getStager().stageToFile(serializedProtoPipeline, PIPELINE_FILE_NAME);
dataflowOptions.setPipelineUrl(stagedPipeline.getLocation());
- // Now rewrite things to be as needed for v1 (mutates the pipeline)
+ // Now rewrite things to be as needed for v1 (mutates the pipeline) if using v1
// This way the job submitted is valid for v1 and v2, simultaneously
- replaceV1Transforms(pipeline);
+ if (!useUnifiedWorker(options)) {
+ replaceV1Transforms(pipeline);
+ }
// Capture the SdkComponents for look up during step translations
SdkComponents dataflowV1Components = SdkComponents.create();
dataflowV1Components.registerEnvironment(