You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2023/01/03 14:31:36 UTC

[GitHub] [beam] JozoVilcek commented on a diff in pull request #24837: Enable async processing for SDF on Spark runner #23852

JozoVilcek commented on code in PR #24837:
URL: https://github.com/apache/beam/pull/24837#discussion_r1060641029


##########
runners/spark/src/main/java/org/apache/beam/runners/spark/SparkCommonPipelineOptions.java:
##########
@@ -84,4 +84,11 @@ static void prepareFilesToStage(SparkCommonPipelineOptions options) {
       PipelineResources.prepareFilesForStaging(options);
     }
   }
+
+  @Description(
+      "Enable/disable async output for operators with possibly large output ( such as splittable DoFn )")
+  @Default.Boolean(true)
+  Boolean getEnableAsyncOperatorOutput();

Review Comment:
   Yes, you are correct. I will think about the better name. However, I wonder, does it make sense to have this parameter at all. When implemented correctly, why would we want to turn it off.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org