You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "tkaymak (via GitHub)" <gi...@apache.org> on 2026/09/01 09:20:35 UTC

Re: [PR] [#36841] Add streaming dispatch seam to the Spark Structured Streaming runner [beam]

tkaymak commented on code in PR #39906:
URL: https://github.com/apache/beam/pull/39906#discussion_r3902658762


##########
runners/spark/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingPipelineOptions.java:
##########
@@ -39,4 +39,38 @@ public interface SparkStructuredStreamingPipelineOptions extends SparkCommonPipe
   boolean getUseActiveSparkSession();
 
   void setUseActiveSparkSession(boolean value);
+
+  @Description(
+      "Watermark delay in milliseconds applied to event timestamps of streaming sources "
+          + "(streaming mode only).")
+  @Default.Long(0)
+  long getWatermarkDelayMillis();
+
+  void setWatermarkDelayMillis(long value);
+
+  // Note: deliberately NOT named getMaxRecordsPerBatch. The legacy Spark runner's
+  // SparkPipelineOptions already declares Long getMaxRecordsPerBatch(); a same-name getter with a
+  // different return type breaks proxy generation for every registered PipelineOptions interface.
+  @Description(

Review Comment:
   Thank you, will take care of this!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org