[GitHub] [flink] aljoscha commented on a change in pull request #8885: [FLINK-12855] [streaming-java][window-assigners] Add functionality that staggers panes on partitions to distribute workload.

[GitBox <gi...@apache.org>]

aljoscha commented on a change in pull request #8885:
URL: https://github.com/apache/flink/pull/8885#discussion_r429186529



##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java
##########
@@ -46,28 +46,44 @@
 
 	private final long size;
 
-	private final long offset;
+	private final long globalOffset;
 
-	private TumblingProcessingTimeWindows(long size, long offset) {
+	private Long staggerOffset = null;
+
+	private final WindowStagger windowStagger;
+
+	private TumblingProcessingTimeWindows(long size, long offset, WindowStagger windowStagger) {
 		if (Math.abs(offset) >= size) {
 			throw new IllegalArgumentException("TumblingProcessingTimeWindows parameters must satisfy abs(offset) < size");
 		}
-
 		this.size = size;
-		this.offset = offset;
+		this.globalOffset = offset;
+		this.windowStagger = windowStagger;
 	}
 
 	@Override
 	public Collection<TimeWindow> assignWindows(Object element, long timestamp, WindowAssignerContext context) {
 		final long now = context.getCurrentProcessingTime();
-		long start = TimeWindow.getWindowStartWithOffset(now, offset, size);
+
+		if (staggerOffset == null) {
+			staggerOffset = windowStagger.getStaggerOffset(context.getCurrentProcessingTime(), size);

Review comment:
       Does `WindowStagger.NATURAL` make sense if we keep the stagger offset fixed per
    `WindowAssigner` instance? I don't know if this was the intention but that's what
    the code does.

##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java
##########
@@ -107,7 +123,48 @@ public static TumblingProcessingTimeWindows of(Time size) {
 	 * @return The time policy.
 	 */
 	public static TumblingProcessingTimeWindows of(Time size, Time offset) {
-		return new TumblingProcessingTimeWindows(size.toMilliseconds(), offset.toMilliseconds());
+		return new TumblingProcessingTimeWindows(size.toMilliseconds(), offset.toMilliseconds(), WindowStagger.ALIGNED);
+	}
+
+	/**
+	 * Creates a new {@code TumblingProcessingTimeWindows} {@link WindowAssigner} that assigns
+	 * elements to time windows based on the element timestamp, offset and a staggering offset sampled
+	 * from uniform distribution(0, window size) for each pane.
+	 *
+	 * @param size The size of the generated windows.
+	 * @param offset The offset which window start would be shifted by.
+	 * @param windowStagger The utility that produces staggering offset in runtime.
+	 *
+	 * @return The time policy.
+	 */
+	public static TumblingProcessingTimeWindows of(Time size, Time offset, WindowStagger windowStagger) throws Exception {
+		return new TumblingProcessingTimeWindows(size.toMilliseconds(), offset.toMilliseconds(), windowStagger);
+	}
+
+	/**
+	 * Creates a new {@code TumblingProcessingTimeWindows} {@link WindowAssigner} that assigns
+	 * elements to time windows based on the element timestamp and a staggering offset sampled
+	 * from uniform distribution(0, window size) for each pane.
+	 *
+	 * @param size The size of the generated windows.
+	 * @return The time policy.
+	 */
+	public static TumblingProcessingTimeWindows withStaggerOf(Time size) {

Review comment:
       I would remove these additional methods, as they are they are quite confusing because `withStaggerOf(13)` does not actually set a stagger offset of `13` but a window size of `13`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org