[GitHub] [beam] mxm commented on a change in pull request #12733: [BEAM-10760] Optimize state cleanup for global window in portable Flink runner

[GitBox <gi...@apache.org>]

mxm commented on a change in pull request #12733:
URL: https://github.com/apache/beam/pull/12733#discussion_r480015675



##########
File path: runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ExecutableStageDoFnOperator.java
##########
@@ -592,12 +593,37 @@ public long applyOutputWatermarkHold(long currentOutputWatermark, long potential
         return currentOutputWatermark;
       }
     } else {
+      // Check if the final watermark was triggered to perform state cleanup for global window
+      if (potentialOutputWatermark == Long.MAX_VALUE
+          && currentOutputWatermark < Long.MAX_VALUE
+          && isStateful) {
+        cleanupGlobalWindowState();
+      }

Review comment:
       ```suggestion
         // Check if the final watermark was triggered to perform state cleanup for global window
         if (potentialOutputWatermark > BoundedWindow.maxTimestamp().getMillis()
             && currentOutputWatermark <= BoundedWindow.maxTimestamp().getMillis()) {
           cleanupGlobalWindowState();
         }
   ```
   
   Would change the condition to specifically target the end of the global window. There is also no need to include the statefulness in the condition because if there is no state, it also won't be cleaned up.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org