[GitHub] [flink] aljoscha commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode

[GitBox <gi...@apache.org>]

aljoscha commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719457511


   Thanks for opening the PR! However, this will not work in the current state. We only know if we use `BATCH` or `STREAMING` execution mode when we're translating the `Transformations` to a `StreamGraph`. Doing the check right in `assignTimestampsAndWatermarks()` will not work because we don't yet know if the job contains only bounded sources or not.
   
   This issue is slightly more complicated but not by much. The steps would be:
   1. Turn `assignTimestampsAndWatermarks()` into a logical operation. This means adding a `TimestampsAndWatermarksTransformation` that has a `WatermarkStrategy` as a field. Use this instead of creating an operator directly in `assignTimestampsAndWatermarks()`
   2. add a `TransformationTranslator` that will be similar to `SourceTransformationTranslator` or `OneInputTransformationTranslator` that translates the `TimestampsAndWatermarksTransformation` to an actual operator. In there, you will now know if we're in `BATCH` or `STREAMING` mode and can create the correct operator with either progressive watermarks or not.
   
   Does that make sense?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org