[GitHub] [flink] sunhaibotb opened a new pull request #9804: [FLINK-14239] Fix the max watermark in StreamSource may arrive the downstream operator early

[GitBox <gi...@apache.org>]

sunhaibotb opened a new pull request #9804: [FLINK-14239] Fix the max watermark in StreamSource may arrive the downstream operator early
URL: https://github.com/apache/flink/pull/9804
 
 
   ## What is the purpose of the change
   
   This pull request fixes the problem that the max watermark in `StreamSource` may arrive the downstream operator early.
   
   For `Source`, the max watermark is emitted in `StreamSource#run currently`. If some records are also output in `close` of `RichSourceFunction`, then the max watermark will reach the downstream operator before these records.
   
   ## Brief change log
   
     - Modify `StreamSource` to move the logic of emitting the max watermark to `close` from the `run` method.
   
   ## Verifying this change
   
   This change modifies some existing tests and can be verified as follows:
   
     - When some records are output in the `close` of `RichSourceFunction`, the max watermark also will arrive the downstream operator after the last record.
     - Do not emit the max watermark when the task is cancelled asynchronously.
     - Do not emit the max watermark when the task is cancelled immediately after it is created.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / **no**)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
     - The serializers: (yes / **no** / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
     - The S3 file system connector: (yes / **no** / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / **no**)
     - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services