You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "mas-chen (via GitHub)" <gi...@apache.org> on 2023/06/27 23:11:25 UTC
[GitHub] [flink] mas-chen commented on a diff in pull request #22806: [FLINK-32362] [SourceAlignment] increase the robustness of announceCombinedWatermark to cover the case task failover
mas-chen commented on code in PR #22806:
URL: https://github.com/apache/flink/pull/22806#discussion_r1244463901
##########
flink-runtime/src/main/java/org/apache/flink/runtime/source/coordinator/SourceCoordinator.java:
##########
@@ -195,9 +195,19 @@ void announceCombinedWatermark() {
"Distributing maxAllowedWatermark={} to subTaskIds={}",
maxAllowedWatermark,
subTaskIds);
- for (Integer subtaskId : subTaskIds) {
- context.sendEventToSourceOperator(
- subtaskId, new WatermarkAlignmentEvent(maxAllowedWatermark));
+ // Because of Java-ThreadPoolExecutor will not schedule the period task
+ // if it throws an exception, so we should handle the potential exception like
+ // "subtask xx is not ready yet to receive events" to increase robustness.
+ try {
+ for (Integer subtaskId : subTaskIds) {
+ context.sendEventToSourceOperator(
+ subtaskId, new WatermarkAlignmentEvent(maxAllowedWatermark));
+ }
Review Comment:
Well, in the current diff, this logic may partially send events. If we want to make the entire operation atomic, we can check `context#registeredReaders()` or `SubtaskGateway` to check that all readers are available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org