[jira] [Created] (FLINK-26029) Generalize the checkpoint protocol of OperatorCoordinator.

["Jiangjie Qin (Jira)" <ji...@apache.org>]

Jiangjie Qin created FLINK-26029:
------------------------------------

             Summary: Generalize the checkpoint protocol of OperatorCoordinator.
                 Key: FLINK-26029
                 URL: https://issues.apache.org/jira/browse/FLINK-26029
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Checkpointing
    Affects Versions: 1.14.3
            Reporter: Jiangjie Qin


Currently the JM opens all the event valves from the OperatorCoordinator to the subtasks after the checkpoint barriers are sent to the Source subtasks. While this works for the Source Operators, it unnecessarily limits general usage of the OperatorCoordinator for other operators.

To generalize the protocol, we can change the JM to open the event valve of the subtasks that have finished the local checkpoint. So the protocol would become following:
 # Let the OC finish processing all the incoming OperatorEvents before the snapshot.
 # Wait until all the outgoing OperatorEvents before the snapshot are sent and acked.
 # Shut the event valve so no outgoing events can be sent to the subtasks.
 # Send checkpoint barriers to the Source operators.
 # Open the corresponding event valve of a subtask when the AcknowledgeCheckpoint messages from that subtask is received. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)