[jira] [Comment Edited] (KAFKA-21312) Snapshottable tasks in the coordinator runtime

["sanghyeok An (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129022#comment-18129022 ] 

sanghyeok An edited comment on KAFKA-21312 at 10/20/26 8:06 AM:
----------------------------------------------------------------

[~squah-confluent] 

Ah, thank you for letting me know!

Let me first share the implementation approach I had in mind.

I was considering extending {{{}CoordinatorTimerImpl{}}}. So far, I've only thought through the overall flow, and I was planning to validate its feasibility through a small PoC before working out the implementation details.
 # I was planning to add snapshot-related fields to {{{}CoordinatorTimerImpl{}}}. Basically, the object to be snapshotted would contain only a deadline value.
 # When the Coordinator calls a method such as {{snapshotWithCancel()}} on {{{}CoordinatorTimerImpl{}}}, the timer currently scheduled in the Timing Wheel would be canceled, and its deadline would be stored in a snapshot object registered with the {{{}SnapshotRegistry{}}}.
 # If the Coordinator successfully persists the records, it would proceed as usual. However, if record persistence fails, I was thinking of calling something like a {{revert()}} method on the timer to restore its previous snapshot, similar to how {{TimelineHashMap}} handles rollback. (I don't remember the exact method name, so this is just a conceptual description.)
 # At this point, I expect that rescheduling the {{TimerTask}} restored to the previous epoch in the Timing Wheel with its original deadline would allow the subsequent processing to follow a flow similar to the existing one.

This is roughly the approach I have in mind. 
What do you think? Thanks for reading this!


was (Author: JIRAUSER303328):
[~squah-confluent] 

Ah, thank you for letting me know!

Let me first share the implementation approach I had in mind.

I was considering extending {{{}CoordinatorTimerImpl{}}}. So far, I've only thought through the overall flow, and I was planning to validate its feasibility through a small PoC before working out the implementation details.
 # I was planning to add snapshot-related fields to {{{}CoordinatorTimerImpl{}}}. Basically, the object to be snapshotted would contain only a deadline value.

 # When the Coordinator calls a method such as {{snapshotWithCancel()}} on {{{}CoordinatorTimerImpl{}}}, the timer currently scheduled in the Timing Wheel would be canceled, and its deadline would be stored in a snapshot object registered with the {{{}SnapshotRegistry{}}}.

 # If the Coordinator successfully persists the records, it would proceed as usual. However, if record persistence fails, I was thinking of calling something like a {{revert()}} method on the timer to restore its previous snapshot, similar to how {{TimelineHashMap}} handles rollback. (I don't remember the exact method name, so this is just a conceptual description.)

 # At this point, I expect that rescheduling the {{TimerTask}} restored to the previous epoch in the Timing Wheel with its original deadline would allow the subsequent processing to follow a flow similar to the existing one.

This is roughly the approach I have in mind. 
What do you think? Thanks for reading this!

> Snapshottable tasks in the coordinator runtime
> ----------------------------------------------
>
>                 Key: KAFKA-21312
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21312
>             Project: Kafka
>          Issue Type: Improvement
>          Components: group-coordinator
>            Reporter: Sean Quah
>            Assignee: Sean Quah
>            Priority: Major
>
> In a couple of places, the coordinator runtime and group coordinator create and destroy tasks, eg. for regex resolution and (soon) asynchronous assignment calculations.
> When coordinator state is rolled back, these tasks are not reverted and, when they complete, we have to think carefully about how we can discard their results if they no longer apply. It would be nice to have these tasks participate in the snapshot mechanism.
> Analogous to KAFKA-19023 but for the CoordinatorExecutor.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)