[GitHub] [kafka] cadonna commented on pull request #11765: [WIP] State updater implementation

[GitBox <gi...@apache.org>]

cadonna commented on pull request #11765:
URL: https://github.com/apache/kafka/pull/11765#issuecomment-1040246456


   @guozhangwang Here is my code for putting restoration on a separate thread so far.
   The idea that I had is to give a reference to the state updater to the `Tasks` container. The `Tasks` container uses this reference to retrieve all active tasks that are in restoration and all standby tasks. That means, methods like `allTasks()` need to be modified to get their result from the state updater and the other data structures in the `Tasks` container. An important invariant is that a task cannot exist in the state updater and a data structure other than the state updater in the `Tasks` container at the same time. A task is either in one or the other. A typical lifecycle of an active task would then be: creation -> added to state updater ->  restoration is done -> removed from state updater (by calling `getRestoredActiveTasks()`) -> added to the appropriate data structure in the `Tasks` container. Standby tasks are created and put in the state updater and stay there. Tasks are recycled (i.e. transformed from active to standby or vice versa) in the state updater.
   
   ToDos:
   - The unit tests of the state updater are failing.
   - The code to recycle tasks are not yet in the state updater
   - The modifications to the `Tasks` container are still missing
   - Integration of the state updater in the normal processing.
   
   The first two points can be done in this PR. The third point should be done in a separate PR since this is already quite big. The last point is also a separate PR.   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org