You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Jose Armando Garcia Sancio (Jira)" <ji...@apache.org> on 2021/07/21 02:24:00 UTC

[jira] [Updated] (KAFKA-13112) Controller's committed offset get out of sync with raft client listener context

     [ https://issues.apache.org/jira/browse/KAFKA-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jose Armando Garcia Sancio updated KAFKA-13112:
-----------------------------------------------
    Description: 
The active controller creates an in-memory snapshot for every offset returned by RaftClient::scheduleAppend and RaftClient::scheduleAtomicAppend. For RaftClient::scheduleAppend, the RaftClient is free to split those records into multiple batches. Because of this when scheduleAppend is use there is no guarantee that the active leader will always have an in-memory snapshot for every "last committed offset".

To get around this problem, when the active controller renounces from leader if there is no snapshot at the last committed offset it will instead.
 # Reset the snapshot registry
 # Unregister the listener from the RaftClient
 # Register a new listener with the RaftClient

> Controller's committed offset get out of sync with raft client listener context
> -------------------------------------------------------------------------------
>
>                 Key: KAFKA-13112
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13112
>             Project: Kafka
>          Issue Type: Bug
>          Components: controller, kraft
>            Reporter: Jose Armando Garcia Sancio
>            Assignee: Jose Armando Garcia Sancio
>            Priority: Blocker
>              Labels: kip-500
>             Fix For: 3.0.0
>
>
> The active controller creates an in-memory snapshot for every offset returned by RaftClient::scheduleAppend and RaftClient::scheduleAtomicAppend. For RaftClient::scheduleAppend, the RaftClient is free to split those records into multiple batches. Because of this when scheduleAppend is use there is no guarantee that the active leader will always have an in-memory snapshot for every "last committed offset".
> To get around this problem, when the active controller renounces from leader if there is no snapshot at the last committed offset it will instead.
>  # Reset the snapshot registry
>  # Unregister the listener from the RaftClient
>  # Register a new listener with the RaftClient



--
This message was sent by Atlassian Jira
(v8.3.4#803005)