You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "David Arthur (Jira)" <ji...@apache.org> on 2023/11/14 18:31:00 UTC

[jira] [Created] (KAFKA-15825) KRaft controller writes empty state to ZK after migration

David Arthur created KAFKA-15825:
------------------------------------

             Summary: KRaft controller writes empty state to ZK after migration
                 Key: KAFKA-15825
                 URL: https://issues.apache.org/jira/browse/KAFKA-15825
             Project: Kafka
          Issue Type: Bug
          Components: controller
    Affects Versions: 3.6.0
            Reporter: David Arthur
            Assignee: David Arthur
             Fix For: 3.7.0, 3.6.1


Immediately following the ZK migration, there is a race condition where the KRaftMigrationDriver can use an empty MetadataImage when performing the full "SYNC_KRAFT_TO_ZK" reconciliation. 

After the next controller failover, or when the controller loads a metadata snapshot, the correct state will be written to ZK. 

The symptom of this bug is that we see the migration complete, and then all the metadata removed from ZK. For example, 

{code}
[KRaftMigrationDriver id=9990] Completed migration of metadata from ZooKeeper to KRaft. 573 records were generated in 2204 ms across 51 batches. The record types were {TOPIC_RECORD=41, PARTITION_RECORD=410, CONFIG_RECORD=121, PRODUCER_IDS_RECORD=1}. The current metadata offset is now 503794 with an epoch of 21. Saw 6 brokers in the migrated metadata [0, 1, 2, 3, 4, 5].
{code}

immediately followed by:

{code}
[KRaftMigrationDriver id=9990] Made the following ZK writes when reconciling with KRaft state: {DeleteBrokerConfig=7, DeleteTopic=41, UpdateTopicConfig=41}
{code}

If affected by this, a quick workaround is to cause the controller to failover.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)