You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Jorge Esteban Quilcate Otoya (Jira)" <ji...@apache.org> on 2023/07/12 13:10:00 UTC

[jira] [Created] (KAFKA-15181) Race condition on partition assigned to TopicBasedRemoteLogMetadataManager

Jorge Esteban Quilcate Otoya created KAFKA-15181:
----------------------------------------------------

             Summary: Race condition on partition assigned to TopicBasedRemoteLogMetadataManager 
                 Key: KAFKA-15181
                 URL: https://issues.apache.org/jira/browse/KAFKA-15181
             Project: Kafka
          Issue Type: Bug
          Components: core
            Reporter: Jorge Esteban Quilcate Otoya
            Assignee: Jorge Esteban Quilcate Otoya


TopicBasedRemoteLogMetadataManager (TBRLMM) uses a cache to be prepared whever partitions are assigned.

When partitions are assigned to the TBRLMM instance, a consumer is started to keep the cache up to date.

If the cache hasn't finalized to build, TBRLMM fails to return remote metadata about partitions that are store on the backing topic. TBRLMM may not recover from this failing state.

A proposal to fix this issue would be wait after a partition is assigned for the consumer to catch up. A similar logic is used at the moment when TBRLMM writes to the topic, and uses send callback to wait for consumer to catch up. This logic can be reused whever a partition is assigned, so when TBRLMM is marked as initialized, cache is ready to serve requests.


Reference: https://github.com/aiven/kafka/issues/33



--
This message was sent by Atlassian Jira
(v8.20.10#820010)