You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Kamal Chandraprakash (Jira)" <ji...@apache.org> on 2024/11/10 08:48:00 UTC
[jira] [Updated] (KAFKA-17980) Add isReady API in RemoteLogMetadataManager
[ https://issues.apache.org/jira/browse/KAFKA-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kamal Chandraprakash updated KAFKA-17980:
-----------------------------------------
Summary: Add isReady API in RemoteLogMetadataManager (was: Add isReady API to RemoteLogMetadataManager)
> Add isReady API in RemoteLogMetadataManager
> -------------------------------------------
>
> Key: KAFKA-17980
> URL: https://issues.apache.org/jira/browse/KAFKA-17980
> Project: Kafka
> Issue Type: Task
> Reporter: Kamal Chandraprakash
> Assignee: Kamal Chandraprakash
> Priority: Major
>
> When a broker comes up online, the RemoteLogMetadataManager (RLMM) gets initialized in the background using the async fashion. The server starts to copy/delete the expired remote log segments, then the RLMM throws exception until it gets initialized. By adding the {{isReady}} method, the server can delay the copying (or) deleting the remote log segments until the RLMM gets initialized. This is a graceful handling to avoid exception/error logs while starting the server.
> (eg) Exception during init
> {noformat}
> [2024-11-06 09:33:16,867] WARN [RemoteLogManager=0 partition=xp3zuxhTQ9uUwk-uQhTx5g:topicA-0] Current task for topic-partition xp3zuxhTQ9uUwk-uQhTx5g:topicA-0 received error but it will be scheduled (kafka.log.remote.RemoteLogManager$RLMTask:793)
> java.lang.IllegalStateException: This instance is in invalid state, initialized: false close: false
> at org.apache.kafka.server.log.remote.metadata.storage.TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed(TopicBasedRemoteLogMetadataManager.java:557)
> at org.apache.kafka.server.log.remote.metadata.storage.TopicBasedRemoteLogMetadataManager.highestOffsetForEpoch(TopicBasedRemoteLogMetadataManager.java:236)
> at kafka.log.remote.RemoteLogManager.findHighestRemoteOffset(RemoteLogManager.java:1827)
> at kafka.log.remote.RemoteLogManager$RLMFollowerTask.execute(RemoteLogManager.java:1434)
> at kafka.log.remote.RemoteLogManager$RLMTask.run(RemoteLogManager.java:784)
> {noformat}
> Proposed to add a new `isReady` API in RLMM:
> {code:java}
> /**
> * Denotes whether the partition metadata is ready to serve.
> *
> * @param topicIdPartition topic partition
> * @return True if the partition is ready to serve for remote storage operations.
> */
> default boolean isReady(TopicIdPartition topicIdPartition) {
> return true;
> }
> {code}
> This task is related to [KIP-1105|https://cwiki.apache.org/confluence/display/KAFKA/KIP-1105%3A+Make+remote+log+manager+thread-pool+configs+dynamic]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)