You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Yu-Lin Chen (Jira)" <ji...@apache.org> on 2025/01/10 10:04:00 UTC

[jira] [Created] (KAFKA-18469) AsyncConsumer fails to retry ListOffsetRequest on ReplicaNotAvailable error without metadata update

Yu-Lin Chen created KAFKA-18469:
-----------------------------------

             Summary: AsyncConsumer fails to retry ListOffsetRequest on ReplicaNotAvailable error without metadata update
                 Key: KAFKA-18469
                 URL: https://issues.apache.org/jira/browse/KAFKA-18469
             Project: Kafka
          Issue Type: Bug
          Components: clients, consumer
            Reporter: Yu-Lin Chen
            Assignee: Yu-Lin Chen


In AsyncConsumer, the ListOffsetRequest is only retried after the metadata update[1]. However, not every retriable errors are followed by a metadata update, such as the ReplicaNotAvailable error from remote storage. This errors leads to Consumer#offsetsForTimes failing after api timeout(60 seconds). 

This issue does not happen in ClassicConsumer. [2] We should keep the behavior aligned.

 

This issue is the root cause of the flaky test KAFKA-18036, where consumer#offsetsForTimes is called before remote metadata cache initialized.

 

[1] https://github.com/apache/kafka/blob/5684fc7a2ee1a4f29cb6d69d713233ed3c297882/clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManager.java#L529-L534

[2] [https://github.com/apache/kafka/blob/5684fc7a2ee1a4f29cb6d69d713233ed3c297882/clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetFetcher.java#L144-L153]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)