[jira] [Comment Edited] (KAFKA-20397) Review to improve metadata error check before blocking on buffer data in consumer

["Nilesh Kumar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-20397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18071104#comment-18071104 ] 

Nilesh Kumar edited comment on KAFKA-20397 at 4/4/26 8:08 PM:
--------------------------------------------------------------

[~lianetm] I’d like to investigate this. I’m planning to verify the {{checkInflightPoll}} vs. {{fetchBuffer.awaitNotEmpty}} window with a small test, and if the scenario holds, propose a fail-fast metadata-error check before blocking on the buffer. 


was (Author: JIRAUSER312520):
[~lianetm] I’d like to investigate this. I’m planning to verify the {{checkInflightPoll}} vs. {{fetchBuffer.awaitNotEmpty}} window with a small test, and if the scenario holds, propose a fail-fast metadata-error check before blocking on the buffer.

> Review to improve metadata error check before blocking on buffer data in consumer
> ---------------------------------------------------------------------------------
>
>                 Key: KAFKA-20397
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20397
>             Project: Kafka
>          Issue Type: Task
>          Components: clients, consumer
>            Reporter: Lianet Magrans
>            Priority: Major
>
> The async consumer checks and throws metadata errors on poll from 
> checkInflightPoll.
> Later on (after checking if there is data already buffered), it may block waiting on buffer data (only reducing wait time if the partitions do not have valid positions).
>  
> If the background thread discovers any metadata error in between the above 2 steps, seems that we would still block on data from the buffer. This task is to review if this could be a valid scenario. Probably edge case but worth exploring if we can do better.
> There is already logic before blocking for bufer data, to reduce the wait time if there are no valid positions. We should consider if it would be good to check for metadata errors at that point too? (before blocking / wasting poll time when there is already an error in hand?)
>  
> Not an exact repro but this concern came out looking at the flow on the test AuthorizerIntegrationTest.testCreatePermissionOnClusterToReadFromNonExistentTopic
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)