[jira] [Created] (KAFKA-13111) Re-evaluate Fetch flow for topic ID usage

["Justine Olshan (Jira)" <ji...@apache.org>]

Justine Olshan created KAFKA-13111:
--------------------------------------

             Summary: Re-evaluate Fetch flow for topic ID usage
                 Key: KAFKA-13111
                 URL: https://issues.apache.org/jira/browse/KAFKA-13111
             Project: Kafka
          Issue Type: Improvement
    Affects Versions: 3.1.0
            Reporter: Justine Olshan
            Assignee: Justine Olshan


For fetch request version 13 we have the current method of handling unknown topic IDs.

* When the receiving broker sees an unknown topic ID in a request, it sends a top-level error back and closes the session
* When we encounter an inconsistent (mismatched) ID in the logs, we return a top-level error and close the session.

Ideally, we want to handle the same way as unknown topic IDs. We hold the topic partition in the session and try to resolve on a future fetch request. 

However, there are a few complications with this approach and this is why we opted to simply close the session. Upon further inspection, closing the session on any new topics (when using topic names, we often see a few unknown topic or partition exceptions) is not ideal.

One way to see similar behavior in the topic ID case is to store unresolved IDs in the session. For compatibility reasons, we will need to add and/or generify a few classes. The general idea is that for sessions using version 13+ we will use a structure that keys using topic ID and partition with an optional name. That way, we can send partition level errors for unknown topic IDs and handle them later in the same session. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)