[jira] [Updated] (KAFKA-13111) Re-evaluate Fetch Sessions when using topic IDs

["Justine Olshan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Justine Olshan updated KAFKA-13111:
-----------------------------------
    Summary: Re-evaluate Fetch Sessions when using topic IDs  (was: Re-evaluate Fetch flow for topic ID usage)

> Re-evaluate Fetch Sessions when using topic IDs
> -----------------------------------------------
>
>                 Key: KAFKA-13111
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13111
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 3.1.0
>            Reporter: Justine Olshan
>            Assignee: Justine Olshan
>            Priority: Major
>
> For fetch request version 13 we have the current method of handling unknown topic IDs.
>  * When the receiving broker sees an unknown topic ID in a request, it sends a top-level error back and closes the session
>  * When the broker encounters an inconsistent (mismatched) ID in the logs, it return a top-level error and closes the session.
> Ideally, we want to handle the same way as unknown topic IDs. We hold the topic partition in the session and try to resolve on a future fetch request. 
> However, there are a few complications with this approach and this is why we opted to simply close the session. One is that many objects in the fetch session are keyed using topic name+partition. We also still need the topic name for a few other checks in the fetch path.
> Still, upon further inspection, closing the session on any new topics (when using topic names, we often see a few unknown topic or partition exceptions) is not ideal.
> One way to see similar behavior in the topic ID case is to store unresolved IDs in the session. For compatibility reasons, we will need to add and/or generify a few classes. The general idea is that for sessions using version 13+ we will use a structure that keys using topic ID and partition with an optional name. That way, we can send partition level errors for unknown topic IDs and handle them later in the same session. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)