[jira] [Updated] (KAFKA-12835) Topic IDs can mismatch on brokers (after interbroker protocol version update)

["David Jacot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Jacot updated KAFKA-12835:
--------------------------------
    Fix Version/s: 2.8.1

> Topic IDs can mismatch on brokers (after interbroker protocol version update)
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-12835
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12835
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 2.8.0
>            Reporter: Ivan Yurchenko
>            Assignee: Justine Olshan
>            Priority: Major
>             Fix For: 3.0.0, 2.8.1
>
>
> We had a Kafka cluster running 2.8 version with interbroker protocol set to 2.7. It had a number of topics and everything was fine.
> Then we decided to update the interbroker protocol to 2.8 by the following procedure:
> 1. Run new brokers with the interbroker protocol set to 2.8.
> 2. Move the data from the old brokers to the new ones (normal partition reassignment API).
> 3. Decommission the old brokers.
> At the stage 2 we had the problem: old brokers started failing on {{LeaderAndIsrRequest}} handling with
> {code:java}
> ERROR [Broker id=<...>] Topic Id in memory: <...> does not match the topic Id for partition <...> provided in the request: <...>. (state.change.logger)
> {code}
> for multiple topics. Topics were not recreated.
> We checked {{partition.metadata}} files and IDs there were indeed different from the values in ZooKeeper. It was fixed by deleting the metadata files (and letting them be recreated).
>  
> The logs, unfortunately, didn't show anything that might point to the cause of the issue (or it happened longer ago than we store the logs).
> We tried to reproduce this also, but no success.
> If the community can point out what to check or beware of in future, it will be great. We'll be happy to provide additional information if needed. Thank you! 
> Sorry for the ticket that might be not very actionable. We hope to at least rise awareness of this issue.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)