[jira] [Updated] (KAFKA-18442) Downgrades to 3.3.2 will fail from versions 3.6.0 and above when JBOD is enabled

["Colin McCabe (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Colin McCabe updated KAFKA-18442:
---------------------------------
    Summary: Downgrades to 3.3.2 will fail from versions 3.6.0 and above when JBOD is enabled  (was: Downgrades to 3.3.2 will fail from versions 3.6.0 and above)

> Downgrades to 3.3.2 will fail from versions 3.6.0 and above when JBOD is enabled
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-18442
>                 URL: https://issues.apache.org/jira/browse/KAFKA-18442
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jonah Hooper
>            Priority: Minor
>
> Downgrading from 3.6.x and above to 3.3.x will fail with the following error:
> {code:java}
> kafka.common.InconsistentBrokerMetadataException: BrokerMetadata is not consistent across log.dirs. This could happen if multiple brokers shared a log directory (log.dirs) or partial data was manually copied from another broker.
> ...{code}
> This is broken in 3.3.2 version of Kafka [BrokerMetatadataCheckpoint.scala|https://github.com/apache/kafka/blob/c9c03dd7ef9ff4edf2596e905cabececc72a9e9d/core/src/main/scala/kafka/server/BrokerMetadataCheckpoint.scala#L186]. In 3.3.2 kafka loads information about metadata directories via {{metadata.properties}} files and expects that the properties are duplicated for all log directories. We crash with a fatal error if they are non-duplicate which at that time would mean that another instance of kafka was using the same log directories.
> In [#14291|https://github.com/apache/kafka/pull/14291] (added in Kafka version 3.6.x), the requirement that the {{metadata.properties}} files is duplicate in each directory was dropped. Each `metadata.properties` file will now contain a non-unique {{directory.id}} field for each dir. This has no effect on versions of kafka running {{kraft}} mode, >= 3.4.x (they care only about uniqueness of node.id and cluster.id) but does affect 3.3.2 and until 2.8.x (the change was added in [#9967|https://github.com/apache/kafka/pull/9967]) since it expects every {{metadata.properties}} file to be the same.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)