You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "FrankYang0529 (via GitHub)" <gi...@apache.org> on 2026/04/22 14:37:42 UTC

[PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

FrankYang0529 opened a new pull request, #22118:
URL: https://github.com/apache/kafka/pull/22118

   Migrate `MetadataCacheTest` to metadata module.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

Posted by "mimaison (via GitHub)" <gi...@apache.org>.
mimaison commented on code in PR #22118:
URL: https://github.com/apache/kafka/pull/22118#discussion_r3130540818


##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;

Review Comment:
   We can use pattern matching `if (cache instanceof KRaftMetadataCache c) {`



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;

Review Comment:
   This can be private static



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ).iterator());
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort((a, b) -> Integer.compare(a.partitionIndex(), b.partitionIndex()));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = Collections.singletonList(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator()))
+        );
+
+        // leader is not available. expect LEADER_NOT_AVAILABLE for any metadata version.
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, true);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionListenerNotAvailableOnLeader(MetadataCache cache) {
+        // when listener name is not present in the metadata cache for a broker, getTopicMetadata should
+        // return LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND errors for old and new versions respectively.
+        ListenerName plaintextListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT);
+        ListenerName sslListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.SSL);
+        BrokerEndpointCollection broker0Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value()),
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9093)
+                .setSecurityProtocol(SecurityProtocol.SSL.id)
+                .setName(sslListenerName.value())
+        ).iterator());
+
+        BrokerEndpointCollection broker1Endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("host1")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value())
+        ).iterator());
+
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(broker0Endpoints),
+            new RegisterBrokerRecord()
+                .setBrokerId(1)
+                .setFenced(false)
+                .setEndPoints(broker1Endpoints)
+        );
+
+        // leader available in cache but listener name not present. expect LISTENER_NOT_FOUND error for new metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LISTENER_NOT_FOUND, true);
+        // leader available in cache but listener name not present. expect LEADER_NOT_AVAILABLE error for old metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+    }
+
+    private void verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(
+        MetadataCache cache,
+        List<RegisterBrokerRecord> brokers,
+        ListenerName listenerName,
+        int leader,
+        Errors expectedError,
+        boolean errorUnavailableListeners
+    ) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> records = new ArrayList<>(brokers);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        int leaderEpoch = 1;
+        int partitionEpoch = 3;
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setPartitionEpoch(partitionEpoch)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(List.of(0))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, errorUnavailableListeners);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(expectedError.code(), partitionMetadata.errorCode());
+        assertFalse(partitionMetadata.isrNodes().isEmpty());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataReplicaNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        int partitionEpoch = 3;
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        BrokerEndpointCollection endPoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false)
+            .setEndPoints(endPoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // replica 1 is not available
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0, 1);
+        List<Integer> isr = List.of(0);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setPartitionEpoch(partitionEpoch)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataIsrNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        BrokerEndpointCollection endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("rack1")
+            .setFenced(false)
+            .setEndPoints(endpoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // isr member 1 is not a registered broker
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataWithNonSupportedSecurityProtocol(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+
+        RegisterBrokerRecord broker = new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("")
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+            ).iterator()));
+
+        TopicRecord topicRecord = new TopicRecord().setName(topic).setTopicId(topicId);
+
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(broker);
+        records.add(topicRecord);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata =
+            cache.getTopicMetadata(Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.SSL), false, false);
+        assertEquals(1, topicMetadata.size());
+        assertEquals(1, topicMetadata.get(0).partitions().size());
+        assertEquals(RecordBatch.NO_PARTITION_LEADER_EPOCH, topicMetadata.get(0).partitions().get(0).leaderId());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getAliveBrokersShouldNotBeMutatedByUpdateCache(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> topicRecords = Collections.singletonList(
+            new TopicRecord().setName(topic).setTopicId(topicId));
+
+        List<Integer> initialBrokerIds = List.of(0, 1, 2);
+        updateCacheWithBrokers(cache, initialBrokerIds, topicId, topicRecords);
+        // This should not change the alive brokers
+        updateCacheWithBrokers(cache, List.of(0, 1, 2, 3), topicId, topicRecords);
+        for (int brokerId : initialBrokerIds) {
+            assertTrue(cache.hasAliveBroker(brokerId));
+        }
+    }
+
+    private void updateCacheWithBrokers(MetadataCache cache, List<Integer> brokerIds,
+                                         Uuid topicId, List<ApiMessage> topicRecords) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId : brokerIds) {
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setRack("")
+                .setFenced(false)
+                .setBrokerEpoch(brokerEpoch)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+                ).iterator())));
+        }
+        records.addAll(topicRecords);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(0)
+            .setLeaderEpoch(0)
+            .setIsr(List.of(0, 1))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetPartitionReplicaEndpoints(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        // Set up broker data for the metadata cache
+        int numBrokers = 10;
+        int fencedBrokerId = numBrokers / 3;
+        List<RegisterBrokerRecord> brokerRecords = new ArrayList<>();
+        for (int brokerId = 0; brokerId < numBrokers; brokerId++) {
+            brokerRecords.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setFenced(brokerId == fencedBrokerId)
+                .setRack("rack" + (brokerId % 3))
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo" + brokerId)
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator())));
+        }
+
+        // Set up a single topic (with many partitions) for the metadata cache
+        String topic = "many-partitions-topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        // Set up a number of partitions such that each different combination of
+        // $replicationFactor brokers is made a replica set for exactly one partition
+        int replicationFactor = 3;
+        List<List<Integer>> replicaSets = getAllReplicaSets(numBrokers, replicationFactor);
+        int numPartitions = replicaSets.size();
+
+        List<PartitionRecord> partitionRecords = new ArrayList<>();
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            List<Integer> replicas = replicaSets.get(partitionId);
+            List<Integer> nonFencedReplicas = new ArrayList<>();
+            for (Integer id : replicas) {
+                if (id != fencedBrokerId) nonFencedReplicas.add(id);
+            }
+            partitionRecords.add(new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(partitionId)
+                .setReplicas(replicas)
+                .setLeader(replicas.get(0))
+                .setIsr(nonFencedReplicas)
+                .setEligibleLeaderReplicas(nonFencedReplicas));
+        }
+
+        // Load the prepared data in the metadata cache
+        List<ApiMessage> records = new ArrayList<>(brokerRecords);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.addAll(partitionRecords);
+        updateCache(cache, records);
+
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            TopicPartition tp = new TopicPartition(topic, partitionId);
+            Map<Integer, Node> brokerIdToNodeMap =
+                cache.getPartitionReplicaEndpoints(tp, listenerName);
+            Set<Integer> replicaSet = brokerIdToNodeMap.keySet();
+            Set<Integer> expectedReplicaSet = new HashSet<>(partitionRecords.get(partitionId).replicas());
+
+            // Verify that we have endpoints for exactly the non-fenced brokers of the replica set
+            if (expectedReplicaSet.contains(fencedBrokerId)) {
+                Set<Integer> replicaSetPlusFenced = new HashSet<>(replicaSet);
+                replicaSetPlusFenced.add(fencedBrokerId);
+                assertEquals(expectedReplicaSet, replicaSetPlusFenced,
+                    "Unexpected partial replica set for partition " + partitionId);
+            } else {
+                assertEquals(expectedReplicaSet, replicaSet,
+                    "Unexpected replica set for partition " + partitionId);
+            }
+
+            // Verify that the endpoint data for each non-fenced replica is as expected
+            for (int brokerId : replicaSet) {
+                Node brokerNode = brokerIdToNodeMap.get(brokerId);
+                if (brokerNode == null) {
+                    throw new AssertionError("No brokerNode for broker " + brokerId + " and partition " + partitionId);
+                }
+                RegisterBrokerRecord expectedBroker = brokerRecords.get(brokerId);
+                BrokerEndpoint expectedEndpoint = expectedBroker.endPoints().find(listenerName.value());
+                assertEquals(expectedEndpoint.host(), brokerNode.host(),
+                    "Unexpected host for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedEndpoint.port(), brokerNode.port(),
+                    "Unexpected port for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedBroker.rack(), brokerNode.rack(),
+                    "Unexpected rack for broker " + brokerId + " and partition " + partitionId);
+            }
+        }
+
+        TopicPartition tp = new TopicPartition(topic, numPartitions);
+        Map<Integer, Node> brokerIdToNodeMap =
+            cache.getPartitionReplicaEndpoints(tp, listenerName);
+        assertTrue(brokerIdToNodeMap.isEmpty());
+    }
+
+    private static List<List<Integer>> getAllReplicaSets(int numBrokers, int replicationFactor) {
+        List<List<Integer>> result = new ArrayList<>();
+        int[] indices = new int[replicationFactor];
+        for (int i = 0; i < replicationFactor; i++) indices[i] = i;
+        while (true) {
+            List<Integer> combo = new ArrayList<>();
+            for (int idx : indices) combo.add(idx);
+            result.add(combo);
+            int i = replicationFactor - 1;
+            while (i >= 0 && indices[i] == numBrokers - replicationFactor + i) i--;
+            if (i < 0) break;
+            indices[i]++;
+            for (int j = i + 1; j < replicationFactor; j++) indices[j] = indices[j - 1] + 1;
+        }
+        return result;
+    }
+
+    @Test
+    public void testIsBrokerFenced() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerFenced(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setFenced((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerFenced(0));
+    }
+
+    @Test
+    public void testIsBrokerInControlledShutdown() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerShuttingDown(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerShuttingDown(0));
+    }
+
+    @Test
+    public void testGetLiveBrokerEpoch() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setBrokerEpoch(100)
+            .setFenced(false));
+
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(1)
+            .setBrokerEpoch(101)
+            .setFenced(true));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertEquals(100L, metadataCache.getAliveBrokerEpoch(0).orElse(-1L));
+        assertEquals(-1L, metadataCache.getAliveBrokerEpoch(1).orElse(-1L));
+    }
+
+    @Test
+    public void testDescribeTopicResponse() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        String topic0 = "test0";
+        String topic1 = "test1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();

Review Comment:
   Again Map.of can be used here and for partitionMap just below
   It seems we can get rid of all the new HashMap call



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ).iterator());
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort((a, b) -> Integer.compare(a.partitionIndex(), b.partitionIndex()));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = Collections.singletonList(

Review Comment:
   We can replace all the Collections calls with Set



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ).iterator());
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort((a, b) -> Integer.compare(a.partitionIndex(), b.partitionIndex()));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = Collections.singletonList(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator()))
+        );
+
+        // leader is not available. expect LEADER_NOT_AVAILABLE for any metadata version.
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, true);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionListenerNotAvailableOnLeader(MetadataCache cache) {
+        // when listener name is not present in the metadata cache for a broker, getTopicMetadata should
+        // return LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND errors for old and new versions respectively.
+        ListenerName plaintextListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT);
+        ListenerName sslListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.SSL);
+        BrokerEndpointCollection broker0Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value()),
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9093)
+                .setSecurityProtocol(SecurityProtocol.SSL.id)
+                .setName(sslListenerName.value())
+        ).iterator());
+
+        BrokerEndpointCollection broker1Endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("host1")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value())
+        ).iterator());
+
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(broker0Endpoints),
+            new RegisterBrokerRecord()
+                .setBrokerId(1)
+                .setFenced(false)
+                .setEndPoints(broker1Endpoints)
+        );
+
+        // leader available in cache but listener name not present. expect LISTENER_NOT_FOUND error for new metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LISTENER_NOT_FOUND, true);
+        // leader available in cache but listener name not present. expect LEADER_NOT_AVAILABLE error for old metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+    }
+
+    private void verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(
+        MetadataCache cache,
+        List<RegisterBrokerRecord> brokers,
+        ListenerName listenerName,
+        int leader,
+        Errors expectedError,
+        boolean errorUnavailableListeners
+    ) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> records = new ArrayList<>(brokers);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        int leaderEpoch = 1;
+        int partitionEpoch = 3;
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setPartitionEpoch(partitionEpoch)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(List.of(0))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, errorUnavailableListeners);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(expectedError.code(), partitionMetadata.errorCode());
+        assertFalse(partitionMetadata.isrNodes().isEmpty());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataReplicaNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        int partitionEpoch = 3;
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        BrokerEndpointCollection endPoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false)
+            .setEndPoints(endPoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // replica 1 is not available
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0, 1);
+        List<Integer> isr = List.of(0);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setPartitionEpoch(partitionEpoch)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataIsrNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        BrokerEndpointCollection endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("rack1")
+            .setFenced(false)
+            .setEndPoints(endpoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // isr member 1 is not a registered broker
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataWithNonSupportedSecurityProtocol(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+
+        RegisterBrokerRecord broker = new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("")
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+            ).iterator()));
+
+        TopicRecord topicRecord = new TopicRecord().setName(topic).setTopicId(topicId);
+
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(broker);
+        records.add(topicRecord);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata =
+            cache.getTopicMetadata(Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.SSL), false, false);
+        assertEquals(1, topicMetadata.size());
+        assertEquals(1, topicMetadata.get(0).partitions().size());
+        assertEquals(RecordBatch.NO_PARTITION_LEADER_EPOCH, topicMetadata.get(0).partitions().get(0).leaderId());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getAliveBrokersShouldNotBeMutatedByUpdateCache(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> topicRecords = Collections.singletonList(
+            new TopicRecord().setName(topic).setTopicId(topicId));
+
+        List<Integer> initialBrokerIds = List.of(0, 1, 2);
+        updateCacheWithBrokers(cache, initialBrokerIds, topicId, topicRecords);
+        // This should not change the alive brokers
+        updateCacheWithBrokers(cache, List.of(0, 1, 2, 3), topicId, topicRecords);
+        for (int brokerId : initialBrokerIds) {
+            assertTrue(cache.hasAliveBroker(brokerId));
+        }
+    }
+
+    private void updateCacheWithBrokers(MetadataCache cache, List<Integer> brokerIds,
+                                         Uuid topicId, List<ApiMessage> topicRecords) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId : brokerIds) {
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setRack("")
+                .setFenced(false)
+                .setBrokerEpoch(brokerEpoch)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+                ).iterator())));
+        }
+        records.addAll(topicRecords);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(0)
+            .setLeaderEpoch(0)
+            .setIsr(List.of(0, 1))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetPartitionReplicaEndpoints(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        // Set up broker data for the metadata cache
+        int numBrokers = 10;
+        int fencedBrokerId = numBrokers / 3;
+        List<RegisterBrokerRecord> brokerRecords = new ArrayList<>();
+        for (int brokerId = 0; brokerId < numBrokers; brokerId++) {
+            brokerRecords.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setFenced(brokerId == fencedBrokerId)
+                .setRack("rack" + (brokerId % 3))
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo" + brokerId)
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator())));
+        }
+
+        // Set up a single topic (with many partitions) for the metadata cache
+        String topic = "many-partitions-topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        // Set up a number of partitions such that each different combination of
+        // $replicationFactor brokers is made a replica set for exactly one partition
+        int replicationFactor = 3;
+        List<List<Integer>> replicaSets = getAllReplicaSets(numBrokers, replicationFactor);
+        int numPartitions = replicaSets.size();
+
+        List<PartitionRecord> partitionRecords = new ArrayList<>();
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            List<Integer> replicas = replicaSets.get(partitionId);
+            List<Integer> nonFencedReplicas = new ArrayList<>();
+            for (Integer id : replicas) {
+                if (id != fencedBrokerId) nonFencedReplicas.add(id);
+            }
+            partitionRecords.add(new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(partitionId)
+                .setReplicas(replicas)
+                .setLeader(replicas.get(0))
+                .setIsr(nonFencedReplicas)
+                .setEligibleLeaderReplicas(nonFencedReplicas));
+        }
+
+        // Load the prepared data in the metadata cache
+        List<ApiMessage> records = new ArrayList<>(brokerRecords);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.addAll(partitionRecords);
+        updateCache(cache, records);
+
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            TopicPartition tp = new TopicPartition(topic, partitionId);
+            Map<Integer, Node> brokerIdToNodeMap =
+                cache.getPartitionReplicaEndpoints(tp, listenerName);
+            Set<Integer> replicaSet = brokerIdToNodeMap.keySet();
+            Set<Integer> expectedReplicaSet = new HashSet<>(partitionRecords.get(partitionId).replicas());
+
+            // Verify that we have endpoints for exactly the non-fenced brokers of the replica set
+            if (expectedReplicaSet.contains(fencedBrokerId)) {
+                Set<Integer> replicaSetPlusFenced = new HashSet<>(replicaSet);
+                replicaSetPlusFenced.add(fencedBrokerId);
+                assertEquals(expectedReplicaSet, replicaSetPlusFenced,
+                    "Unexpected partial replica set for partition " + partitionId);
+            } else {
+                assertEquals(expectedReplicaSet, replicaSet,
+                    "Unexpected replica set for partition " + partitionId);
+            }
+
+            // Verify that the endpoint data for each non-fenced replica is as expected
+            for (int brokerId : replicaSet) {
+                Node brokerNode = brokerIdToNodeMap.get(brokerId);
+                if (brokerNode == null) {
+                    throw new AssertionError("No brokerNode for broker " + brokerId + " and partition " + partitionId);
+                }
+                RegisterBrokerRecord expectedBroker = brokerRecords.get(brokerId);
+                BrokerEndpoint expectedEndpoint = expectedBroker.endPoints().find(listenerName.value());
+                assertEquals(expectedEndpoint.host(), brokerNode.host(),
+                    "Unexpected host for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedEndpoint.port(), brokerNode.port(),
+                    "Unexpected port for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedBroker.rack(), brokerNode.rack(),
+                    "Unexpected rack for broker " + brokerId + " and partition " + partitionId);
+            }
+        }
+
+        TopicPartition tp = new TopicPartition(topic, numPartitions);
+        Map<Integer, Node> brokerIdToNodeMap =
+            cache.getPartitionReplicaEndpoints(tp, listenerName);
+        assertTrue(brokerIdToNodeMap.isEmpty());
+    }
+
+    private static List<List<Integer>> getAllReplicaSets(int numBrokers, int replicationFactor) {
+        List<List<Integer>> result = new ArrayList<>();
+        int[] indices = new int[replicationFactor];
+        for (int i = 0; i < replicationFactor; i++) indices[i] = i;
+        while (true) {
+            List<Integer> combo = new ArrayList<>();
+            for (int idx : indices) combo.add(idx);
+            result.add(combo);
+            int i = replicationFactor - 1;
+            while (i >= 0 && indices[i] == numBrokers - replicationFactor + i) i--;
+            if (i < 0) break;
+            indices[i]++;
+            for (int j = i + 1; j < replicationFactor; j++) indices[j] = indices[j - 1] + 1;
+        }
+        return result;
+    }
+
+    @Test
+    public void testIsBrokerFenced() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerFenced(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setFenced((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerFenced(0));
+    }
+
+    @Test
+    public void testIsBrokerInControlledShutdown() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerShuttingDown(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerShuttingDown(0));
+    }
+
+    @Test
+    public void testGetLiveBrokerEpoch() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setBrokerEpoch(100)
+            .setFenced(false));
+
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(1)
+            .setBrokerEpoch(101)
+            .setFenced(true));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertEquals(100L, metadataCache.getAliveBrokerEpoch(0).orElse(-1L));
+        assertEquals(-1L, metadataCache.getAliveBrokerEpoch(1).orElse(-1L));
+    }
+
+    @Test
+    public void testDescribeTopicResponse() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        String topic0 = "test0";
+        String topic1 = "test1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        // partitionMap key: "topicName:partitionId"
+        Map<String, PartitionRecord> partitionMap = new HashMap<>();
+        partitionMap.put(topic0 + ":0", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(0)
+            .setReplicas(List.of(0, 1, 2))
+            .setLeader(0)
+            .setIsr(List.of(0))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(2))
+            .setLeaderEpoch(0)
+            .setPartitionEpoch(1)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic0 + ":2", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(2)
+            .setReplicas(List.of(0, 2, 3))
+            .setLeader(3)
+            .setIsr(List.of(3))
+            .setEligibleLeaderReplicas(List.of(2))
+            .setLastKnownElr(List.of(0))
+            .setLeaderEpoch(1)
+            .setPartitionEpoch(2)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic0 + ":1", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(1)
+            .setReplicas(List.of(0, 1, 3))
+            .setLeader(0)
+            .setIsr(List.of(0))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(3))
+            .setLeaderEpoch(0)
+            .setPartitionEpoch(2)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic1 + ":0", new PartitionRecord()
+            .setTopicId(topicIds.get(topic1))
+            .setPartitionId(0)
+            .setReplicas(List.of(0, 1, 2))
+            .setLeader(2)
+            .setIsr(List.of(2))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(0))
+            .setLeaderEpoch(10)
+            .setPartitionEpoch(11)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(0)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo0").setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(1)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo1").setPort(9093)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(2)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo2").setPort(9094)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(3)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo3").setPort(9095)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionMap.values());
+        updateCache(metadataCache, records);
+
+        // Basic test
+        List<DescribeTopicPartitionsResponseTopic> result = new ArrayList<>(metadataCache
+            .describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName, t -> 0, 10, false)
+            .topics());
+        assertEquals(2, result.size());
+        DescribeTopicPartitionsResponseTopic resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(3, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0, 1, 2)), resultTopic.partitions(), partitionMap);
+
+        resultTopic = result.get(1);
+        assertEquals(topic1, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic1), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic1, new HashSet<>(List.of(0)), resultTopic.partitions(), partitionMap);
+
+        // Quota reached
+        DescribeTopicPartitionsResponseData response = metadataCache
+            .describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName, t -> 0, 2, false);
+        result = new ArrayList<>(response.topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(2, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0, 1)), resultTopic.partitions(), partitionMap);
+        assertEquals(topic0, response.nextCursor().topicName());
+        assertEquals(2, response.nextCursor().partitionIndex());
+
+        // With start index
+        result = new ArrayList<>(metadataCache
+            .describeTopicResponse(List.of(topic0).iterator(), listenerName,
+                t -> t.equals(topic0) ? 1 : 0, 10, false)
+            .topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(2, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(1, 2)), resultTopic.partitions(), partitionMap);
+
+        // With start index and quota reached
+        response = metadataCache.describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName,
+            t -> t.equals(topic0) ? 2 : 0, 1, false);
+        result = new ArrayList<>(response.topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(2)), resultTopic.partitions(), partitionMap);
+        assertEquals(topic1, response.nextCursor().topicName());
+        assertEquals(0, response.nextCursor().partitionIndex());
+
+        // When the first topic does not exist
+        result = new ArrayList<>(metadataCache.describeTopicResponse(List.of("Non-exist", topic0).iterator(), listenerName,
+            t -> t.equals("Non-exist") ? 1 : 0, 1, false).topics());
+        assertEquals(2, result.size());
+        resultTopic = result.get(0);
+        assertEquals("Non-exist", resultTopic.name());
+        assertEquals(3, resultTopic.errorCode());
+
+        resultTopic = result.get(1);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0)), resultTopic.partitions(), partitionMap);
+    }
+
+    private void checkTopicMetadata(
+        String topic,
+        Set<Integer> partitionIds,
+        List<DescribeTopicPartitionsResponsePartition> partitions,
+        Map<String, PartitionRecord> partitionMap
+    ) {
+        for (DescribeTopicPartitionsResponsePartition partition : partitions) {
+            int partitionId = partition.partitionIndex();
+            assertTrue(partitionIds.contains(partitionId));
+            PartitionRecord expectedPartition = partitionMap.get(topic + ":" + partitionId);
+            assertEquals(0, partition.errorCode());
+            assertEquals(expectedPartition.leaderEpoch(), partition.leaderEpoch());
+            assertEquals(expectedPartition.partitionId(), partition.partitionIndex());
+            assertEquals(expectedPartition.eligibleLeaderReplicas(), partition.eligibleLeaderReplicas());
+            assertEquals(expectedPartition.isr(), partition.isrNodes());
+            assertEquals(expectedPartition.lastKnownElr(), partition.lastKnownElr());
+            assertEquals(expectedPartition.leader(), partition.leaderId());
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetLeaderAndIsr(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        int partitionIndex = 0;
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> isr = List.of(2, 3, 0);
+        List<Integer> replicas = List.of(2, 3, 0, 1, 4);
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setBrokerEpoch(brokerEpoch)
+            .setRack("rack1")
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(listenerName.value())
+            ).iterator())));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(partitionIndex)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        Optional<LeaderAndIsr> leaderAndIsr = cache.getLeaderAndIsr(topic, partitionIndex);
+        assertEquals(Optional.of(leader), leaderAndIsr.map(LeaderAndIsr::leader));
+        assertEquals(Optional.of(leaderEpoch), leaderAndIsr.map(LeaderAndIsr::leaderEpoch));
+        assertEquals(Optional.of(new HashSet<>(isr)), leaderAndIsr.map(lai -> new HashSet<>(lai.isr())));
+        assertEquals(Optional.of(-1), leaderAndIsr.map(LeaderAndIsr::partitionEpoch));
+        assertEquals(Optional.of(LeaderRecoveryState.RECOVERED), leaderAndIsr.map(LeaderAndIsr::leaderRecoveryState));
+    }
+
+    @Test
+    public void testGetOfflineReplicasConsidersDirAssignment() {
+        Map<Integer, List<Integer>> result = offlinePartitions(
+            List.of(
+                new BrokerInfo(0, List.of(Uuid.fromString("broker1logdirjEo71BG0w"))),
+                new BrokerInfo(1, List.of(Uuid.fromString("broker2logdirRmQQgLxgw")))
+            ),
+            List.of(
+                new PartitionInfo2(0, List.of(0, 1),
+                    List.of(Uuid.fromString("broker1logdirjEo71BG0w"), DirectoryId.LOST)),
+                new PartitionInfo2(1, List.of(0, 1),
+                    List.of(Uuid.fromString("unknownlogdirjEo71BG0w"), DirectoryId.UNASSIGNED)),
+                new PartitionInfo2(2, List.of(0, 1),
+                    List.of(DirectoryId.MIGRATING, Uuid.fromString("broker2logdirRmQQgLxgw")))
+            )
+        );
+
+        Map<Integer, List<Integer>> expected = new HashMap<>();
+        expected.put(0, List.of(1));
+        expected.put(1, List.of(0));
+        expected.put(2, Collections.emptyList());
+        assertEquals(expected, result);
+    }
+
+    private static class BrokerInfo {
+        final int id;
+        final List<Uuid> dirs;
+
+        BrokerInfo(int id, List<Uuid> dirs) {
+            this.id = id;
+            this.dirs = dirs;
+        }
+    }
+
+    private static class PartitionInfo2 {

Review Comment:
   Also let's keep the original names Broker and Partition



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ).iterator());
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort((a, b) -> Integer.compare(a.partitionIndex(), b.partitionIndex()));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = Collections.singletonList(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator()))
+        );
+
+        // leader is not available. expect LEADER_NOT_AVAILABLE for any metadata version.
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, true);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionListenerNotAvailableOnLeader(MetadataCache cache) {
+        // when listener name is not present in the metadata cache for a broker, getTopicMetadata should
+        // return LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND errors for old and new versions respectively.
+        ListenerName plaintextListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT);
+        ListenerName sslListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.SSL);
+        BrokerEndpointCollection broker0Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value()),
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9093)
+                .setSecurityProtocol(SecurityProtocol.SSL.id)
+                .setName(sslListenerName.value())
+        ).iterator());
+
+        BrokerEndpointCollection broker1Endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("host1")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value())
+        ).iterator());
+
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(broker0Endpoints),
+            new RegisterBrokerRecord()
+                .setBrokerId(1)
+                .setFenced(false)
+                .setEndPoints(broker1Endpoints)
+        );
+
+        // leader available in cache but listener name not present. expect LISTENER_NOT_FOUND error for new metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LISTENER_NOT_FOUND, true);
+        // leader available in cache but listener name not present. expect LEADER_NOT_AVAILABLE error for old metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+    }
+
+    private void verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(
+        MetadataCache cache,
+        List<RegisterBrokerRecord> brokers,
+        ListenerName listenerName,
+        int leader,
+        Errors expectedError,
+        boolean errorUnavailableListeners
+    ) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> records = new ArrayList<>(brokers);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        int leaderEpoch = 1;
+        int partitionEpoch = 3;
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setPartitionEpoch(partitionEpoch)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(List.of(0))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, errorUnavailableListeners);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(expectedError.code(), partitionMetadata.errorCode());
+        assertFalse(partitionMetadata.isrNodes().isEmpty());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataReplicaNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        int partitionEpoch = 3;
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        BrokerEndpointCollection endPoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false)
+            .setEndPoints(endPoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // replica 1 is not available
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0, 1);
+        List<Integer> isr = List.of(0);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setPartitionEpoch(partitionEpoch)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataIsrNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        BrokerEndpointCollection endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("rack1")
+            .setFenced(false)
+            .setEndPoints(endpoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // isr member 1 is not a registered broker
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadata.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.isrNodes()));
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.replicaNodes()));
+        assertEquals(new HashSet<>(List.of(0)), new HashSet<>(partitionMetadataWithError.isrNodes()));
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataWithNonSupportedSecurityProtocol(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+
+        RegisterBrokerRecord broker = new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("")
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+            ).iterator()));
+
+        TopicRecord topicRecord = new TopicRecord().setName(topic).setTopicId(topicId);
+
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(broker);
+        records.add(topicRecord);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata =
+            cache.getTopicMetadata(Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.SSL), false, false);
+        assertEquals(1, topicMetadata.size());
+        assertEquals(1, topicMetadata.get(0).partitions().size());
+        assertEquals(RecordBatch.NO_PARTITION_LEADER_EPOCH, topicMetadata.get(0).partitions().get(0).leaderId());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getAliveBrokersShouldNotBeMutatedByUpdateCache(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> topicRecords = Collections.singletonList(
+            new TopicRecord().setName(topic).setTopicId(topicId));
+
+        List<Integer> initialBrokerIds = List.of(0, 1, 2);
+        updateCacheWithBrokers(cache, initialBrokerIds, topicId, topicRecords);
+        // This should not change the alive brokers
+        updateCacheWithBrokers(cache, List.of(0, 1, 2, 3), topicId, topicRecords);
+        for (int brokerId : initialBrokerIds) {
+            assertTrue(cache.hasAliveBroker(brokerId));
+        }
+    }
+
+    private void updateCacheWithBrokers(MetadataCache cache, List<Integer> brokerIds,
+                                         Uuid topicId, List<ApiMessage> topicRecords) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId : brokerIds) {
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setRack("")
+                .setFenced(false)
+                .setBrokerEpoch(brokerEpoch)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+                ).iterator())));
+        }
+        records.addAll(topicRecords);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(0)
+            .setLeaderEpoch(0)
+            .setIsr(List.of(0, 1))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetPartitionReplicaEndpoints(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        // Set up broker data for the metadata cache
+        int numBrokers = 10;
+        int fencedBrokerId = numBrokers / 3;
+        List<RegisterBrokerRecord> brokerRecords = new ArrayList<>();
+        for (int brokerId = 0; brokerId < numBrokers; brokerId++) {
+            brokerRecords.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setFenced(brokerId == fencedBrokerId)
+                .setRack("rack" + (brokerId % 3))
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo" + brokerId)
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator())));
+        }
+
+        // Set up a single topic (with many partitions) for the metadata cache
+        String topic = "many-partitions-topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        // Set up a number of partitions such that each different combination of
+        // $replicationFactor brokers is made a replica set for exactly one partition
+        int replicationFactor = 3;
+        List<List<Integer>> replicaSets = getAllReplicaSets(numBrokers, replicationFactor);
+        int numPartitions = replicaSets.size();
+
+        List<PartitionRecord> partitionRecords = new ArrayList<>();
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            List<Integer> replicas = replicaSets.get(partitionId);
+            List<Integer> nonFencedReplicas = new ArrayList<>();
+            for (Integer id : replicas) {
+                if (id != fencedBrokerId) nonFencedReplicas.add(id);
+            }
+            partitionRecords.add(new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(partitionId)
+                .setReplicas(replicas)
+                .setLeader(replicas.get(0))
+                .setIsr(nonFencedReplicas)
+                .setEligibleLeaderReplicas(nonFencedReplicas));
+        }
+
+        // Load the prepared data in the metadata cache
+        List<ApiMessage> records = new ArrayList<>(brokerRecords);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.addAll(partitionRecords);
+        updateCache(cache, records);
+
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            TopicPartition tp = new TopicPartition(topic, partitionId);
+            Map<Integer, Node> brokerIdToNodeMap =
+                cache.getPartitionReplicaEndpoints(tp, listenerName);
+            Set<Integer> replicaSet = brokerIdToNodeMap.keySet();
+            Set<Integer> expectedReplicaSet = new HashSet<>(partitionRecords.get(partitionId).replicas());
+
+            // Verify that we have endpoints for exactly the non-fenced brokers of the replica set
+            if (expectedReplicaSet.contains(fencedBrokerId)) {
+                Set<Integer> replicaSetPlusFenced = new HashSet<>(replicaSet);
+                replicaSetPlusFenced.add(fencedBrokerId);
+                assertEquals(expectedReplicaSet, replicaSetPlusFenced,
+                    "Unexpected partial replica set for partition " + partitionId);
+            } else {
+                assertEquals(expectedReplicaSet, replicaSet,
+                    "Unexpected replica set for partition " + partitionId);
+            }
+
+            // Verify that the endpoint data for each non-fenced replica is as expected
+            for (int brokerId : replicaSet) {
+                Node brokerNode = brokerIdToNodeMap.get(brokerId);
+                if (brokerNode == null) {
+                    throw new AssertionError("No brokerNode for broker " + brokerId + " and partition " + partitionId);
+                }
+                RegisterBrokerRecord expectedBroker = brokerRecords.get(brokerId);
+                BrokerEndpoint expectedEndpoint = expectedBroker.endPoints().find(listenerName.value());
+                assertEquals(expectedEndpoint.host(), brokerNode.host(),
+                    "Unexpected host for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedEndpoint.port(), brokerNode.port(),
+                    "Unexpected port for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedBroker.rack(), brokerNode.rack(),
+                    "Unexpected rack for broker " + brokerId + " and partition " + partitionId);
+            }
+        }
+
+        TopicPartition tp = new TopicPartition(topic, numPartitions);
+        Map<Integer, Node> brokerIdToNodeMap =
+            cache.getPartitionReplicaEndpoints(tp, listenerName);
+        assertTrue(brokerIdToNodeMap.isEmpty());
+    }
+
+    private static List<List<Integer>> getAllReplicaSets(int numBrokers, int replicationFactor) {
+        List<List<Integer>> result = new ArrayList<>();
+        int[] indices = new int[replicationFactor];
+        for (int i = 0; i < replicationFactor; i++) indices[i] = i;
+        while (true) {
+            List<Integer> combo = new ArrayList<>();
+            for (int idx : indices) combo.add(idx);
+            result.add(combo);
+            int i = replicationFactor - 1;
+            while (i >= 0 && indices[i] == numBrokers - replicationFactor + i) i--;
+            if (i < 0) break;
+            indices[i]++;
+            for (int j = i + 1; j < replicationFactor; j++) indices[j] = indices[j - 1] + 1;
+        }
+        return result;
+    }
+
+    @Test
+    public void testIsBrokerFenced() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerFenced(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setFenced((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerFenced(0));
+    }
+
+    @Test
+    public void testIsBrokerInControlledShutdown() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown(false));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertFalse(metadataCache.isBrokerShuttingDown(0));
+
+        delta.replay(new BrokerRegistrationChangeRecord()
+            .setBrokerId(0)
+            .setInControlledShutdown((byte) 1));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertTrue(metadataCache.isBrokerShuttingDown(0));
+    }
+
+    @Test
+    public void testGetLiveBrokerEpoch() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        MetadataDelta delta = new MetadataDelta.Builder().build();
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setBrokerEpoch(100)
+            .setFenced(false));
+
+        delta.replay(new RegisterBrokerRecord()
+            .setBrokerId(1)
+            .setBrokerEpoch(101)
+            .setFenced(true));
+
+        metadataCache.setImage(delta.apply(MetadataProvenance.EMPTY));
+
+        assertEquals(100L, metadataCache.getAliveBrokerEpoch(0).orElse(-1L));
+        assertEquals(-1L, metadataCache.getAliveBrokerEpoch(1).orElse(-1L));
+    }
+
+    @Test
+    public void testDescribeTopicResponse() {
+        KRaftMetadataCache metadataCache = new KRaftMetadataCache(0, () -> KRaftVersion.KRAFT_VERSION_0);
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        String topic0 = "test0";
+        String topic1 = "test1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        // partitionMap key: "topicName:partitionId"
+        Map<String, PartitionRecord> partitionMap = new HashMap<>();
+        partitionMap.put(topic0 + ":0", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(0)
+            .setReplicas(List.of(0, 1, 2))
+            .setLeader(0)
+            .setIsr(List.of(0))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(2))
+            .setLeaderEpoch(0)
+            .setPartitionEpoch(1)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic0 + ":2", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(2)
+            .setReplicas(List.of(0, 2, 3))
+            .setLeader(3)
+            .setIsr(List.of(3))
+            .setEligibleLeaderReplicas(List.of(2))
+            .setLastKnownElr(List.of(0))
+            .setLeaderEpoch(1)
+            .setPartitionEpoch(2)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic0 + ":1", new PartitionRecord()
+            .setTopicId(topicIds.get(topic0))
+            .setPartitionId(1)
+            .setReplicas(List.of(0, 1, 3))
+            .setLeader(0)
+            .setIsr(List.of(0))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(3))
+            .setLeaderEpoch(0)
+            .setPartitionEpoch(2)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+        partitionMap.put(topic1 + ":0", new PartitionRecord()
+            .setTopicId(topicIds.get(topic1))
+            .setPartitionId(0)
+            .setReplicas(List.of(0, 1, 2))
+            .setLeader(2)
+            .setIsr(List.of(2))
+            .setEligibleLeaderReplicas(List.of(1))
+            .setLastKnownElr(List.of(0))
+            .setLeaderEpoch(10)
+            .setPartitionEpoch(11)
+            .setLeaderRecoveryState(LeaderRecoveryState.RECOVERED.value()));
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(0)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo0").setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(1)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo1").setPort(9093)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(2)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo2").setPort(9094)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new RegisterBrokerRecord().setBrokerEpoch(brokerEpoch).setFenced(false).setBrokerId(3)
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint().setHost("foo3").setPort(9095)
+                    .setSecurityProtocol(securityProtocol.id).setName(listenerName.value())
+            ).iterator())));
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionMap.values());
+        updateCache(metadataCache, records);
+
+        // Basic test
+        List<DescribeTopicPartitionsResponseTopic> result = new ArrayList<>(metadataCache
+            .describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName, t -> 0, 10, false)
+            .topics());
+        assertEquals(2, result.size());
+        DescribeTopicPartitionsResponseTopic resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(3, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0, 1, 2)), resultTopic.partitions(), partitionMap);
+
+        resultTopic = result.get(1);
+        assertEquals(topic1, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic1), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic1, new HashSet<>(List.of(0)), resultTopic.partitions(), partitionMap);
+
+        // Quota reached
+        DescribeTopicPartitionsResponseData response = metadataCache
+            .describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName, t -> 0, 2, false);
+        result = new ArrayList<>(response.topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(2, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0, 1)), resultTopic.partitions(), partitionMap);
+        assertEquals(topic0, response.nextCursor().topicName());
+        assertEquals(2, response.nextCursor().partitionIndex());
+
+        // With start index
+        result = new ArrayList<>(metadataCache
+            .describeTopicResponse(List.of(topic0).iterator(), listenerName,
+                t -> t.equals(topic0) ? 1 : 0, 10, false)
+            .topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(2, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(1, 2)), resultTopic.partitions(), partitionMap);
+
+        // With start index and quota reached
+        response = metadataCache.describeTopicResponse(List.of(topic0, topic1).iterator(), listenerName,
+            t -> t.equals(topic0) ? 2 : 0, 1, false);
+        result = new ArrayList<>(response.topics());
+        assertEquals(1, result.size());
+        resultTopic = result.get(0);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(2)), resultTopic.partitions(), partitionMap);
+        assertEquals(topic1, response.nextCursor().topicName());
+        assertEquals(0, response.nextCursor().partitionIndex());
+
+        // When the first topic does not exist
+        result = new ArrayList<>(metadataCache.describeTopicResponse(List.of("Non-exist", topic0).iterator(), listenerName,
+            t -> t.equals("Non-exist") ? 1 : 0, 1, false).topics());
+        assertEquals(2, result.size());
+        resultTopic = result.get(0);
+        assertEquals("Non-exist", resultTopic.name());
+        assertEquals(3, resultTopic.errorCode());
+
+        resultTopic = result.get(1);
+        assertEquals(topic0, resultTopic.name());
+        assertEquals(0, resultTopic.errorCode());
+        assertEquals(topicIds.get(topic0), resultTopic.topicId());
+        assertEquals(1, resultTopic.partitions().size());
+        checkTopicMetadata(topic0, new HashSet<>(List.of(0)), resultTopic.partitions(), partitionMap);
+    }
+
+    private void checkTopicMetadata(
+        String topic,
+        Set<Integer> partitionIds,
+        List<DescribeTopicPartitionsResponsePartition> partitions,
+        Map<String, PartitionRecord> partitionMap
+    ) {
+        for (DescribeTopicPartitionsResponsePartition partition : partitions) {
+            int partitionId = partition.partitionIndex();
+            assertTrue(partitionIds.contains(partitionId));
+            PartitionRecord expectedPartition = partitionMap.get(topic + ":" + partitionId);
+            assertEquals(0, partition.errorCode());
+            assertEquals(expectedPartition.leaderEpoch(), partition.leaderEpoch());
+            assertEquals(expectedPartition.partitionId(), partition.partitionIndex());
+            assertEquals(expectedPartition.eligibleLeaderReplicas(), partition.eligibleLeaderReplicas());
+            assertEquals(expectedPartition.isr(), partition.isrNodes());
+            assertEquals(expectedPartition.lastKnownElr(), partition.lastKnownElr());
+            assertEquals(expectedPartition.leader(), partition.leaderId());
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetLeaderAndIsr(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        int partitionIndex = 0;
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> isr = List.of(2, 3, 0);
+        List<Integer> replicas = List.of(2, 3, 0, 1, 4);
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setBrokerEpoch(brokerEpoch)
+            .setRack("rack1")
+            .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(listenerName.value())
+            ).iterator())));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(partitionIndex)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        Optional<LeaderAndIsr> leaderAndIsr = cache.getLeaderAndIsr(topic, partitionIndex);
+        assertEquals(Optional.of(leader), leaderAndIsr.map(LeaderAndIsr::leader));
+        assertEquals(Optional.of(leaderEpoch), leaderAndIsr.map(LeaderAndIsr::leaderEpoch));
+        assertEquals(Optional.of(new HashSet<>(isr)), leaderAndIsr.map(lai -> new HashSet<>(lai.isr())));
+        assertEquals(Optional.of(-1), leaderAndIsr.map(LeaderAndIsr::partitionEpoch));
+        assertEquals(Optional.of(LeaderRecoveryState.RECOVERED), leaderAndIsr.map(LeaderAndIsr::leaderRecoveryState));
+    }
+
+    @Test
+    public void testGetOfflineReplicasConsidersDirAssignment() {
+        Map<Integer, List<Integer>> result = offlinePartitions(
+            List.of(
+                new BrokerInfo(0, List.of(Uuid.fromString("broker1logdirjEo71BG0w"))),
+                new BrokerInfo(1, List.of(Uuid.fromString("broker2logdirRmQQgLxgw")))
+            ),
+            List.of(
+                new PartitionInfo2(0, List.of(0, 1),
+                    List.of(Uuid.fromString("broker1logdirjEo71BG0w"), DirectoryId.LOST)),
+                new PartitionInfo2(1, List.of(0, 1),
+                    List.of(Uuid.fromString("unknownlogdirjEo71BG0w"), DirectoryId.UNASSIGNED)),
+                new PartitionInfo2(2, List.of(0, 1),
+                    List.of(DirectoryId.MIGRATING, Uuid.fromString("broker2logdirRmQQgLxgw")))
+            )
+        );
+
+        Map<Integer, List<Integer>> expected = new HashMap<>();
+        expected.put(0, List.of(1));
+        expected.put(1, List.of(0));
+        expected.put(2, Collections.emptyList());
+        assertEquals(expected, result);
+    }
+
+    private static class BrokerInfo {
+        final int id;
+        final List<Uuid> dirs;
+
+        BrokerInfo(int id, List<Uuid> dirs) {
+            this.id = id;
+            this.dirs = dirs;
+        }
+    }
+
+    private static class PartitionInfo2 {

Review Comment:
   Could we use record classes?



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();
+        topicIds.put(topic0, Uuid.randomUuid());
+        topicIds.put(topic1, Uuid.randomUuid());
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ).iterator());
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort((a, b) -> Integer.compare(a.partitionIndex(), b.partitionIndex()));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = Collections.singletonList(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(new BrokerEndpointCollection(Collections.singletonList(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ).iterator()))
+        );
+
+        // leader is not available. expect LEADER_NOT_AVAILABLE for any metadata version.
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, true);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionListenerNotAvailableOnLeader(MetadataCache cache) {
+        // when listener name is not present in the metadata cache for a broker, getTopicMetadata should
+        // return LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND errors for old and new versions respectively.
+        ListenerName plaintextListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT);
+        ListenerName sslListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.SSL);
+        BrokerEndpointCollection broker0Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value()),
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9093)
+                .setSecurityProtocol(SecurityProtocol.SSL.id)
+                .setName(sslListenerName.value())
+        ).iterator());
+
+        BrokerEndpointCollection broker1Endpoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("host1")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value())
+        ).iterator());
+
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(broker0Endpoints),
+            new RegisterBrokerRecord()
+                .setBrokerId(1)
+                .setFenced(false)
+                .setEndPoints(broker1Endpoints)
+        );
+
+        // leader available in cache but listener name not present. expect LISTENER_NOT_FOUND error for new metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LISTENER_NOT_FOUND, true);
+        // leader available in cache but listener name not present. expect LEADER_NOT_AVAILABLE error for old metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+    }
+
+    private void verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(
+        MetadataCache cache,
+        List<RegisterBrokerRecord> brokers,
+        ListenerName listenerName,
+        int leader,
+        Errors expectedError,
+        boolean errorUnavailableListeners
+    ) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> records = new ArrayList<>(brokers);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        int leaderEpoch = 1;
+        int partitionEpoch = 3;
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setPartitionEpoch(partitionEpoch)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(List.of(0))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, errorUnavailableListeners);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(expectedError.code(), partitionMetadata.errorCode());
+        assertFalse(partitionMetadata.isrNodes().isEmpty());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataReplicaNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        int partitionEpoch = 3;
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        BrokerEndpointCollection endPoints = new BrokerEndpointCollection(Collections.singletonList(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ).iterator());
+
+        List<ApiMessage> records = new ArrayList<>();
+        records.add(new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setFenced(false)
+            .setEndPoints(endPoints));
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        // replica 1 is not available
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0, 1);
+        List<Integer> isr = List.of(0);
+
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(isr)
+            .setPartitionEpoch(partitionEpoch)
+            .setReplicas(replicas));
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(new HashSet<>(List.of(0, 1)), new HashSet<>(partitionMetadata.replicaNodes()));

Review Comment:
   `new HashSet<>(List.of(0, 1))` -> `Set.of(0, 1)`
   
   Same below and in other tests



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1051 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected final long brokerEpoch = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache) {
+            KRaftMetadataCache c = (KRaftMetadataCache) cache;
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = new HashMap<>();

Review Comment:
   We can use Map.of() here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22118:
URL: https://github.com/apache/kafka/pull/22118#discussion_r3185765337


##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1039 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected static final long BROKER_EPOCH = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {

Review Comment:
   I don't think there will be another metadata cache implementation anytime soon, so would you mind refactoring this to simplify it?



##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10451,12 +10451,12 @@ class KafkaApisTest extends Logging {
 
   private def setupBasicMetadataCache(topic: String, numPartitions: Int, numBrokers: Int, topicId: Uuid): Unit = {
     val updateMetadata = createBasicMetadata(topic, numPartitions, 0, numBrokers, topicId)
-    MetadataCacheTest.updateCache(metadataCache, updateMetadata)
+    MetadataCacheTest.updateCache(metadataCache, updateMetadata.asJava)

Review Comment:
   This is another case where one test scope depends on another. We are currently introducing test fixtures in #22201, so I hope we can fix this as a follow-up.



##########
metadata/src/test/java/org/apache/kafka/metadata/MetadataCacheTest.java:
##########
@@ -0,0 +1,1039 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata;
+
+import org.apache.kafka.common.DirectoryId;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponsePartition;
+import org.apache.kafka.common.message.DescribeTopicPartitionsResponseData.DescribeTopicPartitionsResponseTopic;
+import org.apache.kafka.common.message.MetadataResponseData;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.PartitionRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.TopicRecord;
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.protocol.ApiMessage;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.record.internal.RecordBatch;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.image.MetadataDelta;
+import org.apache.kafka.image.MetadataImage;
+import org.apache.kafka.image.MetadataProvenance;
+import org.apache.kafka.server.common.KRaftVersion;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class MetadataCacheTest {
+
+    protected static final long BROKER_EPOCH = 0L;
+
+    public static Stream<MetadataCache> cacheProvider() {
+        return Stream.of(new KRaftMetadataCache(1, () -> KRaftVersion.KRAFT_VERSION_0));
+    }
+
+    public static void updateCache(MetadataCache cache, List<ApiMessage> records) {
+        if (cache instanceof KRaftMetadataCache c) {
+            MetadataImage image = c.currentImage();
+            MetadataImage partialImage = new MetadataImage(
+                new MetadataProvenance(100L, 10, 1000L, true),
+                image.features(),
+                image.cluster(),
+                image.topics(),
+                image.configs(),
+                image.clientQuotas(),
+                image.producerIds(),
+                image.acls(),
+                image.scram(),
+                image.delegationTokens()
+            );
+            MetadataDelta delta = new MetadataDelta.Builder().setImage(partialImage).build();
+            for (ApiMessage record : records) {
+                delta.replay(record);
+            }
+            c.setImage(delta.apply(new MetadataProvenance(100L, 10, 1000L, true)));
+        } else {
+            throw new RuntimeException("Unsupported cache type");
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataNonExistingTopics(MetadataCache cache) {
+        String topic = "topic";
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata = cache.getTopicMetadata(
+            Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT), false, false);
+        assertTrue(topicMetadata.isEmpty());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadata(MetadataCache cache) {
+        String topic0 = "topic-0";
+        String topic1 = "topic-1";
+
+        Map<String, Uuid> topicIds = Map.of(
+            topic0, Uuid.randomUuid(),
+            topic1, Uuid.randomUuid()
+        );
+
+        List<PartitionRecord> partitionStates = List.of(
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(0)
+                .setLeader(0)
+                .setLeaderEpoch(0)
+                .setIsr(List.of(0, 1, 3))
+                .setReplicas(List.of(0, 1, 3)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic0))
+                .setPartitionId(1)
+                .setLeader(1)
+                .setLeaderEpoch(1)
+                .setIsr(List.of(1, 0))
+                .setReplicas(List.of(1, 2, 0, 4)),
+            new PartitionRecord()
+                .setTopicId(topicIds.get(topic1))
+                .setPartitionId(0)
+                .setLeader(2)
+                .setLeaderEpoch(2)
+                .setIsr(List.of(2, 1))
+                .setReplicas(List.of(2, 1, 3))
+        );
+
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId = 0; brokerId <= 4; brokerId++) {
+            String host = "foo-" + brokerId;
+            BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9092)
+                    .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT).value()),
+                new BrokerEndpoint()
+                    .setHost(host)
+                    .setPort(9093)
+                    .setSecurityProtocol(SecurityProtocol.SSL.id)
+                    .setName(ListenerName.forSecurityProtocol(SecurityProtocol.SSL).value())
+            ));
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setEndPoints(endpoints)
+                .setRack("rack1"));
+        }
+        records.add(new TopicRecord().setName(topic0).setTopicId(topicIds.get(topic0)));
+        records.add(new TopicRecord().setName(topic1).setTopicId(topicIds.get(topic1)));
+        records.addAll(partitionStates);
+        updateCache(cache, records);
+
+        for (SecurityProtocol securityProtocol : new SecurityProtocol[]{SecurityProtocol.PLAINTEXT, SecurityProtocol.SSL}) {
+            ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+            for (String topic : new String[]{topic0, topic1}) {
+                List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+                    cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+                assertEquals(1, topicMetadataList.size());
+
+                MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+                assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+                assertEquals(topic, topicMetadata.name());
+                assertEquals(topicIds.get(topic), topicMetadata.topicId());
+
+                List<PartitionRecord> topicPartitionStates = new ArrayList<>();
+                for (PartitionRecord ps : partitionStates) {
+                    if (ps.topicId().equals(topicIds.get(topic))) {
+                        topicPartitionStates.add(ps);
+                    }
+                }
+
+                List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas =
+                    new ArrayList<>(topicMetadata.partitions());
+                partitionMetadatas.sort(Comparator.comparingInt(MetadataResponseData.MetadataResponsePartition::partitionIndex));
+                assertEquals(topicPartitionStates.size(), partitionMetadatas.size(),
+                    "Unexpected partition count for topic " + topic);
+
+                for (int i = 0; i < partitionMetadatas.size(); i++) {
+                    MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(i);
+                    int partitionId = i;
+                    assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+                    assertEquals(partitionId, partitionMetadata.partitionIndex());
+                    PartitionRecord partitionState = topicPartitionStates.stream()
+                        .filter(ps -> ps.partitionId() == partitionId)
+                        .findFirst()
+                        .orElseThrow(() -> new AssertionError("Unable to find partition state for partition " + partitionId));
+                    assertEquals(partitionState.leader(), partitionMetadata.leaderId());
+                    assertEquals(partitionState.leaderEpoch(), partitionMetadata.leaderEpoch());
+                    assertEquals(partitionState.isr(), partitionMetadata.isrNodes());
+                    assertEquals(partitionState.replicas(), partitionMetadata.replicaNodes());
+                }
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionLeaderNotAvailable(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(new BrokerEndpointCollection(List.of(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                )))
+        );
+
+        // leader is not available. expect LEADER_NOT_AVAILABLE for any metadata version.
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, listenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, true);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataPartitionListenerNotAvailableOnLeader(MetadataCache cache) {
+        // when listener name is not present in the metadata cache for a broker, getTopicMetadata should
+        // return LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND errors for old and new versions respectively.
+        ListenerName plaintextListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.PLAINTEXT);
+        ListenerName sslListenerName = ListenerName.forSecurityProtocol(SecurityProtocol.SSL);
+        BrokerEndpointCollection broker0Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value()),
+            new BrokerEndpoint()
+                .setHost("host0")
+                .setPort(9093)
+                .setSecurityProtocol(SecurityProtocol.SSL.id)
+                .setName(sslListenerName.value())
+        ));
+
+        BrokerEndpointCollection broker1Endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("host1")
+                .setPort(9092)
+                .setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)
+                .setName(plaintextListenerName.value())
+        ));
+
+        List<RegisterBrokerRecord> brokers = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(broker0Endpoints),
+            new RegisterBrokerRecord()
+                .setBrokerId(1)
+                .setFenced(false)
+                .setEndPoints(broker1Endpoints)
+        );
+
+        // leader available in cache but listener name not present. expect LISTENER_NOT_FOUND error for new metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LISTENER_NOT_FOUND, true);
+        // leader available in cache but listener name not present. expect LEADER_NOT_AVAILABLE error for old metadata version
+        verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(cache, brokers, sslListenerName,
+            1, Errors.LEADER_NOT_AVAILABLE, false);
+    }
+
+    private void verifyTopicMetadataPartitionLeaderOrEndpointNotAvailable(
+        MetadataCache cache,
+        List<RegisterBrokerRecord> brokers,
+        ListenerName listenerName,
+        int leader,
+        Errors expectedError,
+        boolean errorUnavailableListeners
+    ) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> records = new ArrayList<>(brokers);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+
+        int leaderEpoch = 1;
+        int partitionEpoch = 3;
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setPartitionEpoch(partitionEpoch)
+            .setLeader(leader)
+            .setLeaderEpoch(leaderEpoch)
+            .setIsr(List.of(0))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, errorUnavailableListeners);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(expectedError.code(), partitionMetadata.errorCode());
+        assertFalse(partitionMetadata.isrNodes().isEmpty());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataReplicaNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        int partitionEpoch = 3;
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+        BrokerEndpointCollection endPoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ));
+
+        // replica 1 is not available
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0, 1);
+        List<Integer> isr = List.of(0);
+
+        List<ApiMessage> records = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setFenced(false)
+                .setEndPoints(endPoints),
+            new TopicRecord().setName(topic).setTopicId(topicId),
+            new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(0)
+                .setLeader(leader)
+                .setLeaderEpoch(leaderEpoch)
+                .setIsr(isr)
+                .setPartitionEpoch(partitionEpoch)
+                .setReplicas(replicas)
+        );
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(List.of(0, 1), partitionMetadata.replicaNodes());
+        assertEquals(List.of(0), partitionMetadata.isrNodes());
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(List.of(0), partitionMetadataWithError.replicaNodes());
+        assertEquals(List.of(0), partitionMetadataWithError.isrNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataIsrNotAvailable(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        BrokerEndpointCollection endpoints = new BrokerEndpointCollection(List.of(
+            new BrokerEndpoint()
+                .setHost("foo")
+                .setPort(9092)
+                .setSecurityProtocol(securityProtocol.id)
+                .setName(listenerName.value())
+        ));
+
+        // isr member 1 is not a registered broker
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+        List<ApiMessage> records = List.of(
+            new RegisterBrokerRecord()
+                .setBrokerId(0)
+                .setRack("rack1")
+                .setFenced(false)
+                .setEndPoints(endpoints),
+            new TopicRecord().setName(topic).setTopicId(topicId),
+            new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(0)
+                .setLeader(leader)
+                .setLeaderEpoch(leaderEpoch)
+                .setIsr(isr)
+                .setReplicas(replicas)
+        );
+        updateCache(cache, records);
+
+        // Validate errorUnavailableEndpoints = false
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadataList =
+            cache.getTopicMetadata(Set.of(topic), listenerName, false, false);
+        assertEquals(1, topicMetadataList.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadata = topicMetadataList.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadata.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatas = topicMetadata.partitions();
+        assertEquals(1, partitionMetadatas.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadata = partitionMetadatas.get(0);
+        assertEquals(0, partitionMetadata.partitionIndex());
+        assertEquals(Errors.NONE.code(), partitionMetadata.errorCode());
+        assertEquals(List.of(0), partitionMetadata.replicaNodes());
+        assertEquals(List.of(0, 1), partitionMetadata.isrNodes());
+
+        // Validate errorUnavailableEndpoints = true
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadatasWithError =
+            cache.getTopicMetadata(Set.of(topic), listenerName, true, false);
+        assertEquals(1, topicMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponseTopic topicMetadataWithError = topicMetadatasWithError.get(0);
+        assertEquals(Errors.NONE.code(), topicMetadataWithError.errorCode());
+
+        List<MetadataResponseData.MetadataResponsePartition> partitionMetadatasWithError = topicMetadataWithError.partitions();
+        assertEquals(1, partitionMetadatasWithError.size());
+
+        MetadataResponseData.MetadataResponsePartition partitionMetadataWithError = partitionMetadatasWithError.get(0);
+        assertEquals(0, partitionMetadataWithError.partitionIndex());
+        assertEquals(Errors.REPLICA_NOT_AVAILABLE.code(), partitionMetadataWithError.errorCode());
+        assertEquals(List.of(0), partitionMetadataWithError.replicaNodes());
+        assertEquals(List.of(0), partitionMetadataWithError.isrNodes());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getTopicMetadataWithNonSupportedSecurityProtocol(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+
+        RegisterBrokerRecord broker = new RegisterBrokerRecord()
+            .setBrokerId(0)
+            .setRack("")
+            .setEndPoints(new BrokerEndpointCollection(List.of(
+                new BrokerEndpoint()
+                    .setHost("foo")
+                    .setPort(9092)
+                    .setSecurityProtocol(securityProtocol.id)
+                    .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+            )));
+
+        TopicRecord topicRecord = new TopicRecord().setName(topic).setTopicId(topicId);
+
+        int leader = 0;
+        int leaderEpoch = 0;
+        List<Integer> replicas = List.of(0);
+        List<Integer> isr = List.of(0, 1);
+
+        List<ApiMessage> records = List.of(
+            broker,
+            topicRecord,
+            new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(0)
+                .setLeader(leader)
+                .setLeaderEpoch(leaderEpoch)
+                .setIsr(isr)
+                .setReplicas(replicas)
+        );
+        updateCache(cache, records);
+
+        List<MetadataResponseData.MetadataResponseTopic> topicMetadata =
+            cache.getTopicMetadata(Set.of(topic), ListenerName.forSecurityProtocol(SecurityProtocol.SSL), false, false);
+        assertEquals(1, topicMetadata.size());
+        assertEquals(1, topicMetadata.get(0).partitions().size());
+        assertEquals(RecordBatch.NO_PARTITION_LEADER_EPOCH, topicMetadata.get(0).partitions().get(0).leaderId());
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void getAliveBrokersShouldNotBeMutatedByUpdateCache(MetadataCache cache) {
+        String topic = "topic";
+        Uuid topicId = Uuid.randomUuid();
+        List<ApiMessage> topicRecords = List.of(
+            new TopicRecord().setName(topic).setTopicId(topicId));
+
+        List<Integer> initialBrokerIds = List.of(0, 1, 2);
+        updateCacheWithBrokers(cache, initialBrokerIds, topicId, topicRecords);
+        // This should not change the alive brokers
+        updateCacheWithBrokers(cache, List.of(0, 1, 2, 3), topicId, topicRecords);
+        for (int brokerId : initialBrokerIds) {
+            assertTrue(cache.hasAliveBroker(brokerId));
+        }
+    }
+
+    private void updateCacheWithBrokers(MetadataCache cache, List<Integer> brokerIds,
+                                         Uuid topicId, List<ApiMessage> topicRecords) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        List<ApiMessage> records = new ArrayList<>();
+        for (int brokerId : brokerIds) {
+            records.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setRack("")
+                .setFenced(false)
+                .setBrokerEpoch(BROKER_EPOCH)
+                .setEndPoints(new BrokerEndpointCollection(List.of(
+                    new BrokerEndpoint()
+                        .setHost("foo")
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(ListenerName.forSecurityProtocol(securityProtocol).value())
+                ))));
+        }
+        records.addAll(topicRecords);
+        records.add(new PartitionRecord()
+            .setTopicId(topicId)
+            .setPartitionId(0)
+            .setLeader(0)
+            .setLeaderEpoch(0)
+            .setIsr(List.of(0, 1))
+            .setReplicas(List.of(0)));
+        updateCache(cache, records);
+    }
+
+    @ParameterizedTest
+    @MethodSource("cacheProvider")
+    public void testGetPartitionReplicaEndpoints(MetadataCache cache) {
+        SecurityProtocol securityProtocol = SecurityProtocol.PLAINTEXT;
+        ListenerName listenerName = ListenerName.forSecurityProtocol(securityProtocol);
+
+        // Set up broker data for the metadata cache
+        int numBrokers = 10;
+        int fencedBrokerId = numBrokers / 3;
+        List<RegisterBrokerRecord> brokerRecords = new ArrayList<>();
+        for (int brokerId = 0; brokerId < numBrokers; brokerId++) {
+            brokerRecords.add(new RegisterBrokerRecord()
+                .setBrokerId(brokerId)
+                .setFenced(brokerId == fencedBrokerId)
+                .setRack("rack" + (brokerId % 3))
+                .setEndPoints(new BrokerEndpointCollection(List.of(
+                    new BrokerEndpoint()
+                        .setHost("foo" + brokerId)
+                        .setPort(9092)
+                        .setSecurityProtocol(securityProtocol.id)
+                        .setName(listenerName.value())
+                ))));
+        }
+
+        // Set up a single topic (with many partitions) for the metadata cache
+        String topic = "many-partitions-topic";
+        Uuid topicId = Uuid.randomUuid();
+
+        // Set up a number of partitions such that each different combination of
+        // $replicationFactor brokers is made a replica set for exactly one partition
+        int replicationFactor = 3;
+        List<List<Integer>> replicaSets = getAllReplicaSets(numBrokers, replicationFactor);
+        int numPartitions = replicaSets.size();
+
+        List<PartitionRecord> partitionRecords = new ArrayList<>();
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            List<Integer> replicas = replicaSets.get(partitionId);
+            List<Integer> nonFencedReplicas = new ArrayList<>();
+            for (Integer id : replicas) {
+                if (id != fencedBrokerId) nonFencedReplicas.add(id);
+            }
+            partitionRecords.add(new PartitionRecord()
+                .setTopicId(topicId)
+                .setPartitionId(partitionId)
+                .setReplicas(replicas)
+                .setLeader(replicas.get(0))
+                .setIsr(nonFencedReplicas)
+                .setEligibleLeaderReplicas(nonFencedReplicas));
+        }
+
+        // Load the prepared data in the metadata cache
+        List<ApiMessage> records = new ArrayList<>(brokerRecords);
+        records.add(new TopicRecord().setName(topic).setTopicId(topicId));
+        records.addAll(partitionRecords);
+        updateCache(cache, records);
+
+        for (int partitionId = 0; partitionId < numPartitions; partitionId++) {
+            TopicPartition tp = new TopicPartition(topic, partitionId);
+            Map<Integer, Node> brokerIdToNodeMap =
+                cache.getPartitionReplicaEndpoints(tp, listenerName);
+            Set<Integer> replicaSet = brokerIdToNodeMap.keySet();
+            Set<Integer> expectedReplicaSet = new HashSet<>(partitionRecords.get(partitionId).replicas());
+
+            // Verify that we have endpoints for exactly the non-fenced brokers of the replica set
+            if (expectedReplicaSet.contains(fencedBrokerId)) {
+                Set<Integer> replicaSetPlusFenced = new HashSet<>(replicaSet);
+                replicaSetPlusFenced.add(fencedBrokerId);
+                assertEquals(expectedReplicaSet, replicaSetPlusFenced,
+                    "Unexpected partial replica set for partition " + partitionId);
+            } else {
+                assertEquals(expectedReplicaSet, replicaSet,
+                    "Unexpected replica set for partition " + partitionId);
+            }
+
+            // Verify that the endpoint data for each non-fenced replica is as expected
+            for (int brokerId : replicaSet) {
+                Node brokerNode = brokerIdToNodeMap.get(brokerId);
+                if (brokerNode == null) {
+                    throw new AssertionError("No brokerNode for broker " + brokerId + " and partition " + partitionId);
+                }
+                RegisterBrokerRecord expectedBroker = brokerRecords.get(brokerId);
+                BrokerEndpoint expectedEndpoint = expectedBroker.endPoints().find(listenerName.value());
+                assertEquals(expectedEndpoint.host(), brokerNode.host(),
+                    "Unexpected host for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedEndpoint.port(), brokerNode.port(),
+                    "Unexpected port for broker " + brokerId + " and partition " + partitionId);
+                assertEquals(expectedBroker.rack(), brokerNode.rack(),
+                    "Unexpected rack for broker " + brokerId + " and partition " + partitionId);
+            }
+        }
+
+        TopicPartition tp = new TopicPartition(topic, numPartitions);
+        Map<Integer, Node> brokerIdToNodeMap =
+            cache.getPartitionReplicaEndpoints(tp, listenerName);
+        assertTrue(brokerIdToNodeMap.isEmpty());
+    }
+
+    private static List<List<Integer>> getAllReplicaSets(int numBrokers, int replicationFactor) {

Review Comment:
   This part is a bit hard to follow. Would you mind adding some comments to clarify the logic?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #22118:
URL: https://github.com/apache/kafka/pull/22118


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

Posted by "FrankYang0529 (via GitHub)" <gi...@apache.org>.
FrankYang0529 commented on PR #22118:
URL: https://github.com/apache/kafka/pull/22118#issuecomment-4377200843

   @chia7712 Thanks for the review. I add comment to `getAllReplicaSets` and remove `cacheProvider`. I will use test fixtures in the followup PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19009: Move MetadataCacheTest to metadata module [kafka]

Posted by "FrankYang0529 (via GitHub)" <gi...@apache.org>.
FrankYang0529 commented on PR #22118:
URL: https://github.com/apache/kafka/pull/22118#issuecomment-4370179263

   @mimaison Sorry for the late reply. I addressed all comments. Could you review again? Thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org