[jira] [Created] (KAFKA-10795) Automated protocol should expose Map keyed by TopicPartition

["David Jacot (Jira)" <ji...@apache.org>]

David Jacot created KAFKA-10795:
-----------------------------------

             Summary: Automated protocol should expose Map keyed by TopicPartition
                 Key: KAFKA-10795
                 URL: https://issues.apache.org/jira/browse/KAFKA-10795
             Project: Kafka
          Issue Type: Improvement
            Reporter: David Jacot
            Assignee: David Jacot


It is common in the AK protocol to encode topics and partitions. In these cases, we usually have a list/map of topic objects which have a name attribute and a partitions attribute. Then, the partitions attribute is a list/map of partition objects which have at least a partition attribute.

{code:javascript}
{ "name": "Topics", "type": "[]OffsetForLeaderTopicResult", "versions": "0+",
   "about": "Each topic we fetched offsets for.", "fields": [
      { "name": "Topic", "type": "string", "versions": "0+", "entityType": "topicName",
        "mapKey": true, "about": "The topic name." },
      { "name": "Partitions", "type": "[]EpochEndOffset", "versions": "0+",
        "about": "Each partition in the topic we fetched offsets for.", "fields": [
         { "name": "Partition", "type": "int32", "versions": "0+",
          "about": "The partition index." },
          ...
      ]}
    ]}
{code}

The caveat is that most of the internals of AK works with Map<TopicPartition, T> so we have to translate from the automated protocol data structures to the the internal one, and vice-versa.

{code:java}
Map<TopicPartition, EpochEndOffset> endOffsets = new HashMap<>();

for (OffsetForLeaderTopicResult topic : response.data().topics()) {
  for (EpochEndOffset partition : topic.partitions()) {
    TopicPartition topicPartition = new TopicPartition(topic.topic(), partition.partition());
    endOffsets.put(topicPartition, partition);
  }
}
{code}

Ideally, we would like to have the right data structures exposed by the automated protocol.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)