Re: [PR] KAFKA-15283:[1/N] Client support for OffsetCommit with topic ID [kafka]

["lianetm (via GitHub)" <gi...@apache.org>]

lianetm commented on code in PR #19577:
URL: https://github.com/apache/kafka/pull/19577#discussion_r2121188219


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -698,15 +699,22 @@ class OffsetCommitRequestState extends RetriableRequestState {
         }
 
         public NetworkClientDelegate.UnsentRequest toUnsentRequest() {
+            Map<String, Uuid> topicIds = metadata.topicIds();
+            boolean canUseTopicIds = true;
             Map<String, OffsetCommitRequestData.OffsetCommitRequestTopic> requestTopicDataMap = new HashMap<>();
             for (Map.Entry<TopicPartition, OffsetAndMetadata> entry : offsets.entrySet()) {
                 TopicPartition topicPartition = entry.getKey();
                 OffsetAndMetadata offsetAndMetadata = entry.getValue();
+                Uuid topicId = topicIds.getOrDefault(topicPartition.topic(), Uuid.ZERO_UUID);
+                if (topicId.equals(Uuid.ZERO_UUID)) {
+                    canUseTopicIds = false;
+                }
 
                 OffsetCommitRequestData.OffsetCommitRequestTopic topic = requestTopicDataMap
                     .getOrDefault(topicPartition.topic(),
                         new OffsetCommitRequestData.OffsetCommitRequestTopic()
                             .setName(topicPartition.topic())
+                            .setTopicId(topicId)

Review Comment:
   the trick is that the topic ID we're passing here won't be obsolete (we just got it from metadata on line 702). This flow: 
   - we have `topicId1` + topicName:foo
   - client fetches and updates position to p1
   - topic recreated -> `topicId2` + topicName:foo
   - [our commit logic here] get topicId from metadata for foo -> that's `topicId2` now
   - [our commit logic here] commit position p1 for `topicId2` -> boom, this is wrong. The broker will find the topicId2 and the name attached to it (foo), but it's actually a different topic from the one where we had consumed the position p1. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org