[GitHub] [kafka] showuon commented on a change in pull request #10644: [KAFKA-12635] auto sync consumer offset 0 when translated offset larger than partition end offset

[GitBox <gi...@apache.org>]

showuon commented on a change in pull request #10644:
URL: https://github.com/apache/kafka/pull/10644#discussion_r794275490



##########
File path: connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java
##########
@@ -243,6 +251,45 @@ private void refreshIdleConsumerGroupOffset() {
             }
         }
     }
+    
+    private void refreshTopicPartitionEndOffset() {
+
+        Map<TopicPartition, Long> topicPartitionOffsets = new HashMap<>();
+        for (Entry<String, Map<TopicPartition, OffsetAndMetadata>> group : getConvertedUpstreamOffset().entrySet()) {
+            Map<TopicPartition, OffsetAndMetadata> convertedUpstreamOffset = group.getValue();
+            for (Entry<TopicPartition, OffsetAndMetadata> offset : convertedUpstreamOffset.entrySet()) {
+                topicPartitionOffsets.put(offset.getKey(), offset.getValue().offset());
+            }
+        }
+        if (topicPartitionOffsets.keySet().size() != 0) {
+            KafkaConsumer<byte[], byte[]> consumer = MirrorUtils.newConsumer(config.targetConsumerConfig());
+            consumer.assign(topicPartitionOffsets.keySet());
+            topicPartitionOffsets.forEach(consumer::seek);
+            consumer.poll(Duration.ofMillis(500));
+            topicPartitionEndOffset = consumer.endOffsets(topicPartitionOffsets.keySet());
+            consumer.close();
+        }

Review comment:
       `targetAdminClient.listOffsets()` is easier and more efficient to get the LEO.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org