[GitHub] [kafka] C0urante commented on a change in pull request #10907: KAFKA-10000: Exactly-once support for source connectors (KIP-618)

[GitBox <gi...@apache.org>]

C0urante commented on a change in pull request #10907:
URL: https://github.com/apache/kafka/pull/10907#discussion_r665526680



##########
File path: connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
##########
@@ -337,15 +353,29 @@ private void readToLogEnd() {
                 } else {
                     log.trace("Behind end offset {} for {}; last-read offset is {}",
                             endOffset, topicPartition, lastConsumedOffset);
-                    poll(Integer.MAX_VALUE);
+                    if (topicContainsTransactions) {
+                        // The consumer won't return from its poll method if a transaction is aborted, even though
+                        // its position will advance. So, we poll for at most one second, then give ourselves another
+                        // chance to check whether we've reached the end of the topic.
+                        poll(1000);

Review comment:
       I've filed https://issues.apache.org/jira/browse/KAFKA-12980 and plan to begin work on it. Hopefully we can leverage that improvement here to avoid the `poll(1000)` hack.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org