You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by jg...@apache.org on 2019/02/01 22:18:02 UTC

[kafka] branch trunk updated: KAFKA-7873; Always seek to beginning in KafkaBasedLog (#6203)

This is an automated email from the ASF dual-hosted git repository.

jgus pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new 176ea0d  KAFKA-7873; Always seek to beginning in KafkaBasedLog (#6203)
176ea0d is described below

commit 176ea0d0f9b4386cdc0e3b9bab560a11e3d3d516
Author: Randall Hauch <rh...@gmail.com>
AuthorDate: Fri Feb 1 14:17:52 2019 -0800

    KAFKA-7873; Always seek to beginning in KafkaBasedLog (#6203)
    
    Explicitly seek KafkaBasedLog’s consumer to the beginning of the topic partitions, rather than potentially use committed offsets (which would be unexpected) if group.id is set or rely upon `auto.offset.reset=earliest` if the group.id is null.
    
    This should not change existing behavior but should remove some potential issues introduced with KIP-287 if `group.id` is not set in the consumer configurations. Note that even if `group.id` is set, we still always want to consume from the beginning.
    
    Reviewers: Jason Gustafson <ja...@confluent.io>
---
 .../src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java    | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java b/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
index ea9b4c6..9d77d21 100644
--- a/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
+++ b/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
@@ -146,6 +146,10 @@ public class KafkaBasedLog<K, V> {
             partitions.add(new TopicPartition(partition.topic(), partition.partition()));
         consumer.assign(partitions);
 
+        // Always consume from the beginning of all partitions. Necessary to ensure that we don't use committed offsets
+        // when a 'group.id' is specified (if offsets happen to have been committed unexpectedly).
+        consumer.seekToBeginning(partitions);
+
         readToLogEnd();
 
         thread = new WorkThread();