You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by jg...@apache.org on 2019/02/01 22:18:02 UTC
[kafka] branch trunk updated: KAFKA-7873;
Always seek to beginning in KafkaBasedLog (#6203)
This is an automated email from the ASF dual-hosted git repository.
jgus pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new 176ea0d KAFKA-7873; Always seek to beginning in KafkaBasedLog (#6203)
176ea0d is described below
commit 176ea0d0f9b4386cdc0e3b9bab560a11e3d3d516
Author: Randall Hauch <rh...@gmail.com>
AuthorDate: Fri Feb 1 14:17:52 2019 -0800
KAFKA-7873; Always seek to beginning in KafkaBasedLog (#6203)
Explicitly seek KafkaBasedLog’s consumer to the beginning of the topic partitions, rather than potentially use committed offsets (which would be unexpected) if group.id is set or rely upon `auto.offset.reset=earliest` if the group.id is null.
This should not change existing behavior but should remove some potential issues introduced with KIP-287 if `group.id` is not set in the consumer configurations. Note that even if `group.id` is set, we still always want to consume from the beginning.
Reviewers: Jason Gustafson <ja...@confluent.io>
---
.../src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java b/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
index ea9b4c6..9d77d21 100644
--- a/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
+++ b/connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
@@ -146,6 +146,10 @@ public class KafkaBasedLog<K, V> {
partitions.add(new TopicPartition(partition.topic(), partition.partition()));
consumer.assign(partitions);
+ // Always consume from the beginning of all partitions. Necessary to ensure that we don't use committed offsets
+ // when a 'group.id' is specified (if offsets happen to have been committed unexpectedly).
+ consumer.seekToBeginning(partitions);
+
readToLogEnd();
thread = new WorkThread();