[jira] [Created] (KAFKA-13988) Mirrormaker 2 auto.offset.reset=latest not working

["Daniel Florek (Jira)" <ji...@apache.org>]

Daniel Florek created KAFKA-13988:
-------------------------------------

             Summary: Mirrormaker 2 auto.offset.reset=latest not working
                 Key: KAFKA-13988
                 URL: https://issues.apache.org/jira/browse/KAFKA-13988
             Project: Kafka
          Issue Type: Bug
         Environment: Source Kafka cluster running on Ubuntu 20
Source Kafka cluster Kafka v0.10

Target Kafka cluster running in AWS MSK
Target Kafka cluster Kafka v2.6.2

Mirrormaker version 3.2.0 running on Ubuntu 20.
            Reporter: Daniel Florek


Hi. 

I have problem setting up mirroring with MM2 from latest offset between 2 clusters. In logs I can see that Consumer that is consuming topics has auto.offset.reset property set to latest. But still topics are read from offset 0. I am using following configuration:

 
{code:java}
clusters = A, B

A.bootstrap.servers = broker-01A:9092
B.bootstrap.servers = broker-01B:9092,broker-02B:9092,broker-03B:9092

replication.policy.class = org.apache.kafka.connect.mirror.IdentityReplicationPolicy

#Enable replication between clusters and define topics which should be replicated
A->B.enabled = true
A->B.topics = .*
A->B.replication.factor=3
A->B.emit.heartbeats.enabled = true
A->B.emit.checkpoints.enabled = true

auto.offset.reset=latest
consumer.auto.offset.reset=latest
A.consumer.auto.offset.reset=latest
B.consumer.auto.offset.reset=latest
refresh.topics.enabled=true

heartbeats.topic.replication.factor=1
checkpoints.topic.replication.factor=1
offset-syncs.topic.replication.factor=1

config.storage.replication.factor = 1
offset.storage.replication.factor = 1
status.storage.replication.factor = 1 {code}

I am using Kafka 3.2.0 for Mirrormaker 2. Source kafka cluster is 1 broker running on EC2 instance in AWS (quite an old version I think 0.10). Target kafka cluster contains 3 brokers running in AWS MSK (version 2.6.2). 

Could you point me what I am doing wrong? Or is this possibly a bug?

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)