[jira] [Resolved] (KAFKA-15985) Mirrormaker 2 offset sync is incomplete

["Philipp Dallig (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philipp Dallig resolved KAFKA-15985.
------------------------------------
    Resolution: Fixed

> Mirrormaker 2 offset sync is incomplete
> ---------------------------------------
>
>                 Key: KAFKA-15985
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15985
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 3.5.1
>            Reporter: Philipp Dallig
>            Priority: Major
>
> We are currently trying to migrate between two Kafka clusters using Mirrormaker2
> new kafka cluster version: 7.5.2-ccs
> old kafka cluster version: kafka_2.13-2.8.0
> The Mirrormaker 2 process runs on the new cluster (target cluster).
> My main problem: The lag in the target cluster is not the same as in the source cluster.
> I have set up a producer and consumer against the old Kafka cluster. If I stop both. The lag in the old Kafka cluster is 0, while it is > 0 in the new Kafka cluster.
> target cluster
> {code}
> GROUP           TOPIC                    PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
> test-sync-5     kafka-replication-test-5 0          3637            3668            31              -               -               -
> {code}
> source cluster
> {code}
> GROUP           TOPIC                    PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
> test-sync-5     kafka-replication-test-5 0          3668            3668            0               -               -               -
> {code}
> MM2 configuration without connection properties.
> {code}
> t-kafka->t-extkafka.enabled = true
> t-kafka->t-extkafka.topics = ops_filebeat, kafka-replication-.*
> t-kafka->t-extkafka.sync.topic.acls.enabled = false
> t-kafka->t-extkafka.sync.group.offsets.enabled = true
> t-kafka->t-extkafka.sync.group.offsets.interval.seconds = 30
> t-kafka->t-extkafka.refresh.groups.interval.seconds = 30
> t-kafka->t-extkafka.offset-syncs.topic.location = target
> t-kafka->t-extkafka.emit.checkpoints.interval.seconds = 30
> t-kafka->t-extkafka.replication.policy.class = org.apache.kafka.connect.mirror.IdentityReplicationPolicy
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)