[jira] [Commented] (KAFKA-12893) MM2 fails to replicate if starting two+ nodes same time

["Daniel Urban (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357372#comment-17357372 ] 

Daniel Urban commented on KAFKA-12893:
--------------------------------------

You might be hitting the issue of MM2 not running the Connect REST Server, thus not supporting follower->leader communication in the cluster.

See KIP-710 ([https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+support+for+distributed+mode+in+dedicated+MirrorMaker+2.0+clusters)] for details.

The way to identify the situation is to check whether the MirrorSourceConnector is hosted on the leader or the follower node. If the leader is hosting it, replication should be fine, if the follower, you probably never get to a point where topics are actually picked up.

> MM2 fails to replicate if starting two+ nodes same time
> -------------------------------------------------------
>
>                 Key: KAFKA-12893
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12893
>             Project: Kafka
>          Issue Type: Bug
>          Components: mirrormaker
>    Affects Versions: 2.8.0
>            Reporter: Tommi Vainikainen
>            Priority: Major
>
> I've observed a situation where starting more than one MM2 node in parallel, MM2 fails to start replication ie. replication flow seems to be stuck without action. I used exactly same mm2.properties file to start only one at a time, and the replication flow was proceeding smoothly.
> In my setup dc1 has topic "mytopic1" and there is a producer with approx 1 msg/sec, and I'm trying to repilcate this to dc2. What I observed is that dc1.mytopic1 is created when initially launching two paraller MM2 instances, but no messages gets written into the topic as I would expect. If I kill MM2 instances, and only start one MM2 node, then MM2 starts replicating the messages in mytopic1.
> My mm2.properties:
> clusters=dc2, dc1
> dc1->dc2.emit.heartbeats.enabled=true
> dc1->dc2.enabled=true
> dc1->dc2.sync.group.offsets.enabled=false
> dc1->dc2.sync.group.offsets.interval.seconds=45
> dc1->dc2.topics=mytopic1
> dc1->dc2.topics.exclude=
> dc1.bootstrap.servers=tvainika-dc1-dev-sandbox.aivencloud.com:12693
> dc1.security.protocol=SSL
> dc1.ssl.keystore.type=PKCS12
> dc1.ssl.keystore.location=dc1/client.keystore.p12
> dc1.ssl.keystore.password=secret
> dc1.ssl.key.password=secret
> dc1.ssl.truststore.location=dc1/client.truststore.jks
> dc1.ssl.truststore.password=secret
> dc2.bootstrap.servers=tvainika-dc2-dev-sandbox.aivencloud.com:12693
> dc2.security.protocol=SSL
> dc2.ssl.keystore.type=PKCS12
> dc2.ssl.keystore.location=dc2/client.keystore.p12
> dc2.ssl.keystore.password=secret
> dc2.ssl.key.password=secret
> dc2.ssl.truststore.location=dc2/client.truststore.jks
> dc2.ssl.truststore.password=secret
> tasks.max=3



--
This message was sent by Atlassian Jira
(v8.3.4#803005)