kafka mm2 not replicate topic and data

[Calvin Chen <pi...@hotmail.com>]

Hi all

I'm testing Kafka MM2, and noticed topic and data are not replicated, could anyone help, thanks a lot!

My kafka version is kafka_2.13-2.8.0, two kafka cluster: sc2-dev-shared-kafka01 and sc2-dev-shared-kafka02, MM2 processes runs in each node of kafka clusters(3 MM2 processes in kafka01 and 3 MM2 processes in kafka02).

I config MM2 in kafka01 to replicate data to kafk02, I config MM2 in kafka02 to replicate data to kafka01.

MM2 config in kafka01:
clusters=sc2-dev-shared-kafka01,sc2-dev-shared-kafka02
sc2-dev-shared-kafka01.bootstrap.servers=sc2-dev-shared-kafka01a.eng.vmware.com:9092, sc2-dev-shared-kafka01b.eng.vmware.com:9092, sc2-dev-shared-kafka01c.eng.vmware.com:9092
sc2-dev-shared-kafka02.bootstrap.servers=sc2-dev-shared-kafka02a.eng.vmware.com:9092, sc2-dev-shared-kafka02b.eng.vmware.com:9092, sc2-dev-shared-kafka02c.eng.vmware.com:9092

sc2-dev-shared-kafka01->sc2-dev-shared-kafka02.enabled=true
sc2-dev-shared-kafka02->sc2-dev-shared-kafka01.enabled=false
topics=.*
groups=.*
tasks.max=10

checkpoints.topic.replication.factor=2
heartbeats.topic.replication.factor=2
offset-syncs.topic.replication.factor=2

offset.storage.replication.factor=2
status.storage.replication.factor=2
config.storage.replication.factor=2

MM2 config in kafka02:
clusters=sc2-dev-shared-kafka01,sc2-dev-shared-kafka02
sc2-dev-shared-kafka01.bootstrap.servers=sc2-dev-shared-kafka01a.eng.vmware.com:9092, sc2-dev-shared-kafka01b.eng.vmware.com:9092, sc2-dev-shared-kafka01c.eng.vmware.com:9092
sc2-dev-shared-kafka02.bootstrap.servers=sc2-dev-shared-kafka02a.eng.vmware.com:9092, sc2-dev-shared-kafka02b.eng.vmware.com:9092, sc2-dev-shared-kafka02c.eng.vmware.com:9092

sc2-dev-shared-kafka01->sc2-dev-shared-kafka02.enabled=false
sc2-dev-shared-kafka02->sc2-dev-shared-kafka01.enabled=true
topics=.*
groups=.*
tasks.max=10

checkpoints.topic.replication.factor=2
heartbeats.topic.replication.factor=2
offset-syncs.topic.replication.factor=2
offset.storage.replication.factor=2
status.storage.replication.factor=2
config.storage.replication.factor=2

I make MM2 run as service in each kafka node, and service is good, there is no error from MM2 logs.

I create topic-20 and topic-25 in different kafka clusters, I hope those topics will be replicated to remote cluster, but only part of them are replicated, it has been 30 minutes past, I think there should be some problem here, and I see topics list in those two cluster are different, looks like sc2-dev-shared-kafka02 missed some topic which should be replicated from sc2-dev-shared-kafka01.

/mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-topics.sh --zookeeper sc2-dev-shared-kafka01a:2181 --list
__consumer_offsets
_schemas
heartbeats
mm2-configs.sc2-dev-shared-kafka02.internal
mm2-offsets.sc2-dev-shared-kafka02.internal
mm2-status.sc2-dev-shared-kafka02.internal
sc2-dev-shared-kafka02._schemas
sc2-dev-shared-kafka02.checkpoints.internal
sc2-dev-shared-kafka02.heartbeats
sc2-dev-shared-kafka02.topic-25
topic-20
/mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-topics.sh --zookeeper sc2-dev-shared-kafka02a:2181 --list
__consumer_offsets
_schemas
heartbeats
mm2-configs.sc2-dev-shared-kafka01.internal
mm2-offset-syncs.sc2-dev-shared-kafka01.internal
mm2-offsets.sc2-dev-shared-kafka01.internal
mm2-status.sc2-dev-shared-kafka01.internal
topic-25
/mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 >

Also, when I send message to topic-25 in sc2-dev-shared-kafka02, I don't see data is replicated to sc2-dev-shared-kafka01:

/mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-console-producer.sh --bootstrap-server sc2-dev-shared-kafka02a:9092 --topic topic-25
>hello
>do you copy?
>
/mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-console-consumer.sh --bootstrap-server sc2-dev-shared-kafka01a:9092 --topic sc2-dev-shared-kafka02.topic-25 --from-beginning