You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Randall Hauch (Jira)" <ji...@apache.org> on 2021/02/12 18:07:00 UTC
[jira] [Comment Edited] (KAFKA-12326) MM2 fails to start to due
missing required configuration "bootstrap.servers"
[ https://issues.apache.org/jira/browse/KAFKA-12326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283893#comment-17283893 ]
Randall Hauch edited comment on KAFKA-12326 at 2/12/21, 6:06 PM:
-----------------------------------------------------------------
Yeah, the fix in KAFKA-10021 definitely caused this problem with the MirrorMaker 2 executable. I've verified that the *MirrorMaker 2 connector and Connect are unaffected*.
I've replicated the problem building a local 2.6.2 snapshot archive, installing it, and then running the Kafka quickstart to create and populate a `quickstart` topic:
{code}
$ bin/zookeeper-server-start.sh config/zookeeper.properties
$ bin/kafka-server-start.sh config/server.properties
$ bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
$ bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
This is my first event
This is my second event
{code}
I then changed the local `config/connect-mirror-maker.properties` configuration file the local installation to reference the local Kafka broker and to replicate the `quickstart` topic:
{code}
clusters = A, B
A.bootstrap.servers = localhost:9092
B.bootstrap.servers = localhost:9092
A->B.enabled = true
A->B.topics = quickstart-events
B->A.enabled = false
...
{code}
and then running MirrorMaker 2 executable:
{code}
$ bin/connect-mirror-maker.sh config/connect-mirror-maker.properties
{code}
Doing this with the current `2.6` branch resulted in the exception that [~dajac] mentioned.
There is a simple one-line fix for the `MirrorMaker` class that will pass the correct worker properties to the new `SharedTopicAdmin` instance.
was (Author: rhauch):
Yeah, the fix in KAFKA-10021 definitely caused this problem with the MirrorMaker 2 executable. I've verified that the *MirrorMaker 2 connector and Connect are unaffected*.
I've replicated the problem building a local 2.6.2 snapshot archive, installing it, and then running the Kafka quickstart to create and populate a `quickstart` topic:
{code}
$ bin/zookeeper-server-start.sh config/zookeeper.properties
$ bin/kafka-server-start.sh config/server.properties
{code}
$ bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
$ bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
This is my first event
This is my second event
{code}
I then changed the local `config/connect-mirror-maker.properties` configuration file the local installation to reference the local Kafka broker and to replicate the `quickstart` topic:
{code}
clusters = A, B
A.bootstrap.servers = localhost:9092
B.bootstrap.servers = localhost:9092
A->B.enabled = true
A->B.topics = quickstart-events
B->A.enabled = false
...
{code}
and then running MirrorMaker 2 executable:
{code}
$ bin/connect-mirror-maker.sh config/connect-mirror-maker.properties
{code}
Doing this with the current `2.6` branch resulted in the exception that [~dajac] mentioned.
There is a simple one-line fix for the `MirrorMaker` class that will pass the correct worker properties to the new `SharedTopicAdmin` instance.
> MM2 fails to start to due missing required configuration "bootstrap.servers"
> ----------------------------------------------------------------------------
>
> Key: KAFKA-12326
> URL: https://issues.apache.org/jira/browse/KAFKA-12326
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.5.2, 2.8.0, 2.7.1, 2.6.2
> Reporter: David Jacot
> Assignee: Randall Hauch
> Priority: Blocker
> Fix For: 2.5.2, 2.8.0, 2.7.1, 2.6.2
>
>
> I just tried to mirror a topic with MM2 between two clusters with a really simple configuration. Basically, the one provided in the repo with the correct `bootstrap.servers`.
> MM2 fails with the following error:
> {noformat}
> [2021-02-12 17:05:50,389] ERROR [Worker clientId=connect-1, groupId=B-mm2] Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder:324)
> org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.
> at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:493)
> at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:483)
> at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
> at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
> at org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:233)
> at org.apache.kafka.clients.admin.Admin.create(Admin.java:145)
> at org.apache.kafka.connect.util.TopicAdmin.<init>(TopicAdmin.java:277)
> at org.apache.kafka.connect.util.SharedTopicAdmin.createAdmin(SharedTopicAdmin.java:143)
> at java.util.concurrent.atomic.AtomicReference.updateAndGet(AtomicReference.java:179)
> at org.apache.kafka.connect.util.SharedTopicAdmin.topicAdmin(SharedTopicAdmin.java:82)
> at org.apache.kafka.connect.util.SharedTopicAdmin.get(SharedTopicAdmin.java:72)
> at org.apache.kafka.connect.util.SharedTopicAdmin.get(SharedTopicAdmin.java:44)
> at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:163)
> at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:136)
> at org.apache.kafka.connect.runtime.Worker.start(Worker.java:197)
> at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:128)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:310)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> It seems that a regression was introduced by [https://github.com/apache/kafka/commit/982ea2f6a471c217c7400a725c9504d9d8348d02]. MM2 starts when the commit is reverted.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)