[jira] [Created] (KAFKA-14980) MirrorMaker consumers don't get configs prefixed with source.cluster

["Mickael Maison (Jira)" <ji...@apache.org>]

Mickael Maison created KAFKA-14980:
--------------------------------------

             Summary: MirrorMaker consumers don't get configs prefixed with source.cluster
                 Key: KAFKA-14980
                 URL: https://issues.apache.org/jira/browse/KAFKA-14980
             Project: Kafka
          Issue Type: Bug
          Components: mirrormaker
    Affects Versions: 3.5.0
            Reporter: Mickael Maison


As part of KAFKA-14021, we made a change to MirrorConnectorConfig.sourceConsumerConfig() to grab all configs that start with "source.". Previously it was grabbing configs prefixed with "source.cluster.". 

This means existing connector configuration stop working, as configurations such as bootstrap.servers are not passed to source consumers.

For example, the following connector configuration was valid in 3.4 and now makes the connector tasks fail:

{code:json}
{
    "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "name": "source",
    "topics": "test",
    "tasks.max": "30",
    "source.cluster.alias": "one",
    "target.cluster.alias": "two",
    "source.cluster.bootstrap.servers": "localhost:9092",
   "target.cluster.bootstrap.servers": "localhost:29092"
}
{code}


The connector attempts to start source consumers with bootstrap.servers = [] and the task crash with 


{noformat}
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:837)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:671)
	at org.apache.kafka.connect.mirror.MirrorUtils.newConsumer(MirrorUtils.java:59)
	at org.apache.kafka.connect.mirror.MirrorSourceTask.start(MirrorSourceTask.java:103)
	at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:274)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
	at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:75)
	at 
org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:181)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
{noformat}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)