You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by vishnu murali <vi...@gmail.com> on 2020/05/02 15:21:27 UTC

Connect-Mirror Error

Hey Guys,

Here By i am posting stack trace occured in the connect-distributed while
giving mirror connector configurations:

*Post*:http://localhost:8083/connectors

*Request json Body:*
{
    "name": "us-west-sourc",
    "config": {
        "connector.class":
"org.apache.kafka.connect.mirror.MirrorSourceConnector",
        "source.cluster.alias": "cluster 9092",
        "target.cluster.alias": "cluster 9091",
        "source.cluster.bootstrap.servers": "localhost:9092",
        "topics": "vis-city"

    }
}

it is saying as bootstrap.servers which has default value is missing?

even if i tried that config in request also it is responding the same...

what may be the problem?

Actually i am trying to copy data from topic in one cluster to another
cluster..

1)By using MirrorMaker command it is possible.

But i want to do the same using connector by giving request and then copy
the data?

What change i need to do?


[2020-05-02 20:40:43,304] ERROR WorkerConnector{id=us-west-sourc} Error
while starting connector (org.apache.kafka.connect.runtime.WorkerConnector)
org.apache.kafka.common.config.ConfigException: Missing required
configuration "bootstrap.servers" which has no default value.
        at
org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:477)
        at
org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:467)
        at
org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
        at
org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
        at
org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:216)
        at org.apache.kafka.clients.admin.Admin.create(Admin.java:71)
        at
org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
        at
org.apache.kafka.connect.mirror.MirrorSourceConnector.start(MirrorSourceConnector.java:115)
        at
org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:110)
        at
org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:135)
        at
org.apache.kafka.connect.runtime.WorkerConnector.transitionTo(WorkerConnector.java:195)
        at
org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:257)
        at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:1190)
        at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1300(DistributedHerder.java:126)
        at
org.apache.kafka.connect.runtime.distributed.DistributedHerder$14.call(DistributedHerder.java:1206)
        at
org.apache.kafka.connect.runtime.distributed.DistributedHerder$14.call(DistributedHerder.java:1202)
        at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:830)

Re: Connect-Mirror Error

Posted by mandeep gandhi <we...@gmail.com>.
Hi,


I looked at the code[0] and looks like you need to specify bootstrap
servers for both source and target. Did you happen to try this as well?

* {
* "name": "MirrorSourceConnector",
* "connector.class":
"org.apache.kafka.connect.mirror.MirrorSourceConnector",
* "replication.factor": "1",
* "source.cluster.alias": "backup",
* "target.cluster.alias": "primary",
* "source.cluster.bootstrap.servers": "vip1:9092",
* "target.cluster.bootstrap.servers": "vip2:9092",
* "topics": ".*test-topic-.*",
* "groups": "consumer-group-.*",
* "emit.checkpoints.interval.seconds": "1",
* "emit.heartbeats.interval.seconds": "1",
* "sync.topic.acls.enabled": "false"
* } [0] -
https://github.com/apache/kafka/blob/8fd967e10cdfbb6e48ef1f590b8902bbf1080a71/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java#L279

On Sat, May 2, 2020 at 8:52 PM vishnu murali <vi...@gmail.com>
wrote:

> Hey Guys,
>
> Here By i am posting stack trace occured in the connect-distributed while
> giving mirror connector configurations:
>
> *Post*:http://localhost:8083/connectors
>
> *Request json Body:*
> {
>     "name": "us-west-sourc",
>     "config": {
>         "connector.class":
> "org.apache.kafka.connect.mirror.MirrorSourceConnector",
>         "source.cluster.alias": "cluster 9092",
>         "target.cluster.alias": "cluster 9091",
>         "source.cluster.bootstrap.servers": "localhost:9092",
>         "topics": "vis-city"
>
>     }
> }
>
> it is saying as bootstrap.servers which has default value is missing?
>
> even if i tried that config in request also it is responding the same...
>
> what may be the problem?
>
> Actually i am trying to copy data from topic in one cluster to another
> cluster..
>
> 1)By using MirrorMaker command it is possible.
>
> But i want to do the same using connector by giving request and then copy
> the data?
>
> What change i need to do?
>
>
> [2020-05-02 20:40:43,304] ERROR WorkerConnector{id=us-west-sourc} Error
> while starting connector (org.apache.kafka.connect.runtime.WorkerConnector)
> org.apache.kafka.common.config.ConfigException: Missing required
> configuration "bootstrap.servers" which has no default value.
>         at
> org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:477)
>         at
> org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:467)
>         at
>
> org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
>         at
>
> org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
>         at
>
> org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:216)
>         at org.apache.kafka.clients.admin.Admin.create(Admin.java:71)
>         at
> org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
>         at
>
> org.apache.kafka.connect.mirror.MirrorSourceConnector.start(MirrorSourceConnector.java:115)
>         at
>
> org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:110)
>         at
>
> org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:135)
>         at
>
> org.apache.kafka.connect.runtime.WorkerConnector.transitionTo(WorkerConnector.java:195)
>         at
> org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:257)
>         at
>
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:1190)
>         at
>
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1300(DistributedHerder.java:126)
>         at
>
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$14.call(DistributedHerder.java:1206)
>         at
>
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$14.call(DistributedHerder.java:1202)
>         at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>         at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>         at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>         at java.base/java.lang.Thread.run(Thread.java:830)
>