What to use to copy data from one kafka to another ?

[JP MB <jo...@gmail.com>]

Hi guys,
My use case is simply copying data from one Kafka to another. When
searching on google, the immediate answer seems to be Mirror Maker, so we
jumped to the most recent version MM2.

The thing is I don't want active/active replication, the consumers from
cluster A will be different from the consumers from cluster B. MM2 creates
several topics in the source cluster(heartbeats, mm2-offsets, mm2-status,
mm2-configs), which is undesired for us because we don't want to pollute
the source.

Is there a way to disable this? If no, what should I use?

Regards,
José Brandão

Re: What to use to copy data from one kafka to another ?

[JP MB <jo...@gmail.com>]

Yes, I saw that it would fit my use case but that brings another question.
Is MM1 deprecated or it will be discontinued ?

Regards

Em qui., 14 de mai. de 2020 às 14:46, Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> escreveu:

> You could use Mirror Maker 1, it's a basic "consume from topic A in cluster
> X, produce to topic A in cluster Y" app.
>
> On Fri, 15 May 2020, 12:13 am JP MB, <jo...@gmail.com> wrote:
>
> > Hi guys,
> > My use case is simply copying data from one Kafka to another. When
> > searching on google, the immediate answer seems to be Mirror Maker, so we
> > jumped to the most recent version MM2.
> >
> > The thing is I don't want active/active replication, the consumers from
> > cluster A will be different from the consumers from cluster B. MM2
> creates
> > several topics in the source cluster(heartbeats, mm2-offsets, mm2-status,
> > mm2-configs), which is undesired for us because we don't want to pollute
> > the source.
> >
> > Is there a way to disable this? If no, what should I use?
> >
> > Regards,
> > José Brandão
> >
>

Re: What to use to copy data from one kafka to another ?

[Liam Clarke-Hutchinson <li...@adscale.co.nz>]

You could use Mirror Maker 1, it's a basic "consume from topic A in cluster
X, produce to topic A in cluster Y" app.

On Fri, 15 May 2020, 12:13 am JP MB, <jo...@gmail.com> wrote:

> Hi guys,
> My use case is simply copying data from one Kafka to another. When
> searching on google, the immediate answer seems to be Mirror Maker, so we
> jumped to the most recent version MM2.
>
> The thing is I don't want active/active replication, the consumers from
> cluster A will be different from the consumers from cluster B. MM2 creates
> several topics in the source cluster(heartbeats, mm2-offsets, mm2-status,
> mm2-configs), which is undesired for us because we don't want to pollute
> the source.
>
> Is there a way to disable this? If no, what should I use?
>
> Regards,
> José Brandão
>