Re: Mirror Maker 2 - High Throughput Identity Mirroring

[Antón Rodríguez Yuste <an...@galiglobal.com>]

Hi Ryanne,

Is there a PR or code I could take a look at or just the KIP? "shallow
mirroring" seems very interesting for our use cases, and I would like to
evaluate it internally.

Thanks,

Antón

On Thu, Jul 29, 2021 at 7:02 PM Ryanne Dolan <ry...@gmail.com> wrote:

> Jamie, this would depend on KIP-712 (or similar) aka "shallow mirroring".
> This is a work in progress, but I'm optimistic it'll happen at some point.
>
> ftr, "IdentityReplicationPolicy" has landed for the upcoming release, tho
> "identity" in that context just means that topics aren't renamed.
>
> Ryanne
>
> On Thu, Jul 29, 2021, 11:37 AM Jamie <ja...@aol.co.uk.invalid> wrote:
>
> > Hi All,
> > This blog post:
> > https://blog.cloudera.com/a-look-inside-kafka-mirrormaker-2/ mentions
> > that "High Throughput Identity Mirroring" (when the compression is the
> same
> > in both the source and destination cluster) will soon be coming to MM2
> > which would avoid the MM2 consumer decompressing the data only for the
> MM2
> > producer to then re-compress it again.
> > Has this feature been implemented yet in MM2?
> > Many Thanks,
> > Jamie
>

Re: Mirror Maker 2 - High Throughput Identity Mirroring

[Ryanne Dolan <ry...@gmail.com>]

Henry Cai has a PR somewhere.

Ryanne

On Wed, Mar 2, 2022, 3:36 AM Antón Rodríguez Yuste <an...@galiglobal.com>
wrote:

> Hi Ryanne,
>
> Is there a PR or code I could take a look at or just the KIP? "shallow
> mirroring" seems very interesting for our use cases, and I would like to
> evaluate it internally.
>
> Thanks,
>
> Antón
>
> On Thu, Jul 29, 2021 at 7:02 PM Ryanne Dolan <ry...@gmail.com>
> wrote:
>
> > Jamie, this would depend on KIP-712 (or similar) aka "shallow mirroring".
> > This is a work in progress, but I'm optimistic it'll happen at some
> point.
> >
> > ftr, "IdentityReplicationPolicy" has landed for the upcoming release, tho
> > "identity" in that context just means that topics aren't renamed.
> >
> > Ryanne
> >
> > On Thu, Jul 29, 2021, 11:37 AM Jamie <ja...@aol.co.uk.invalid>
> wrote:
> >
> > > Hi All,
> > > This blog post:
> > > https://blog.cloudera.com/a-look-inside-kafka-mirrormaker-2/ mentions
> > > that "High Throughput Identity Mirroring" (when the compression is the
> > same
> > > in both the source and destination cluster) will soon be coming to MM2
> > > which would avoid the MM2 consumer decompressing the data only for the
> > MM2
> > > producer to then re-compress it again.
> > > Has this feature been implemented yet in MM2?
> > > Many Thanks,
> > > Jamie
> >
>

Re: Mirror Maker 2 - High Throughput Identity Mirroring

[Ryanne Dolan <ry...@gmail.com>]

Henry Cai has a PR somewhere.

Ryanne

On Wed, Mar 2, 2022, 3:36 AM Antón Rodríguez Yuste <an...@galiglobal.com>
wrote:

> Hi Ryanne,
>
> Is there a PR or code I could take a look at or just the KIP? "shallow
> mirroring" seems very interesting for our use cases, and I would like to
> evaluate it internally.
>
> Thanks,
>
> Antón
>
> On Thu, Jul 29, 2021 at 7:02 PM Ryanne Dolan <ry...@gmail.com>
> wrote:
>
> > Jamie, this would depend on KIP-712 (or similar) aka "shallow mirroring".
> > This is a work in progress, but I'm optimistic it'll happen at some
> point.
> >
> > ftr, "IdentityReplicationPolicy" has landed for the upcoming release, tho
> > "identity" in that context just means that topics aren't renamed.
> >
> > Ryanne
> >
> > On Thu, Jul 29, 2021, 11:37 AM Jamie <ja...@aol.co.uk.invalid>
> wrote:
> >
> > > Hi All,
> > > This blog post:
> > > https://blog.cloudera.com/a-look-inside-kafka-mirrormaker-2/ mentions
> > > that "High Throughput Identity Mirroring" (when the compression is the
> > same
> > > in both the source and destination cluster) will soon be coming to MM2
> > > which would avoid the MM2 consumer decompressing the data only for the
> > MM2
> > > producer to then re-compress it again.
> > > Has this feature been implemented yet in MM2?
> > > Many Thanks,
> > > Jamie
> >
>