You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mickael Maison <mi...@gmail.com> on 2020/04/16 16:44:25 UTC
[DISCUSS] KIP-597: MirrorMaker2 internal topics Formatters
Hi,
I have submitted a small KIP to provide Formatters for the
MirrorMaker2 internal topics.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters
Looking forward to your feedback. Thanks
Re: [DISCUSS] KIP-597: MirrorMaker2 internal topics Formatters
Posted by Christopher Egerton <ch...@confluent.io>.
Hi Mickael,
Thanks! This looks great, +1 non-binding.
Cheers,
Chris
On Sun, May 3, 2020 at 4:13 AM Mickael Maison <mi...@gmail.com>
wrote:
> Thanks Chris for the feedback.
>
> 1. I've added sample output for the 3 formatters
> 2. Good idea, instead of deleting the existing trait, let's mark it as
> deprecated and make it extend the new interface. I've updated to the
> KIP.
>
>
> I've also opened a draft PR: https://github.com/apache/kafka/pull/8604
>
> On Fri, Apr 24, 2020 at 12:17 AM Christopher Egerton
> <ch...@confluent.io> wrote:
> >
> > Hi Mickael,
> >
> > The KIP looks great and the additional formatters seem like excellent
> tools
> > for debugging and diving into the internals of Mirror Maker 2.0. I've
> got a
> > few thoughts on the proposed changes:
> >
> > 1. Would it be possible to provide examples for what the actual output
> > might look like when the console consumer is run with the newly-proposed
> > formatters, and the commands users could run to make that happen?
> >
> > 2. Even though the existing MessageFormatter trait is currently in the
> core
> > project, it's still mentioned in the usage for the console consumer, and
> > it's possible that users may have already implemented their own
> formatters
> > based on this information, which is part of the public API. At the same
> > time, if we do want to make this an official part of the public API for
> > Kafka, I do agree that it'd be best to make that interface available in
> the
> > clients project. What do you think about retaining the existing
> > "kafka.common.MessageFormatter" trait, but making it extend from the
> > newly-proposed "org.apache.kafka.common.MessageFormatter" interface? That
> > way, any existing tools that extend from the old MessageFormatter trait
> > will still work, and we can have our shiny new interface in the clients
> > package available for anyone who wants to roll their own in the future.
> >
> > Looking forward to your thoughts on this!
> >
> > Cheers,
> >
> > Chris
> >
> > On Thu, Apr 16, 2020 at 10:21 AM Ryanne Dolan <ry...@gmail.com>
> wrote:
> >
> > > Thanks Mickael, this will be very useful.
> > >
> > > Ryanne
> > >
> > > On Thu, Apr 16, 2020 at 11:44 AM Mickael Maison <
> mickael.maison@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I have submitted a small KIP to provide Formatters for the
> > > > MirrorMaker2 internal topics.
> > > >
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters
> > > >
> > > > Looking forward to your feedback. Thanks
> > > >
> > >
>
Re: [DISCUSS] KIP-597: MirrorMaker2 internal topics Formatters
Posted by Mickael Maison <mi...@gmail.com>.
Thanks Chris for the feedback.
1. I've added sample output for the 3 formatters
2. Good idea, instead of deleting the existing trait, let's mark it as
deprecated and make it extend the new interface. I've updated to the
KIP.
I've also opened a draft PR: https://github.com/apache/kafka/pull/8604
On Fri, Apr 24, 2020 at 12:17 AM Christopher Egerton
<ch...@confluent.io> wrote:
>
> Hi Mickael,
>
> The KIP looks great and the additional formatters seem like excellent tools
> for debugging and diving into the internals of Mirror Maker 2.0. I've got a
> few thoughts on the proposed changes:
>
> 1. Would it be possible to provide examples for what the actual output
> might look like when the console consumer is run with the newly-proposed
> formatters, and the commands users could run to make that happen?
>
> 2. Even though the existing MessageFormatter trait is currently in the core
> project, it's still mentioned in the usage for the console consumer, and
> it's possible that users may have already implemented their own formatters
> based on this information, which is part of the public API. At the same
> time, if we do want to make this an official part of the public API for
> Kafka, I do agree that it'd be best to make that interface available in the
> clients project. What do you think about retaining the existing
> "kafka.common.MessageFormatter" trait, but making it extend from the
> newly-proposed "org.apache.kafka.common.MessageFormatter" interface? That
> way, any existing tools that extend from the old MessageFormatter trait
> will still work, and we can have our shiny new interface in the clients
> package available for anyone who wants to roll their own in the future.
>
> Looking forward to your thoughts on this!
>
> Cheers,
>
> Chris
>
> On Thu, Apr 16, 2020 at 10:21 AM Ryanne Dolan <ry...@gmail.com> wrote:
>
> > Thanks Mickael, this will be very useful.
> >
> > Ryanne
> >
> > On Thu, Apr 16, 2020 at 11:44 AM Mickael Maison <mi...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I have submitted a small KIP to provide Formatters for the
> > > MirrorMaker2 internal topics.
> > >
> > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters
> > >
> > > Looking forward to your feedback. Thanks
> > >
> >
Re: [DISCUSS] KIP-597: MirrorMaker2 internal topics Formatters
Posted by Christopher Egerton <ch...@confluent.io>.
Hi Mickael,
The KIP looks great and the additional formatters seem like excellent tools
for debugging and diving into the internals of Mirror Maker 2.0. I've got a
few thoughts on the proposed changes:
1. Would it be possible to provide examples for what the actual output
might look like when the console consumer is run with the newly-proposed
formatters, and the commands users could run to make that happen?
2. Even though the existing MessageFormatter trait is currently in the core
project, it's still mentioned in the usage for the console consumer, and
it's possible that users may have already implemented their own formatters
based on this information, which is part of the public API. At the same
time, if we do want to make this an official part of the public API for
Kafka, I do agree that it'd be best to make that interface available in the
clients project. What do you think about retaining the existing
"kafka.common.MessageFormatter" trait, but making it extend from the
newly-proposed "org.apache.kafka.common.MessageFormatter" interface? That
way, any existing tools that extend from the old MessageFormatter trait
will still work, and we can have our shiny new interface in the clients
package available for anyone who wants to roll their own in the future.
Looking forward to your thoughts on this!
Cheers,
Chris
On Thu, Apr 16, 2020 at 10:21 AM Ryanne Dolan <ry...@gmail.com> wrote:
> Thanks Mickael, this will be very useful.
>
> Ryanne
>
> On Thu, Apr 16, 2020 at 11:44 AM Mickael Maison <mi...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have submitted a small KIP to provide Formatters for the
> > MirrorMaker2 internal topics.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters
> >
> > Looking forward to your feedback. Thanks
> >
>
Re: [DISCUSS] KIP-597: MirrorMaker2 internal topics Formatters
Posted by Ryanne Dolan <ry...@gmail.com>.
Thanks Mickael, this will be very useful.
Ryanne
On Thu, Apr 16, 2020 at 11:44 AM Mickael Maison <mi...@gmail.com>
wrote:
> Hi,
>
> I have submitted a small KIP to provide Formatters for the
> MirrorMaker2 internal topics.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters
>
> Looking forward to your feedback. Thanks
>