You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jason Gustafson <ja...@confluent.io> on 2017/07/17 22:41:04 UTC
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
+Users
Thanks for the KIP. I think tracking the rebalance time separately will
help resolve some confusion about the performance results given the
rebalance delay in KIP-134. And it seems generally useful to know how much
overhead is coming from the rebalance in any case.
-Jason
On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> Hi all, I opened up a new KIP<https://cwiki.apache.org/
> confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> counting and showing rebalance time in the output. Be free to leave your
> comments here. Thanks in advance.
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Guozhang Wang <wa...@gmail.com>.
I agree with Jason that we are just adding a new field so the impacted
parser tools maybe limited. This additional information would be very
useful.
Guozhang
On Wed, Jul 19, 2017 at 11:57 AM, Jason Gustafson <ja...@confluent.io>
wrote:
> Ismael, I debated that also, but the main point was to make users aware of
> the rebalance latency (with KIP-134 in mind). I'm guessing no one would
> notice if it required another option. Note that the KIP does preserve the
> existing fields (and in the same order), so if it is parsed as generic csv
> data, it should be fine. But yeah, it could break some dumb parsers. In
> general, I think we should at least allow ourselves compatible changes
> given the output format that we have chosen for a tool.
>
> -Jason
>
> On Wed, Jul 19, 2017 at 7:54 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > I think this is a good chance although it's unfortunate that it's likely
> to
> > break code that is parsing the output of the performance tool. Would it
> > make sense to only enable this if an option is provided?
> >
> > Ismael
> >
> > On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > +Users
> > >
> > > Thanks for the KIP. I think tracking the rebalance time separately will
> > > help resolve some confusion about the performance results given the
> > > rebalance delay in KIP-134. And it seems generally useful to know how
> > much
> > > overhead is coming from the rebalance in any case.
> > >
> > > -Jason
> > >
> > > On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> > >
> > > > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > > > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > > > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > > > counting and showing rebalance time in the output. Be free to leave
> > your
> > > > comments here. Thanks in advance.
> > > >
> > >
> >
>
--
-- Guozhang
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Ismael Juma <is...@juma.me.uk>.
OK, sounds good. Let's just make sure we note this in the upgrade notes.
Ismael
On Wed, Jul 19, 2017 at 11:57 AM, Jason Gustafson <ja...@confluent.io>
wrote:
> Ismael, I debated that also, but the main point was to make users aware of
> the rebalance latency (with KIP-134 in mind). I'm guessing no one would
> notice if it required another option. Note that the KIP does preserve the
> existing fields (and in the same order), so if it is parsed as generic csv
> data, it should be fine. But yeah, it could break some dumb parsers. In
> general, I think we should at least allow ourselves compatible changes
> given the output format that we have chosen for a tool.
>
> -Jason
>
> On Wed, Jul 19, 2017 at 7:54 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > I think this is a good chance although it's unfortunate that it's likely
> to
> > break code that is parsing the output of the performance tool. Would it
> > make sense to only enable this if an option is provided?
> >
> > Ismael
> >
> > On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > +Users
> > >
> > > Thanks for the KIP. I think tracking the rebalance time separately will
> > > help resolve some confusion about the performance results given the
> > > rebalance delay in KIP-134. And it seems generally useful to know how
> > much
> > > overhead is coming from the rebalance in any case.
> > >
> > > -Jason
> > >
> > > On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> > >
> > > > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > > > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > > > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > > > counting and showing rebalance time in the output. Be free to leave
> > your
> > > > comments here. Thanks in advance.
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Ismael Juma <is...@juma.me.uk>.
OK, sounds good. Let's just make sure we note this in the upgrade notes.
Ismael
On Wed, Jul 19, 2017 at 11:57 AM, Jason Gustafson <ja...@confluent.io>
wrote:
> Ismael, I debated that also, but the main point was to make users aware of
> the rebalance latency (with KIP-134 in mind). I'm guessing no one would
> notice if it required another option. Note that the KIP does preserve the
> existing fields (and in the same order), so if it is parsed as generic csv
> data, it should be fine. But yeah, it could break some dumb parsers. In
> general, I think we should at least allow ourselves compatible changes
> given the output format that we have chosen for a tool.
>
> -Jason
>
> On Wed, Jul 19, 2017 at 7:54 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > I think this is a good chance although it's unfortunate that it's likely
> to
> > break code that is parsing the output of the performance tool. Would it
> > make sense to only enable this if an option is provided?
> >
> > Ismael
> >
> > On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > +Users
> > >
> > > Thanks for the KIP. I think tracking the rebalance time separately will
> > > help resolve some confusion about the performance results given the
> > > rebalance delay in KIP-134. And it seems generally useful to know how
> > much
> > > overhead is coming from the rebalance in any case.
> > >
> > > -Jason
> > >
> > > On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> > >
> > > > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > > > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > > > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > > > counting and showing rebalance time in the output. Be free to leave
> > your
> > > > comments here. Thanks in advance.
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Jason Gustafson <ja...@confluent.io>.
Ismael, I debated that also, but the main point was to make users aware of
the rebalance latency (with KIP-134 in mind). I'm guessing no one would
notice if it required another option. Note that the KIP does preserve the
existing fields (and in the same order), so if it is parsed as generic csv
data, it should be fine. But yeah, it could break some dumb parsers. In
general, I think we should at least allow ourselves compatible changes
given the output format that we have chosen for a tool.
-Jason
On Wed, Jul 19, 2017 at 7:54 AM, Ismael Juma <is...@juma.me.uk> wrote:
> I think this is a good chance although it's unfortunate that it's likely to
> break code that is parsing the output of the performance tool. Would it
> make sense to only enable this if an option is provided?
>
> Ismael
>
> On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > +Users
> >
> > Thanks for the KIP. I think tracking the rebalance time separately will
> > help resolve some confusion about the performance results given the
> > rebalance delay in KIP-134. And it seems generally useful to know how
> much
> > overhead is coming from the rebalance in any case.
> >
> > -Jason
> >
> > On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > > counting and showing rebalance time in the output. Be free to leave
> your
> > > comments here. Thanks in advance.
> > >
> >
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Jason Gustafson <ja...@confluent.io>.
Ismael, I debated that also, but the main point was to make users aware of
the rebalance latency (with KIP-134 in mind). I'm guessing no one would
notice if it required another option. Note that the KIP does preserve the
existing fields (and in the same order), so if it is parsed as generic csv
data, it should be fine. But yeah, it could break some dumb parsers. In
general, I think we should at least allow ourselves compatible changes
given the output format that we have chosen for a tool.
-Jason
On Wed, Jul 19, 2017 at 7:54 AM, Ismael Juma <is...@juma.me.uk> wrote:
> I think this is a good chance although it's unfortunate that it's likely to
> break code that is parsing the output of the performance tool. Would it
> make sense to only enable this if an option is provided?
>
> Ismael
>
> On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > +Users
> >
> > Thanks for the KIP. I think tracking the rebalance time separately will
> > help resolve some confusion about the performance results given the
> > rebalance delay in KIP-134. And it seems generally useful to know how
> much
> > overhead is coming from the rebalance in any case.
> >
> > -Jason
> >
> > On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > > counting and showing rebalance time in the output. Be free to leave
> your
> > > comments here. Thanks in advance.
> > >
> >
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Ismael Juma <is...@juma.me.uk>.
I think this is a good chance although it's unfortunate that it's likely to
break code that is parsing the output of the performance tool. Would it
make sense to only enable this if an option is provided?
Ismael
On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io> wrote:
> +Users
>
> Thanks for the KIP. I think tracking the rebalance time separately will
> help resolve some confusion about the performance results given the
> rebalance delay in KIP-134. And it seems generally useful to know how much
> overhead is coming from the rebalance in any case.
>
> -Jason
>
> On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > counting and showing rebalance time in the output. Be free to leave your
> > comments here. Thanks in advance.
> >
>
Re: [DISCUSS] KIP-177 Consumer perf tool should count rebalance time
Posted by Ismael Juma <is...@juma.me.uk>.
I think this is a good chance although it's unfortunate that it's likely to
break code that is parsing the output of the performance tool. Would it
make sense to only enable this if an option is provided?
Ismael
On Mon, Jul 17, 2017 at 3:41 PM, Jason Gustafson <ja...@confluent.io> wrote:
> +Users
>
> Thanks for the KIP. I think tracking the rebalance time separately will
> help resolve some confusion about the performance results given the
> rebalance delay in KIP-134. And it seems generally useful to know how much
> overhead is coming from the rebalance in any case.
>
> -Jason
>
> On Thu, Jul 13, 2017 at 4:15 PM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi all, I opened up a new KIP<https://cwiki.apache.org/
> > confluence/display/ARIES/KIP-177%3A+Consumer+perf+tool+
> > should+count+rebalance+time> (KIP-177) concerning consumer perf tool
> > counting and showing rebalance time in the output. Be free to leave your
> > comments here. Thanks in advance.
> >
>