You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Guozhang Wang <wa...@gmail.com> on 2017/03/01 07:12:21 UTC
Re: Chatty StreamThread commit messages
Hey Steven,
That is a good question, and I think your proposal makes sense. Could you
file a JIRA for this change to keep track of it?
Guozhang
On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
sschlansker@opentable.com> wrote:
> Hi everyone, running with Kafka Streams 0.10.2.0, I see this every commit
> interval:
>
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1] o.a.k.s.p.internals.StreamThread
> - stream-thread [StreamThread-1] Committing task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1] o.a.k.s.p.internals.StreamThread
> - stream-thread [StreamThread-1] Committing task StreamTask 2_31
>
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
>
> Does anyone else find the chattiness of this message annoying? I would
> personally prefer something more like:
>
> # existing message is fine at TRACE level for diagnostics
> "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
> # normal fast case, wrap them all up into one summary line
> "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
> # some kind of threshold / messaging in case it doesn't complete quickly
> or logs an exception
> "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms"
>
> Thoughts?
>
>
--
-- Guozhang
Re: Chatty StreamThread commit messages
Posted by Steven Schlansker <ss...@opentable.com>.
Thank you everyone for the feedback! I filed
https://issues.apache.org/jira/browse/KAFKA-4829
> On Mar 1, 2017, at 8:22 AM, Michael Noll <mi...@confluent.io> wrote:
>
> Good point, Steven. +1 here.
>
> On Wed, Mar 1, 2017 at 8:52 AM, Damian Guy <da...@gmail.com> wrote:
>
>> +1
>> On Wed, 1 Mar 2017 at 07:15, Guozhang Wang <wa...@gmail.com> wrote:
>>
>>> Hey Steven,
>>>
>>> That is a good question, and I think your proposal makes sense. Could you
>>> file a JIRA for this change to keep track of it?
>>>
>>> Guozhang
>>>
>>> On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
>>> sschlansker@opentable.com> wrote:
>>>
>>>> Hi everyone, running with Kafka Streams 0.10.2.0, I see this every
>> commit
>>>> interval:
>>>>
>>>> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
>>> o.a.k.s.p.internals.StreamThread
>>>> - stream-thread [StreamThread-1] Committing task StreamTask 1_31
>>>> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
>>> o.a.k.s.p.internals.StreamThread
>>>> - stream-thread [StreamThread-1] Committing task StreamTask 2_31
>>>>
>>>> We have ~10 tasks in our topology, 4 topics, and 32 partitions per
>> topic.
>>>> This means every commit interval we log a few hundred lines of the
>> above
>>>> which is an order of magnitude chattier than anything else in the log
>>>> during normal operations.
>>>>
>>>> Does anyone else find the chattiness of this message annoying? I would
>>>> personally prefer something more like:
>>>>
>>>> # existing message is fine at TRACE level for diagnostics
>>>> "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
>>>> # normal fast case, wrap them all up into one summary line
>>>> "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
>>>> # some kind of threshold / messaging in case it doesn't complete
>> quickly
>>>> or logs an exception
>>>> "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in
>>> 100ms"
>>>>
>>>> Thoughts?
>>>>
>>>>
>>>
>>>
>>> --
>>> -- Guozhang
>>>
>>
Re: Chatty StreamThread commit messages
Posted by Michael Noll <mi...@confluent.io>.
Good point, Steven. +1 here.
On Wed, Mar 1, 2017 at 8:52 AM, Damian Guy <da...@gmail.com> wrote:
> +1
> On Wed, 1 Mar 2017 at 07:15, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hey Steven,
> >
> > That is a good question, and I think your proposal makes sense. Could you
> > file a JIRA for this change to keep track of it?
> >
> > Guozhang
> >
> > On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
> > sschlansker@opentable.com> wrote:
> >
> > > Hi everyone, running with Kafka Streams 0.10.2.0, I see this every
> commit
> > > interval:
> > >
> > > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> > o.a.k.s.p.internals.StreamThread
> > > - stream-thread [StreamThread-1] Committing task StreamTask 1_31
> > > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> > o.a.k.s.p.internals.StreamThread
> > > - stream-thread [StreamThread-1] Committing task StreamTask 2_31
> > >
> > > We have ~10 tasks in our topology, 4 topics, and 32 partitions per
> topic.
> > > This means every commit interval we log a few hundred lines of the
> above
> > > which is an order of magnitude chattier than anything else in the log
> > > during normal operations.
> > >
> > > Does anyone else find the chattiness of this message annoying? I would
> > > personally prefer something more like:
> > >
> > > # existing message is fine at TRACE level for diagnostics
> > > "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
> > > # normal fast case, wrap them all up into one summary line
> > > "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
> > > # some kind of threshold / messaging in case it doesn't complete
> quickly
> > > or logs an exception
> > > "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in
> > 100ms"
> > >
> > > Thoughts?
> > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>
Re: Chatty StreamThread commit messages
Posted by Damian Guy <da...@gmail.com>.
+1
On Wed, 1 Mar 2017 at 07:15, Guozhang Wang <wa...@gmail.com> wrote:
> Hey Steven,
>
> That is a good question, and I think your proposal makes sense. Could you
> file a JIRA for this change to keep track of it?
>
> Guozhang
>
> On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
> sschlansker@opentable.com> wrote:
>
> > Hi everyone, running with Kafka Streams 0.10.2.0, I see this every commit
> > interval:
> >
> > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread
> > - stream-thread [StreamThread-1] Committing task StreamTask 1_31
> > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread
> > - stream-thread [StreamThread-1] Committing task StreamTask 2_31
> >
> > We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> > This means every commit interval we log a few hundred lines of the above
> > which is an order of magnitude chattier than anything else in the log
> > during normal operations.
> >
> > Does anyone else find the chattiness of this message annoying? I would
> > personally prefer something more like:
> >
> > # existing message is fine at TRACE level for diagnostics
> > "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
> > # normal fast case, wrap them all up into one summary line
> > "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
> > # some kind of threshold / messaging in case it doesn't complete quickly
> > or logs an exception
> > "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in
> 100ms"
> >
> > Thoughts?
> >
> >
>
>
> --
> -- Guozhang
>