You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by noah <ia...@gmail.com> on 2015/09/14 15:06:33 UTC
Tools/recommendations to debug performance issues?
We're using 0.8.2.1 processing maybe 1 million messages per hour. Each
message includes tracking information with a timestamp for when it was
produced, and a timestamp for when it was consumed, to give us roughly the
amount of time it spent in Kafka. On average this number is in the seconds
and our upper percentiles are in the minutes.
What metrics and settings can we look at to figure out why we might be
spending so much time in Kafka?
Re: Tools/recommendations to debug performance issues?
Posted by Gwen Shapira <gw...@confluent.io>.
Kafka also collects very useful metrics on request times and their
breakdown.
They are under kafka.network.<request name>
On Mon, Sep 14, 2015 at 6:59 AM, Rahul Jain <ra...@gmail.com> wrote:
> Have you checked the consumer lag? You can use the offset checker tool to
> see if there is a lag.
> On 14 Sep 2015 18:36, "noah" <ia...@gmail.com> wrote:
>
> > We're using 0.8.2.1 processing maybe 1 million messages per hour. Each
> > message includes tracking information with a timestamp for when it was
> > produced, and a timestamp for when it was consumed, to give us roughly
> the
> > amount of time it spent in Kafka. On average this number is in the
> seconds
> > and our upper percentiles are in the minutes.
> >
> > What metrics and settings can we look at to figure out why we might be
> > spending so much time in Kafka?
> >
>
Re: Tools/recommendations to debug performance issues?
Posted by Rahul Jain <ra...@gmail.com>.
Have you checked the consumer lag? You can use the offset checker tool to
see if there is a lag.
On 14 Sep 2015 18:36, "noah" <ia...@gmail.com> wrote:
> We're using 0.8.2.1 processing maybe 1 million messages per hour. Each
> message includes tracking information with a timestamp for when it was
> produced, and a timestamp for when it was consumed, to give us roughly the
> amount of time it spent in Kafka. On average this number is in the seconds
> and our upper percentiles are in the minutes.
>
> What metrics and settings can we look at to figure out why we might be
> spending so much time in Kafka?
>