High system.io.await on Kafka brokers?

[wenxing zheng <we...@gmail.com>]

Dear all,

We got a kafka cluster with 5 nodes, and from the metrics of datadog, we
found that regularly the elapse for sending to kafka was more than 200ms,
and there was a peek on the system.io.await.

Please help to advice what would be the problem and any hints?
[image: image.png]

Kind regards, Wenxing

Re: High system.io.await on Kafka brokers?

[wenxing zheng <we...@gmail.com>]

Thanks to Sam. We are ongoing the evaluation.

On 2019/01/22 09:50:53, Sam Pegler <sa...@infectiousmedia.com> wrote: 
> Sounds like you're reaching the limits of what your disks will do either on
> reads or writes.  Debug it as you would any other disk based app,
> https://haydenjames.io/linux-server-performance-disk-io-slowing-application/
> might help.
> 
> 
> 
> On Tue, 22 Jan 2019 at 09:19, wenxing zheng <we...@gmail.com> wrote:
> 
> > Dear all,
> >
> > We got a kafka cluster with 5 nodes, and from the metrics of datadog, we
> > found that regularly the elapse for sending to kafka was more than 200ms,
> > and there was a peek on the system.io.await.
> >
> > Please help to advice what would be the problem and any hints?
> > [image: image.png]
> >
> > Kind regards, Wenxing
> >
> 

Re: High system.io.await on Kafka brokers?

[Sam Pegler <sa...@infectiousmedia.com>]

Sounds like you're reaching the limits of what your disks will do either on
reads or writes.  Debug it as you would any other disk based app,
https://haydenjames.io/linux-server-performance-disk-io-slowing-application/
might help.



On Tue, 22 Jan 2019 at 09:19, wenxing zheng <we...@gmail.com> wrote:

> Dear all,
>
> We got a kafka cluster with 5 nodes, and from the metrics of datadog, we
> found that regularly the elapse for sending to kafka was more than 200ms,
> and there was a peek on the system.io.await.
>
> Please help to advice what would be the problem and any hints?
> [image: image.png]
>
> Kind regards, Wenxing
>