You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Neil Harkins <nh...@gmail.com> on 2014/08/05 21:23:26 UTC
kafka.SocketServerStats write rate metrics seem wrong
Hi.
I'm seeing some odd numbers from kafka.SocketServerStats.
Ideally, I'd like to have stats broken down per-topic, e.g.
what's our most written/read topics? For write rates, I've got
a separate process iterating topics every minute, doing this:
(head_offset_now - head_offset_last) / (time_now - time_last)
and inserting that into graphite (was previously inserting the
head_offset, but graphite's derivative() on 2000+ topics
was a bit much).
a graphite sumSeries() across them shows 20~25 MB/s
on a single-instance kafka 0.7.2, which sounds correct,
but when I compare that to the kafka.SocketServerStats
we're collecting in graphite, derivative(TotalBytesWritten)
shows 3.0GB/s, and BytesWrittenPerSecond shows
200KB/s, neither of which jive with my offset rate.
Is anyone aware of any weirdness with those stats?
thanks in advance for any insight,
-neil
Re: kafka.SocketServerStats write rate metrics seem wrong
Posted by Jun Rao <ju...@gmail.com>.
TotalBytesWritten is probably for sending the responses. You probably
should look at TotalBytesRead for incoming data rate.
Thanks,
Jun
On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins <nh...@gmail.com> wrote:
> Hi.
> I'm seeing some odd numbers from kafka.SocketServerStats.
>
> Ideally, I'd like to have stats broken down per-topic, e.g.
> what's our most written/read topics? For write rates, I've got
> a separate process iterating topics every minute, doing this:
> (head_offset_now - head_offset_last) / (time_now - time_last)
> and inserting that into graphite (was previously inserting the
> head_offset, but graphite's derivative() on 2000+ topics
> was a bit much).
>
> a graphite sumSeries() across them shows 20~25 MB/s
> on a single-instance kafka 0.7.2, which sounds correct,
> but when I compare that to the kafka.SocketServerStats
> we're collecting in graphite, derivative(TotalBytesWritten)
> shows 3.0GB/s, and BytesWrittenPerSecond shows
> 200KB/s, neither of which jive with my offset rate.
>
> Is anyone aware of any weirdness with those stats?
>
> thanks in advance for any insight,
> -neil
>
Re: kafka.SocketServerStats write rate metrics seem wrong
Posted by Neil Harkins <nh...@gmail.com>.
No compression is configured, the spool files
can be /usr/bin/strings'd to show the msgs.
On Tue, Aug 5, 2014 at 2:13 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Hello Neil,
>
> Did you use any compression on your producers?
>
> Guozhang
>
>
> On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins <nh...@gmail.com> wrote:
>>
>> Hi.
>> I'm seeing some odd numbers from kafka.SocketServerStats.
>>
>> Ideally, I'd like to have stats broken down per-topic, e.g.
>> what's our most written/read topics? For write rates, I've got
>> a separate process iterating topics every minute, doing this:
>> (head_offset_now - head_offset_last) / (time_now - time_last)
>> and inserting that into graphite (was previously inserting the
>> head_offset, but graphite's derivative() on 2000+ topics
>> was a bit much).
>>
>> a graphite sumSeries() across them shows 20~25 MB/s
>> on a single-instance kafka 0.7.2, which sounds correct,
>> but when I compare that to the kafka.SocketServerStats
>> we're collecting in graphite, derivative(TotalBytesWritten)
>> shows 3.0GB/s, and BytesWrittenPerSecond shows
>> 200KB/s, neither of which jive with my offset rate.
>>
>> Is anyone aware of any weirdness with those stats?
>>
>> thanks in advance for any insight,
>> -neil
>
>
>
>
> --
> -- Guozhang
Re: kafka.SocketServerStats write rate metrics seem wrong
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Neil,
Did you use any compression on your producers?
Guozhang
On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins <nh...@gmail.com> wrote:
> Hi.
> I'm seeing some odd numbers from kafka.SocketServerStats.
>
> Ideally, I'd like to have stats broken down per-topic, e.g.
> what's our most written/read topics? For write rates, I've got
> a separate process iterating topics every minute, doing this:
> (head_offset_now - head_offset_last) / (time_now - time_last)
> and inserting that into graphite (was previously inserting the
> head_offset, but graphite's derivative() on 2000+ topics
> was a bit much).
>
> a graphite sumSeries() across them shows 20~25 MB/s
> on a single-instance kafka 0.7.2, which sounds correct,
> but when I compare that to the kafka.SocketServerStats
> we're collecting in graphite, derivative(TotalBytesWritten)
> shows 3.0GB/s, and BytesWrittenPerSecond shows
> 200KB/s, neither of which jive with my offset rate.
>
> Is anyone aware of any weirdness with those stats?
>
> thanks in advance for any insight,
> -neil
>
--
-- Guozhang