[jira] [Commented] (KAFKA-2631) Expose data volume via JMX

["Eno Thereska (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14953242#comment-14953242 ] 

Eno Thereska commented on KAFKA-2631:
-------------------------------------

It turns out we already have a Meter that measures a broker's incoming bytes (just for the leader).  
It's the one below val bytesInRate = newMeter("BytesInPerSec", "bytes", TimeUnit.SECONDS, tags)

It's primarily used to eventually calculate a rate, but it also keeps track of the cumulative number of bytes received (across all topics, at a leader). 

However, I did notice it captures the original producer payload + record header, so not just a customer's payload. For small messages the header can dominate and the customer might be surprised that they send a message of size 1 byte and 28 bytes were actually consumed. I think that is still ok.


> Expose data volume via JMX
> --------------------------
>
>                 Key: KAFKA-2631
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2631
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 0.8.2.1
>            Reporter: Eno Thereska
>            Assignee: Eno Thereska
>             Fix For: 0.8.2.1
>
>
> It would be good to expose the volume of data stored on a broker. This must include only the volume created by a client and must not include the volume of data created due to replication. In practice this means only a leader needs to keep track of the volume of the data received by a producer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)