You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/09/09 22:20:52 UTC
[GitHub] [kafka] wushujames opened a new pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
wushujames opened a new pull request #9276:
URL: https://github.com/apache/kafka/pull/9276
Add docs on the following JMX metrics
kafka.log,type=Log,name=Size
kafka.log,type=Log,name=NumLogSegments
kafka.log,type=Log,name=LogStartOffset
kafka.log,type=Log,name=LogEndOffset
To test this, I opened the docs in a web browser
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjinleekr commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
dongjinleekr commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-706104273
@wushujames Could you make some updates? I think it would be good for this PR to be included in the 2.7.0 release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on a change in pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#discussion_r535511409
##########
File path: docs/ops.html
##########
@@ -1129,6 +1128,26 @@ <h4><a id="remote_jmx" href="#remote_jmx">Security Considerations for Remote Mon
<td>kafka.server:type=BrokerTopicMetrics,name=ReassignmentBytesInPerSec</td>
<td></td>
</tr>
+ <tr>
+ <td>Size of a partition on disk (in bytes)</td>
+ <td>kafka.log:type=Log,name=Size,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The size of a partition on disk, measured in bytes.</td>
+ </tr>
+ <tr>
+ <td>Number of log segments in a partition</td>
+ <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The number of log segments in a partition.</td>
Review comment:
I think this is fine :)
##########
File path: docs/ops.html
##########
@@ -1129,6 +1128,26 @@ <h4><a id="remote_jmx" href="#remote_jmx">Security Considerations for Remote Mon
<td>kafka.server:type=BrokerTopicMetrics,name=ReassignmentBytesInPerSec</td>
<td></td>
</tr>
+ <tr>
+ <td>Size of a partition on disk (in bytes)</td>
+ <td>kafka.log:type=Log,name=Size,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The size of a partition on disk, measured in bytes.</td>
+ </tr>
+ <tr>
+ <td>Number of log segments in a partition</td>
+ <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The number of log segments in a partition.</td>
+ </tr>
+ <tr>
+ <td>First offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogStartOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The first offset in a partition.</td>
+ </tr>
+ <tr>
+ <td>Last offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogEndOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The last offset in a partition.</td>
Review comment:
This is the last uncomitted offset: after the local append it would be incremented, not waiting for other replicas to ack.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-697994325
The pull request lists lots of failed checks. However, this pull request only changed an HTML file, and it seems unrelated to the test failures in those checks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang merged pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
guozhangwang merged pull request #9276:
URL: https://github.com/apache/kafka/pull/9276
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-706452504
@dongjinleekr All builds failed, but none of the failures seem related to my change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-706365247
@dongjinleekr : I've pulled in the latest changes from trunk. I'll check back in after the builds have completed.
Aside from pulling the latest changes from trunk, were there any other updates/changes you think are needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on a change in pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#discussion_r485955266
##########
File path: docs/ops.html
##########
@@ -1129,6 +1128,26 @@ <h4><a id="remote_jmx" href="#remote_jmx">Security Considerations for Remote Mon
<td>kafka.server:type=BrokerTopicMetrics,name=ReassignmentBytesInPerSec</td>
<td></td>
</tr>
+ <tr>
+ <td>Size of a partition on disk (in bytes)</td>
+ <td>kafka.log:type=Log,name=Size,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The size of a partition on disk, measured in bytes.</td>
+ </tr>
+ <tr>
+ <td>Number of log segments in a partition</td>
+ <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The number of log segments in a partition.</td>
Review comment:
I wasn't sure what information to add for this column of "normal value".
* For date-based retention topics, it should roughly be sizeOfTopic/segmentSize
* But for compacted topics, that isn't the case
* If this is too high, it will mean more file handles.
* If it is too high, it might inefficient (is this true?)
Anyway, I wasn't sure what to put, so I left it simple.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on a change in pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#discussion_r485956062
##########
File path: docs/ops.html
##########
@@ -1129,6 +1128,26 @@ <h4><a id="remote_jmx" href="#remote_jmx">Security Considerations for Remote Mon
<td>kafka.server:type=BrokerTopicMetrics,name=ReassignmentBytesInPerSec</td>
<td></td>
</tr>
+ <tr>
+ <td>Size of a partition on disk (in bytes)</td>
+ <td>kafka.log:type=Log,name=Size,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The size of a partition on disk, measured in bytes.</td>
+ </tr>
+ <tr>
+ <td>Number of log segments in a partition</td>
+ <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The number of log segments in a partition.</td>
+ </tr>
+ <tr>
+ <td>First offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogStartOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The first offset in a partition.</td>
+ </tr>
+ <tr>
+ <td>Last offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogEndOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The last offset in a partition.</td>
Review comment:
Tag @hachikuji
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on a change in pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#discussion_r485955567
##########
File path: docs/ops.html
##########
@@ -1129,6 +1128,26 @@ <h4><a id="remote_jmx" href="#remote_jmx">Security Considerations for Remote Mon
<td>kafka.server:type=BrokerTopicMetrics,name=ReassignmentBytesInPerSec</td>
<td></td>
</tr>
+ <tr>
+ <td>Size of a partition on disk (in bytes)</td>
+ <td>kafka.log:type=Log,name=Size,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The size of a partition on disk, measured in bytes.</td>
+ </tr>
+ <tr>
+ <td>Number of log segments in a partition</td>
+ <td>kafka.log:type=Log,name=NumLogSegments,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The number of log segments in a partition.</td>
+ </tr>
+ <tr>
+ <td>First offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogStartOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The first offset in a partition.</td>
+ </tr>
+ <tr>
+ <td>Last offset in a partition</td>
+ <td>kafka.log:type=Log,name=LogEndOffset,topic=([-.\w]+),partition=([0-9]+)</td>
+ <td>The last offset in a partition.</td>
Review comment:
Is this the last committed offset? Or, is it uncommitted offsets? Meaning, if a producer appends a record there but the followers haven't yet acked it, will this value get incremented?
Does that level of detail matter, in the docs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] wushujames commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
wushujames commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-689856879
@guozhangwang Can I get a review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjinleekr commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics
Posted by GitBox <gi...@apache.org>.
dongjinleekr commented on pull request #9276:
URL: https://github.com/apache/kafka/pull/9276#issuecomment-706104273
@wushujames Could you make some updates? I think it would be good for this PR to be included in the 2.7.0 release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org