You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by gu...@apache.org on 2019/10/15 18:15:14 UTC
[kafka] branch 2.4 updated: KAFKA-8942: Document RocksDB metrics
This is an automated email from the ASF dual-hosted git repository.
guozhang pushed a commit to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/2.4 by this push:
new 2667773 KAFKA-8942: Document RocksDB metrics
2667773 is described below
commit 26677731f87116154f74da421d4f440b80599da1
Author: Bruno Cadonna <br...@confluent.io>
AuthorDate: Tue Oct 15 11:10:42 2019 -0700
KAFKA-8942: Document RocksDB metrics
Author: Bruno Cadonna <br...@confluent.io>
Reviewers: Guozhang Wang <wa...@gmail.com>
Closes #7490 from cadonna/AK8942-docs-rocksdb_metrics
Minor comments
(cherry picked from commit 9c80a06466dbfade96308ef26c20c6555612191e)
Signed-off-by: Guozhang Wang <wa...@gmail.com>
---
docs/ops.html | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 102 insertions(+)
diff --git a/docs/ops.html b/docs/ops.html
index 05d7de1..d87d16a 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -1979,6 +1979,108 @@ All the following metrics have a recording level of <code>debug</code>:
</tbody>
</table>
+ <h5><a id="kafka_streams_rocksdb_monitoring" href="#kafka_streams_rocksdb_monitoring">RocksDB Metrics</a></h5>
+ All the following metrics have a recording level of <code>debug</code>.
+ The metrics are collected every minute from the RocksDB state stores.
+ If a state store consists of multiple RocksDB instances as it is the case for aggregations over time and session windows,
+ each metric reports an aggregation over the RocksDB instances of the state store.
+ Note that the <code>store-scope</code> for built-in RocksDB state stores are currently the following:
+ <ul>
+ <li><code>rocksdb-state</code> (for RocksDB backed key-value store)</li>
+ <li><code>rocksdb-window-state</code> (for RocksDB backed window store)</li>
+ <li><code>rocksdb-session-state</code> (for RocksDB backed session store)</li>
+ </ul>
+
+ <table class="data-table">
+ <tbody>
+ <tr>
+ <th>Metric/Attribute name</th>
+ <th>Description</th>
+ <th>Mbean name</th>
+ </tr>
+ <tr>
+ <td>bytes-written-rate</td>
+ <td>The average number of bytes written per second to the RocksDB state store.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>bytes-written-total</td>
+ <td>The total number of bytes written to the RocksDB state store.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>bytes-read-rate</td>
+ <td>The average number of bytes read per second from the RocksDB state store.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>bytes-read-total</td>
+ <td>The total number of bytes read from the RocksDB state store.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>memtable-bytes-flushed-rate</td>
+ <td>The average number of bytes flushed per second from the memtable to disk.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>memtable-bytes-flushed-total</td>
+ <td>The total number of bytes flushed from the memtable to disk.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>memtable-hit-ratio</td>
+ <td>The ratio of memtable hits relative to all lookups to the memtable.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>block-cache-data-hit-ratio</td>
+ <td>The ratio of block cache hits for data blocks relative to all lookups for data blocks to the block cache.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>block-cache-index-hit-ratio</td>
+ <td>The ratio of block cache hits for index blocks relative to all lookups for index blocks to the block cache.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>block-cache-filter-hit-ratio</td>
+ <td>The ratio of block cache hits for filter blocks relative to all lookups for filter blocks to the block cache.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>write-stall-duration-avg</td>
+ <td>The average duration of write stalls in ms.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>write-stall-duration-total</td>
+ <td>The total duration of write stalls in ms.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>bytes-read-compaction-rate</td>
+ <td>The average number of bytes read per second during compaction.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>bytes-written-compaction-rate</td>
+ <td>The average number of bytes written per second during compaction.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>number-open-files</td>
+ <td>The number of current open files.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>number-file-errors-total</td>
+ <td>The total number of file errors occurred.</td>
+ <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
+ </tr>
+ </tbody>
+ </table>
+
<h5><a id="kafka_streams_cache_monitoring" href="#kafka_streams_cache_monitoring">Record Cache Metrics</a></h5>
All the following metrics have a recording level of <code>debug</code>: