You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by ma...@apache.org on 2023/01/31 05:50:44 UTC
[iotdb] branch master updated: [IOTDB-5418] Add metric for mods file (#8932)
This is an automated email from the ASF dual-hosted git repository.
marklau99 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new 174242a678 [IOTDB-5418] Add metric for mods file (#8932)
174242a678 is described below
commit 174242a6784abcc1f50e8d14c8ee385981fe0106
Author: Liu Xuxin <37...@users.noreply.github.com>
AuthorDate: Tue Jan 31 13:50:38 2023 +0800
[IOTDB-5418] Add metric for mods file (#8932)
---
docs/UserGuide/Monitor-Alert/Metric-Tool.md | 265 ++++++++----
docs/zh/UserGuide/Monitor-Alert/Metric-Tool.md | 447 ++++++++++++---------
.../iotdb/db/engine/TsFileMetricManager.java | 28 ++
.../execute/task/CrossSpaceCompactionTask.java | 16 +
.../compaction/execute/utils/CompactionUtils.java | 13 +
.../db/engine/modification/ModificationFile.java | 9 +
.../iotdb/db/engine/storagegroup/DataRegion.java | 25 ++
.../iotdb/db/service/metrics/FileMetrics.java | 18 +
8 files changed, 537 insertions(+), 284 deletions(-)
diff --git a/docs/UserGuide/Monitor-Alert/Metric-Tool.md b/docs/UserGuide/Monitor-Alert/Metric-Tool.md
index c0ee62faaa..d1fa1c5ae6 100644
--- a/docs/UserGuide/Monitor-Alert/Metric-Tool.md
+++ b/docs/UserGuide/Monitor-Alert/Metric-Tool.md
@@ -51,7 +51,9 @@ Belows are some typical application scenarios
running abnormally.
## 2. Who will use metric framework?
-Any person cares about the system's status, including but not limited to RD, QA, SRE, DBA, can use the metrics to work more efficiently.
+
+Any person cares about the system's status, including but not limited to RD, QA, SRE, DBA, can use the metrics to work
+more efficiently.
## 3. What is metrics?
@@ -60,19 +62,36 @@ Any person cares about the system's status, including but not limited to RD, QA,
In IoTDB's metric module, each metrics is uniquely identified by `Metric Name` and `Tags`.
- `Metric Name`: Metric type name, such as `logback_events` means log events.
-- `Tags`: indicator classification, in the form of Key-Value pairs, each indicator can have 0 or more categories, common Key-Value pairs:
- - `name = xxx`: The name of the monitored object, which is the description of **business logic**. For example, for a monitoring item of type `Metric Name = entry_seconds_count`, the meaning of name refers to the monitored business interface.
- - `type = xxx`: Monitoring indicator type subdivision, which is a description of **monitoring indicator** itself. For example, for monitoring items of type `Metric Name = point`, the meaning of type refers to the specific type of monitoring points.
- - `status = xxx`: The status of the monitored object is a description of **business logic**. For example, for monitoring items of type `Metric Name = Task`, this parameter can be used to distinguish the status of the monitored object.
- - `user = xxx`: The relevant user of the monitored object is a description of **business logic**. For example, count the total points written by the `root` user.
- - Customize according to the specific situation: For example, there is a level classification under logback_events_total, which is used to indicate the number of logs under a specific level.
-- `Metric Level`: The level of metric managing level, The default startup level is `Core` level, the recommended startup level is `Important level`, and the audit strictness is `Core > Important > Normal > All`
- - `Core`: Core metrics of the system, used by the **operation and maintenance personnel**, which is related to the **performance, stability, and security** of the system, such as the status of the instance, the load of the system, etc.
- - `Important`: Important metrics of the module, which is used by **operation and maintenance and testers**, and is directly related to **the running status of each module**, such as the number of merged files, execution status, etc.
- - `Normal`: Normal metrics of the module, used by **developers** to facilitate **locating the module** when problems occur, such as specific key operation situations in the merger.
- - `All`: All metrics of the module, used by **module developers**, often used when the problem is reproduced, so as to solve the problem quickly.
+- `Tags`: indicator classification, in the form of Key-Value pairs, each indicator can have 0 or more categories, common
+ Key-Value pairs:
+ - `name = xxx`: The name of the monitored object, which is the description of **business logic**. For example, for a
+ monitoring item of type `Metric Name = entry_seconds_count`, the meaning of name refers to the monitored business
+ interface.
+ - `type = xxx`: Monitoring indicator type subdivision, which is a description of **monitoring indicator** itself.
+ For example, for monitoring items of type `Metric Name = point`, the meaning of type refers to the specific type
+ of monitoring points.
+ - `status = xxx`: The status of the monitored object is a description of **business logic**. For example, for
+ monitoring items of type `Metric Name = Task`, this parameter can be used to distinguish the status of the
+ monitored object.
+ - `user = xxx`: The relevant user of the monitored object is a description of **business logic**. For example, count
+ the total points written by the `root` user.
+ - Customize according to the specific situation: For example, there is a level classification under
+ logback_events_total, which is used to indicate the number of logs under a specific level.
+- `Metric Level`: The level of metric managing level, The default startup level is `Core` level, the recommended startup
+ level is `Important level`, and the audit strictness is `Core > Important > Normal > All`
+ - `Core`: Core metrics of the system, used by the **operation and maintenance personnel**, which is related to the *
+ *performance, stability, and security** of the system, such as the status of the instance, the load of the system,
+ etc.
+ - `Important`: Important metrics of the module, which is used by **operation and maintenance and testers**, and is
+ directly related to **the running status of each module**, such as the number of merged files, execution status,
+ etc.
+ - `Normal`: Normal metrics of the module, used by **developers** to facilitate **locating the module** when problems
+ occur, such as specific key operation situations in the merger.
+ - `All`: All metrics of the module, used by **module developers**, often used when the problem is reproduced, so as
+ to solve the problem quickly.
### 3.2. External data format for metrics
+
- IoTDB provides metrics in JMX, Prometheus and IoTDB formats:
- For JMX, metrics can be obtained through ```org.apache.iotdb.metrics```.
- For Prometheus, the value of the metrics can be obtained through the externally exposed port
@@ -87,18 +106,22 @@ If you want to add your own metrics data in IoTDB, please see
the [IoTDB Metric Framework] (https://github.com/apache/iotdb/tree/master/metrics) document.
### 4.1. Core level metrics
-Core-level metrics are enabled by default during system operation. The addition of each Core-level metrics needs to be carefully evaluated. The current Core-level metrics are as follows:
+
+Core-level metrics are enabled by default during system operation. The addition of each Core-level metrics needs to be
+carefully evaluated. The current Core-level metrics are as follows:
#### 4.1.1. Cluster
+
| Metric | Tags | Type | Description |
-| ----------- | ----------------------------------------------- | --------- | --------------------------------------------------- |
+|-------------|-------------------------------------------------|-----------|-----------------------------------------------------|
| config_node | name="total",status="Registered/Online/Unknown" | AutoGauge | The number of registered/online/unknown confignodes |
| data_node | name="total",status="Registered/Online/Unknown" | AutoGauge | The number of registered/online/unknown datanodes |
| points | database="{{database}}", type="flush" | Gauge | The point number of last flushed memtable |
#### 4.1.2. IoTDB process
+
| Metric | Tags | Type | Description |
-| ----------------- | -------------- | --------- | ------------------------------------------------------ |
+|-------------------|----------------|-----------|--------------------------------------------------------|
| process_cpu_load | name="process" | AutoGauge | The current CPU usage of IoTDB process, Unit: % |
| process_cpu_time | name="process" | AutoGauge | The total CPU time occupied of IoTDB process, Unit: ns |
| process_max_mem | name="memory" | AutoGauge | The maximum available memory of IoTDB process |
@@ -106,8 +129,9 @@ Core-level metrics are enabled by default during system operation. The addition
| process_free_mem | name="memory" | AutoGauge | The free available memory of IoTDB process |
#### 4.1.3. System
+
| Metric | Tags | Type | Description |
-| ------------------------------ | ------------- | --------- | ---------------------------------------------------------- |
+|--------------------------------|---------------|-----------|------------------------------------------------------------|
| sys_cpu_load | name="system" | AutoGauge | The current CPU usage of system, Unit: % |
| sys_cpu_cores | name="system" | Gauge | The available number of CPU cores |
| sys_total_physical_memory_size | name="memory" | Gauge | The maximum physical memory of system |
@@ -121,14 +145,16 @@ Core-level metrics are enabled by default during system operation. The addition
### 4.2. Important level metrics
#### 4.2.1. Cluster
+
| Metric | Tags | Type | Description |
-| ------------------------- | ------------------------------------------------- | ----- | ------------------------------------------------ |
+|---------------------------|---------------------------------------------------|-------|--------------------------------------------------|
| cluster_node_leader_count | name="{{ip}}:{{port}}" | Gauge | The count of consensus group leader on each node |
| cluster_node_status | name="{{ip}}:{{port}}",type="ConfigNode/DataNode" | Gauge | The current node status, 0=Unkonwn 1=online |
#### 4.2.2. Node
+
| Metric | Tags | Type | Description |
-| -------- | ------------------------------------------ | --------- | ------------------------------------------------------------- |
+|----------|--------------------------------------------|-----------|---------------------------------------------------------------|
| quantity | name="database" | AutoGauge | The number of database |
| quantity | name="timeSeries" | AutoGauge | The number of timeseries |
| quantity | name="pointsIn" | Counter | The number of write points |
@@ -138,8 +164,9 @@ Core-level metrics are enabled by default during system operation. The addition
| region | name="{{ip}}:{{port}}",type="DataRegion" | Gauge | The number of DataRegion in PartitionTable of specific node |
#### 4.2.3. IoTConsensus
+
| Metric | Tags | Type | Description |
-| ------------ | -------------------------------------------------------------------------------------------- | --------- | --------------------------------------------------------------------- |
+|--------------|----------------------------------------------------------------------------------------------|-----------|-----------------------------------------------------------------------|
| mutli_leader | name="logDispatcher-{{IP}}:{{Port}}", region="{{region}}", type="currentSyncIndex" | AutoGauge | The sync index of synchronization thread in replica group |
| mutli_leader | name="logDispatcher-{{IP}}:{{Port}}", region="{{region}}", type="cachedRequestInMemoryQueue" | AutoGauge | The size of cache requests of synchronization thread in replica group |
| mutli_leader | name="IoTConsensusServerImpl", region="{{region}}", type="searchIndex" | AutoGauge | The write process of main process in replica group |
@@ -153,8 +180,9 @@ Core-level metrics are enabled by default during system operation. The addition
| stage | name="iot_consensus", region="{{region}}", type="syncLogTimePerRequest" | Histogram | The time consumed to sync log in asynchronous callback process |
#### 4.2.4. Cache
+
| Metric | Tags | Type | Description |
-| --------- | ---------------------------------- | --------- | ------------------------------------------------------------------------ |
+|-----------|------------------------------------|-----------|--------------------------------------------------------------------------|
| cache_hit | name="chunk" | AutoGauge | The cache hit ratio of ChunkCache, Unit: % |
| cache_hit | name="schema" | AutoGauge | The cache hit ratio of SchemaCache, Unit: % |
| cache_hit | name="timeSeriesMeta" | AutoGauge | The cache hit ratio of TimeseriesMetadataCache, Unit: % |
@@ -167,8 +195,9 @@ Core-level metrics are enabled by default during system operation. The addition
| cache | name="DataPartition", type="all" | Counter | The access number of SDataPartition Cache |
#### 4.2.5. Interface
+
| Metric | Tags | Type | Description |
-| --------------------- | ---------------------------------------------------- | --------- | -------------------------------------------------------------- |
+|-----------------------|------------------------------------------------------|-----------|----------------------------------------------------------------|
| statement_execution | interface="{{interface}}", type="{{statement_type}}" | Timer | The time consumed of operations in client |
| entry | name="{{interface}}" | Timer | The time consumed of thrift operations |
| thrift_connections | name="ConfigNodeRPC" | AutoGauge | The number of thrift internal connections in ConfigNode |
@@ -181,8 +210,9 @@ Core-level metrics are enabled by default during system operation. The addition
| thrift_active_threads | name="ClientRPC-Service" | AutoGauge | The number of thrift active connections of client |
#### 4.2.6. Memory
+
| Metric | Tags | Type | Description |
-| ------ | ------------------------------------ | --------- | ------------------------------------------------------------------ |
+|--------|--------------------------------------|-----------|--------------------------------------------------------------------|
| mem | name="database_{{name}}" | AutoGauge | The memory usage of DataRegion in DataNode, Unit: byte |
| mem | name="chunkMetaData_{{name}}" | AutoGauge | The memory usage of chunkMetaData when writting TsFile, Unit: byte |
| mem | name="IoTConsensus" | AutoGauge | The memory usage of IoTConsensus, Unit: byte |
@@ -190,16 +220,18 @@ Core-level metrics are enabled by default during system operation. The addition
| mem | name="schema_region_total_remaining" | AutoGauge | The memory remaining for all SchemaRegion, Unit: byte |
#### 4.2.7. Task
+
| Metric | Tags | Type | Description |
-| --------- | ------------------------------------------------- | --------- | ------------------------------------- |
+|-----------|---------------------------------------------------|-----------|---------------------------------------|
| queue | name="compaction_inner", status="running/waiting" | Gauge | The number of inner compaction tasks |
| queue | name="compaction_cross", status="running/waiting" | Gauge | The number of cross compatcion tasks |
| cost_task | name="inner_compaction/cross_compaction/flush" | Gauge | The time consumed of compaction tasks |
| queue | name="flush",status="running/waiting" | AutoGauge | The number of flush tasks |
#### 4.2.8. Compaction
+
| Metric | Tags | Type | Description |
-| --------------------- | --------------------------------------------------- | ------- | -------------------------------------- |
+|-----------------------|-----------------------------------------------------|---------|----------------------------------------|
| data_written | name="compaction", type="aligned/not-aligned/total" | Counter | The written size of compaction |
| data_read | name="compaction" | Counter | The read size of compaction |
| compaction_task_count | name = "inner_compaction", type="sequence" | Counter | The number of inner sequence compction |
@@ -209,13 +241,14 @@ Core-level metrics are enabled by default during system operation. The addition
#### 4.2.9. File
| Metric | Tags | Type | Description |
-| ---------- |---------------------------| --------- |-----------------------------------------------------------------------------|
+|------------|---------------------------|-----------|-----------------------------------------------------------------------------|
| file_size | name="wal" | AutoGauge | The size of WAL file, Unit: byte |
| file_size | name="seq" | AutoGauge | The size of sequence TsFile, Unit: byte |
| file_size | name="unseq" | AutoGauge | The size of unsequence TsFile, Unit: byte |
| file_size | name="inner-seq-temp" | AutoGauge | The size of inner sequence space compaction temporal file |
| file_size | name="inner-unseq-temp" | AutoGauge | The size of inner unsequence space compaction temporal file |
| file_size | name="cross-temp" | AutoGauge | The size of cross space compaction temoporal file |
+| file_size | name="mods | AutoGauge | The size of modification files |
| file_count | name="wal" | AutoGauge | The count of WAL file |
| file_count | name="seq" | AutoGauge | The count of sequence TsFile |
| file_count | name="unseq" | AutoGauge | The count of unsequence TsFile |
@@ -223,33 +256,36 @@ Core-level metrics are enabled by default during system operation. The addition
| file_count | name="inner-unseq-temp" | AutoGauge | The count of inner unsequence space compaction temporal file |
| file_count | name="cross-temp" | AutoGauge | The count of cross space compaction temporal file |
| file_count | name="open_file_handlers" | AutoGauge | The count of open files of the IoTDB process, only supports Linux and MacOS |
+| file_count | name="mods | AutoGauge | The count of modification file |
#### 4.2.10. IoTDB Process
| Metric | Tags | Type | Description |
-| --------------------- | -------------- | --------- | ------------------------------------------- |
+|-----------------------|----------------|-----------|---------------------------------------------|
| process_used_mem | name="memory" | AutoGauge | The used memory of IoTDB process |
| process_mem_ratio | name="memory" | AutoGauge | The used memory ratio of IoTDB process |
| process_threads_count | name="process" | AutoGauge | The number of thread of IoTDB process |
| process_status | name="process" | AutoGauge | The status of IoTDB process, 1=live, 0=dead |
#### 4.2.11. Log
+
| Metric | Tags | Type | Description |
-| -------------- | ----------------------------------- | ------- | ------------------------ |
+|----------------|-------------------------------------|---------|--------------------------|
| logback_events | level="trace/debug/info/warn/error" | Counter | The number of log events |
#### 4.2.12. JVM Thread
| Metric | Tags | Type | Description |
-| -------------------------- | ------------------------------------------------------------- | --------- | ---------------------------------------- |
+|----------------------------|---------------------------------------------------------------|-----------|------------------------------------------|
| jvm_threads_live_threads | | AutoGauge | The number of live thread |
| jvm_threads_daemon_threads | | AutoGauge | The number of daemon thread |
| jvm_threads_peak_threads | | AutoGauge | The number of peak thread |
| jvm_threads_states_threads | state="runnable/blocked/waiting/timed-waiting/new/terminated" | AutoGauge | The number of thread in different states |
#### 4.2.13. JVM GC
+
| Metric | Tags | Type | Description |
-| ----------------------------- | ----------------------------------------------------- | --------- | --------------------------------------------------------------------------- |
+|-------------------------------|-------------------------------------------------------|-----------|-----------------------------------------------------------------------------|
| jvm_gc_pause | action="end of major GC/end of minor GC",cause="xxxx" | Timer | The number and time consumed of Young GC/Full Gc caused by different reason |
| |
| jvm_gc_concurrent_phase_time | action="{{action}}",cause="{{cause}}" | Timer | The number and time consumed of Young GC/Full Gc caused by different |
@@ -260,8 +296,9 @@ Core-level metrics are enabled by default during system operation. The addition
| jvm_gc_memory_allocated_bytes | | Counter | The accumulative value of positive memory growth of allocated memory |
#### 4.2.14. JVM Memory
+
| Metric | Tags | Type | Description |
-| ------------------------------- | ------------------------------- | --------- | --------------------------- |
+|---------------------------------|---------------------------------|-----------|-----------------------------|
| jvm_buffer_memory_used_bytes | id="direct/mapped" | AutoGauge | The used size of buffer |
| jvm_buffer_total_capacity_bytes | id="direct/mapped" | AutoGauge | The max size of buffer |
| jvm_buffer_count_buffers | id="direct/mapped" | AutoGauge | The number of buffer |
@@ -270,17 +307,20 @@ Core-level metrics are enabled by default during system operation. The addition
| jvm_memory_used_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | The used memory of JVM |
#### 4.2.15. JVM Class
+
| Metric | Tags | Type | Description |
-| ---------------------------- | ---- | --------- | ---------------------------- |
+|------------------------------|------|-----------|------------------------------|
| jvm_classes_unloaded_classes | | AutoGauge | The number of unloaded class |
| jvm_classes_loaded_classes | | AutoGauge | The number of loaded class |
#### 4.2.16. JVM Compilation
+
| Metric | Tags | Type | Description |
-| ----------------------- | --------------------------------------------- | --------- | -------------------------------- |
+|-------------------------|-----------------------------------------------|-----------|----------------------------------|
| jvm_compilation_time_ms | {compiler="HotSpot 64-Bit Tiered Compilers",} | AutoGauge | The time consumed in compilation |
#### 4.2.17. Query Planning
+
| Metric | Tags | Type | Description |
|-----------------|------------------------------|-------|-----------------------------------------------------|
| query_plan_cost | stage="sql_parser" | Timer | The SQL parsing time-consuming |
@@ -291,12 +331,14 @@ Core-level metrics are enabled by default during system operation. The addition
| query_plan_cost | stage="schema_fetcher" | Timer | The schema information fetching time-consuming |
#### 4.2.18. Plan Dispatcher
+
| Metric | Tags | Type | Description |
|------------|---------------------------|-------|--------------------------------------------------------------|
| dispatcher | stage="wait_for_dispatch" | Timer | The distribution plan dispatcher time-consuming |
| dispatcher | stage="dispatch_read" | Timer | The distribution plan dispatcher time-consuming (only query) |
#### 4.2.19. Query Resource
+
| Metric | Tags | Type | Description |
|----------------|--------------------------|------|--------------------------------------------|
| query_resource | type="sequence_tsfile" | Rate | The access frequency of sequence tsfiles |
@@ -305,6 +347,7 @@ Core-level metrics are enabled by default during system operation. The addition
| query_resource | type="working_memtable" | Rate | The access frequency of working memtables |
#### 4.2.20. Data Exchange
+
| Metric | Tags | Type | Description |
|---------------------|------------------------------------------------------------------------|-----------|-----------------------------------------------------------------|
| data_exchange_cost | operation="source_handle_get_tsblock", type="local/remote" | Timer | The time-consuming that source handles receive TsBlock |
@@ -318,6 +361,7 @@ Core-level metrics are enabled by default during system operation. The addition
| data_exchange_count | name="on_acknowledge_data_block_num", type="server/caller" | Histogram | The number of acknowledged TsBlocks by source handles |
#### 4.2.21. Query Task Schedule
+
| Metric | Tags | Type | Description |
|------------------|--------------------------------|-----------|--------------------------------------------------|
| driver_scheduler | name="ready_queued_time" | Timer | The queuing time of ready queue |
@@ -326,6 +370,7 @@ Core-level metrics are enabled by default during system operation. The addition
| driver_scheduler | name="block_queued_task_count" | AutoGauge | The number of tasks queued in the blocking queue |
#### 4.2.22. Query Execution
+
| Metric | Tags | Type | Description |
|--------------------------|-------------------------------------------------------------------------------------|---------|-----------------------------------------------------------------------------------------|
| query_execution | stage="local_execution_planner" | Timer | The time-consuming of operator tree construction |
@@ -352,12 +397,14 @@ Core-level metrics are enabled by default during system operation. The addition
### 4.3. Normal level Metrics
#### 4.3.1. Cluster
+
| Metric | Tags | Type | Description |
-| ------ | -------------------------------------------------------------- | --------- | ------------------------------------------------------------------ |
+|--------|----------------------------------------------------------------|-----------|--------------------------------------------------------------------|
| region | name="{{DatabaseName}}",type="SchemaRegion/DataRegion" | AutoGauge | The number of DataRegion/SchemaRegion of database in specific node |
| slot | name="{{DatabaseName}}",type="schemaSlotNumber/dataSlotNumber" | AutoGauge | The number of DataSlot/SchemaSlot of database in specific node |
### 4.4. All Metric
+
Currently there is no All level metrics, and it will continue to be added in the future.
## 5. How to get these metrics？
@@ -366,15 +413,22 @@ The relevant configuration of the metric module is in `conf/iotdb-{datanode/conf
configuration items support hot loading through the `load configuration` command.
### 5.1. JMX
-For metrics exposed externally using JMX, you can view them through Jconsole. After entering the Jconsole monitoring page, you will first see an overview of various running conditions of IoTDB. Here you can see heap memory information, thread information, class information, and the server's CPU usage.
+
+For metrics exposed externally using JMX, you can view them through Jconsole. After entering the Jconsole monitoring
+page, you will first see an overview of various running conditions of IoTDB. Here you can see heap memory information,
+thread information, class information, and the server's CPU usage.
#### 5.1.1. Obtain metric data
-After connecting to JMX, you can find the "MBean" named "org.apache.iotdb.metrics" through the "MBeans" tab, and you can view the specific values of all monitoring metrics in the sidebar.
+
+After connecting to JMX, you can find the "MBean" named "org.apache.iotdb.metrics" through the "MBeans" tab, and you can
+view the specific values of all monitoring metrics in the sidebar.
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" alt="metric-jmx" src="https://user-images.githubusercontent.com/46039728/204018765-6fda9391-ebcf-4c80-98c5-26f34bd74df0.png">
#### 5.1.2. Get other relevant data
-After connecting to JMX, you can find the "MBean" named "org.apache.iotdb.service" through the "MBeans" tab, as shown in the image below, to understand the basic status of the service
+
+After connecting to JMX, you can find the "MBean" named "org.apache.iotdb.service" through the "MBeans" tab, as shown in
+the image below, to understand the basic status of the service
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/46039728/149951720-707f1ee8-32ee-4fde-9252-048caebd232e.png"> <br>
@@ -386,17 +440,20 @@ the sidebar `org.apache.iotdb.db.service` to view the cache hit ratio:
### 5.2. Prometheus
#### 5.2.1. The mapping from metric type to prometheus format
-> For metrics whose Metric Name is name and Tags are K1=V1, ..., Kn=Vn, the mapping is as follows, where value is a specific value
-| Metric Type | Mapping |
-| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Counter | name_total{k1="V1", ..., Kn="Vn"} value |
+> For metrics whose Metric Name is name and Tags are K1=V1, ..., Kn=Vn, the mapping is as follows, where value is a
+> specific value
+
+| Metric Type | Mapping |
+|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Counter | name_total{k1="V1", ..., Kn="Vn"} value |
| AutoGauge、Gauge | name{k1="V1", ..., Kn="Vn"} value |
-| Histogram | name_max{k1="V1", ..., Kn="Vn"} value <br> name_sum{k1="V1", ..., Kn="Vn"} value <br> name_count{k1="V1", ..., Kn="Vn"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
-| Rate | name_total{k1="V1", ..., Kn="Vn"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m1"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m5"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m15"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="mean"} value |
-| Timer | name_seconds_max{k1="V1", ..., Kn="Vn"} value <br> name_seconds_sum{k1="V1", ..., Kn="Vn"} value <br> name_seconds_count{k1="V1", ..., Kn="Vn"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
+| Histogram | name_max{k1="V1", ..., Kn="Vn"} value <br> name_sum{k1="V1", ..., Kn="Vn"} value <br> name_count{k1="V1", ..., Kn="Vn"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
+| Rate | name_total{k1="V1", ..., Kn="Vn"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m1"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m5"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m15"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="mean"} value |
+| Timer | name_seconds_max{k1="V1", ..., Kn="Vn"} value <br> name_seconds_sum{k1="V1", ..., Kn="Vn"} value <br> name_seconds_count{k1="V1", ..., Kn="Vn"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
#### 5.2.2. Config File
+
1) Taking DataNode as an example, modify the iotdb-datanode.properties configuration file as follows:
```properties
@@ -465,6 +522,7 @@ The following documents may help you have a good journey with Prometheus and Gra
[Grafana query metrics from Prometheus](https://prometheus.io/docs/visualization/grafana/#grafana-support-for-prometheus)
#### 5.2.4. Apache IoTDB Dashboard
+
We provide the Apache IoTDB Dashboard, and the rendering shown in Grafana is as follows:
@@ -472,8 +530,10 @@ We provide the Apache IoTDB Dashboard, and the rendering shown in Grafana is as
##### 5.2.4.1. How to get Apache IoTDB Dashboard
1. You can obtain the json files of Dashboards by GitHub:
- 1. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-ConfigNode-Dashboard.json">Apache IoTDB ConfigNode Dashboard</a>
- 2. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-DataNode-Dashboard.json">Apache IoTDB DataNode Dashboard</a>
+ 1. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-ConfigNode-Dashboard.json">
+ Apache IoTDB ConfigNode Dashboard</a>
+ 2. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-DataNode-Dashboard.json">
+ Apache IoTDB DataNode Dashboard</a>
2. You can visit [Grafana Dashboard official website](https://grafana.com/grafana/dashboards/), search
for `Apache IoTDB Dashboard` and use
@@ -481,12 +541,16 @@ When creating Grafana, you can select the json file you just downloaded to `Impo
data source for Apache IoTDB Dashboard.
##### 5.2.4.2. Apache IoTDB ConfigNode Dashboard Instructions
-> Except for the metrics specified specially, the following metrics are guaranteed to be available in the monitoring framework at the Important levels.
+
+> Except for the metrics specified specially, the following metrics are guaranteed to be available in the monitoring
+> framework at the Important levels.
- `Overview`: system overview
- `Registered Node`: The number of registered ConfigNode/DataNode
- - `DataNode`(Only visible in the leader of ConfigNode): The status of the cluster DataNode, including Online and Unknown.
- - `ConfigNode`(Only visible in the leader of ConfigNode): The status of the cluster ConfigNode, including Online and Unknown.
+ - `DataNode`(Only visible in the leader of ConfigNode): The status of the cluster DataNode, including Online and
+ Unknown.
+ - `ConfigNode`(Only visible in the leader of ConfigNode): The status of the cluster ConfigNode, including Online and
+ Unknown.
- `The Status Of Node`: The status of specific nodes in the cluster, including Online and Unknown.
- `Region`: Region overview
- `Region Number`: the number of Regions, including the total number, the number of DataRegions and the number of
@@ -517,9 +581,11 @@ data source for Apache IoTDB Dashboard.
- `Log Number Per Minute`: the average number of logs per minute of the IoTDB process
- `The Time Consumed of Compliation Per Minute`: average compilation time per minute
- `The Number Of Class`: The number of classes loaded and unloaded by the JVM
-
+
##### 5.2.4.3. Apache IoTDB DataNode Dashboard Instructions
-> Except for the metrics specified specially, the following metrics are guaranteed to be available in the monitoring framework at the Important levels.
+
+> Except for the metrics specified specially, the following metrics are guaranteed to be available in the monitoring
+> framework at the Important levels.
- `Overview`: system overview
- `The Number Of Entity`: the number of entities, including time series, etc.
@@ -535,36 +601,62 @@ data source for Apache IoTDB Dashboard.
- `Thrift Connection`: the number of Thrift connections established
- `Thrift Active Thread`: The number of active Thrift connections established
- `Query Engine`: query engine
- - `The time consumed of query plan stages(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of the query plan stages time consumption
- - `The time consumed of plan dispatch stages(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of query plan dispatch stages time consumption
- - `The time consumed of query execution stages(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the query execution stages time consumption
- - `The time consumed of operator execution stages(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of operator exection time consumption
- - `The time consumed of query aggregation(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of query aggregation time consumption
- - `The time consumed of query scan(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of query file/memory time consumption
- - `The usage of query resource(avg\50%\75%\100%)`: the average\median\upper quartile\maximum of the number of accesses to different resources
- - `The time consumed of query data exchange(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of query data exchange time consumption
- - `The count of data exchange(avg)`: the average number of query data exchange
- - `The count of data exchange`: the distribution of the number of query data exchange (minimum, lower quartile, median, upper quartile, maximum)
- - `The number of query queue`: the size of different query queues
- - `The time consumed of query schedule time(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of query task scheduling time
+ - `The time consumed of query plan stages(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of the
+ query plan stages time consumption
+ - `The time consumed of plan dispatch stages(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of
+ query plan dispatch stages time consumption
+ - `The time consumed of query execution stages(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value
+ of the query execution stages time consumption
+ - `The time consumed of operator execution stages(avg\50%\75%\100%)`: the average\median\upper quartile\maximum
+ value of operator exection time consumption
+ - `The time consumed of query aggregation(avg\50%\75%\100%)`: The average\median\upper quartile\maximum value of
+ query aggregation time consumption
+ - `The time consumed of query scan(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of query
+ file/memory time consumption
+ - `The usage of query resource(avg\50%\75%\100%)`: the average\median\upper quartile\maximum of the number of
+ accesses to different resources
+ - `The time consumed of query data exchange(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of
+ query data exchange time consumption
+ - `The count of data exchange(avg)`: the average number of query data exchange
+ - `The count of data exchange`: the distribution of the number of query data exchange (minimum, lower quartile,
+ median, upper quartile, maximum)
+ - `The number of query queue`: the size of different query queues
+ - `The time consumed of query schedule time(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of
+ query task scheduling time
- `Query Interface`: Query files/time-consuming specific time-consuming conditions
- - `The time consumed of load timesereis metadata(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query spent loading timeseries metadata from different sources
- - `The time consumed of read timeseries metadata(avg\50%\75%\100%)`: the average\median\upper quartile\maximum of time query spent reading time seriesmetadata from different sources value
- - `The time consumed of timeseries metadata modification(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to modify different types of timeseries metadata
- - `The time consumed of load chunk metadata list(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query spent loading different types of chunk metadata
- - `The time consumed of chunk metadata modification(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to modify different types of chunk metadata
- - `The time consumed of chunk metadata filter(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query spent to filter different types of chunk metadata
- - `The time consumed of construct chunk reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query spent on constructing different types of chunk readers
- - `The time consumed of read chunk(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to read different types of Chunk
- - `The time consumed of init chunk reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to initialize different types of chunk readers
- - `The time consumed of build tsblock from page reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to construct TsBlock from Page Reader
- - `The time consumed of build tsblock from merge reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to construct TsBlock from Merge Reader
+ - `The time consumed of load timesereis metadata(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value
+ of the time query spent loading timeseries metadata from different sources
+ - `The time consumed of read timeseries metadata(avg\50%\75%\100%)`: the average\median\upper quartile\maximum of
+ time query spent reading time seriesmetadata from different sources value
+ - `The time consumed of timeseries metadata modification(avg\50%\75%\100%)`: the average\median\upper
+ quartile\maximum value of the time query consumed to modify different types of timeseries metadata
+ - `The time consumed of load chunk metadata list(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value
+ of the time query spent loading different types of chunk metadata
+ - `The time consumed of chunk metadata modification(avg\50%\75%\100%)`: the average\median\upper quartile\maximum
+ value of the time query consumed to modify different types of chunk metadata
+ - `The time consumed of chunk metadata filter(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of
+ the time query spent to filter different types of chunk metadata
+ - `The time consumed of construct chunk reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value
+ of the time query spent on constructing different types of chunk readers
+ - `The time consumed of read chunk(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time
+ query consumed to read different types of Chunk
+ - `The time consumed of init chunk reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the
+ time query consumed to initialize different types of chunk readers
+ - `The time consumed of build tsblock from page reader(avg\50%\75%\100%)`: the average\median\upper quartile\maximum
+ value of the time query consumed to construct TsBlock from Page Reader
+ - `The time consumed of build tsblock from merge reader(avg\50%\75%\100%)`: the average\median\upper
+ quartile\maximum value of the time query consumed to construct TsBlock from Merge Reader
- `Query Data Exchange`: Query the specific time-consuming situation of data transmission
- - `The time consumed of source handle get tsblock(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query consumed to obtain TsBlock from different sources
- - `The time consumed of source handle deserialize tsblock(avg\50%\75%\100%)`: the average\median\upper quartile\maximum of the time query consumed to deserialize TsBlock from different sources value
- - `The time consumed of sink handle send tsblock(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of the time query spent sending TsBlock to different places
- - `The time consumed of on acknowledge data block event task(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value of time query consumed to acknowlege data block event from different places
- - `The time consumed of get data block event task(avg\50%\75%\100%)`: query the average\median\upper quartile\maximum value of time query consumed to get data block from different places
+ - `The time consumed of source handle get tsblock(avg\50%\75%\100%)`: the average\median\upper quartile\maximum
+ value of the time query consumed to obtain TsBlock from different sources
+ - `The time consumed of source handle deserialize tsblock(avg\50%\75%\100%)`: the average\median\upper
+ quartile\maximum of the time query consumed to deserialize TsBlock from different sources value
+ - `The time consumed of sink handle send tsblock(avg\50%\75%\100%)`: the average\median\upper quartile\maximum value
+ of the time query spent sending TsBlock to different places
+ - `The time consumed of on acknowledge data block event task(avg\50%\75%\100%)`: the average\median\upper
+ quartile\maximum value of time query consumed to acknowlege data block event from different places
+ - `The time consumed of get data block event task(avg\50%\75%\100%)`: query the average\median\upper
+ quartile\maximum value of time query consumed to get data block from different places
- `Engine`:
- `Task Number`: the number of tasks in different states in the system
- `The Time Consumed Of Tasking`: Time consumption of tasks in different states in the system
@@ -613,15 +705,18 @@ data source for Apache IoTDB Dashboard.
### 5.3. IoTDB
#### 5.3.1. IoTDB mapping relationship of metrics
-> For metrics whose Metric Name is name and Tags are K1=V1, ..., Kn=Vn, the mapping is as follows, taking root.__system.metric.`ip:port` as an example by default
-| Metric Type | Mapping [...]
-| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
-| Counter | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
+> For metrics whose Metric Name is name and Tags are K1=V1, ..., Kn=Vn, the mapping is as follows, taking root.__
+> system.metric.`ip:port` as an example by default
+
+| Metric Type | Mapping [...]
+|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Counter | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
| AutoGauge、Gauge | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
-| Histogram | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p75 <br> root.__system.metric.`ip:port [...]
-| Rate | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m1 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m5 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m15 [...]
-| Timer | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:por [...]
+| Histogram | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p75 <br> root.__system.metric.`ip:port` [...]
+| Rate | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m1 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m5 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m15 [...]
+| Timer | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port [...]
#### 5.3.2. Obtain metrics
+
According to the above mapping relationship, related IoTDB query statements can be formed to obtain metrics
\ No newline at end of file
diff --git a/docs/zh/UserGuide/Monitor-Alert/Metric-Tool.md b/docs/zh/UserGuide/Monitor-Alert/Metric-Tool.md
index cdc4a349c3..2934caf054 100644
--- a/docs/zh/UserGuide/Monitor-Alert/Metric-Tool.md
+++ b/docs/zh/UserGuide/Monitor-Alert/Metric-Tool.md
@@ -19,7 +19,8 @@
-->
-在 IoTDB 的运行过程中，我们希望对 IoTDB 的状态进行观测，以便于排查系统问题或者及时发现系统潜在的风险，能够**反映系统运行状态的一系列指标**就是系统监控指标。
+在 IoTDB 的运行过程中，我们希望对 IoTDB 的状态进行观测，以便于排查系统问题或者及时发现系统潜在的风险，能够**反映系统运行状态的一系列指标
+**就是系统监控指标。
## 1. 什么场景下会使用到监控?
@@ -56,18 +57,22 @@
- `Metric Name`：指标类型名称，比如`logback_events`表示日志事件。
- `Tags`：指标分类，形式为Key-Value对，每个指标下面可以有0到多个分类，常见的Key-Value对：
- - `name = xxx`：被监控对象的名称，是对**业务逻辑**的说明。比如对于`Metric Name = entry_seconds_count`类型的监控项，name的含义是指被监控的业务接口。
- - `type = xxx`：监控指标类型细分，是对**监控指标**本身的说明。比如对于`Metric Name = point`类型的监控项，type的含义是指监控具体是什么类型的点数。
- - `status = xxx`：被监控对象的状态，是对**业务逻辑**的说明。比如对于`Metric Name = Task`类型的监控项可以通过该参数，从而区分被监控对象的状态。
- - `user = xxx`：被监控对象的相关用户，是对**业务逻辑**的说明。比如统计`root`用户的写入总点数。
- - 根据具体情况自定义：比如logback_events_total下有一个level的分类，用来表示特定级别下的日志数量。
-- `Metric Level`：**指标管理级别**，默认启动级别为`Core`级别，建议启动级别为`Important级别`，审核严格程度`Core > Important > Normal > All`
+ - `name = xxx`：被监控对象的名称，是对**业务逻辑**的说明。比如对于`Metric Name = entry_seconds_count`
+ 类型的监控项，name的含义是指被监控的业务接口。
+ - `type = xxx`：监控指标类型细分，是对**监控指标**本身的说明。比如对于`Metric Name = point`
+ 类型的监控项，type的含义是指监控具体是什么类型的点数。
+ - `status = xxx`：被监控对象的状态，是对**业务逻辑**的说明。比如对于`Metric Name = Task`类型的监控项可以通过该参数，从而区分被监控对象的状态。
+ - `user = xxx`：被监控对象的相关用户，是对**业务逻辑**的说明。比如统计`root`用户的写入总点数。
+ - 根据具体情况自定义：比如logback_events_total下有一个level的分类，用来表示特定级别下的日志数量。
+- `Metric Level`：**指标管理级别**，默认启动级别为`Core`级别，建议启动级别为`Important级别`
+ ，审核严格程度`Core > Important > Normal > All`
- `Core`：系统的核心指标，供**系统内核和运维人员**使用，关乎系统的**性能、稳定性、安全性**，比如实例的状况，系统的负载等。
- `Important`：模块的重要指标，供**运维和测试人员**使用，直接关乎**每个模块的运行状态**，比如合并文件个数、执行情况等。
- `Normal`：模块的一般指标，供**开发人员**使用，方便在出现问题时**定位模块**，比如合并中的特定关键操作情况。
- `All`：模块的全部指标，供**模块开发人员**使用，往往在复现问题的时候使用，从而快速解决问题。
-
+
### 3.2. 监控指标对外获取数据格式
+
- IoTDB 对外提供 JMX、 Prometheus 和 IoTDB 格式的监控指标：
- 对于 JMX ，可以通过```org.apache.iotdb.metrics```获取系统监控指标指标。
- 对于 Prometheus ，可以通过对外暴露的端口获取监控指标的值
@@ -75,7 +80,8 @@
## 4. 监控指标有哪些？
-目前，IoTDB 对外提供一些主要模块的监控指标，并且随着新功能的开发以及系统优化或者重构，监控指标也会同步添加和更新。如果想自己在 IoTDB
+目前，IoTDB 对外提供一些主要模块的监控指标，并且随着新功能的开发以及系统优化或者重构，监控指标也会同步添加和更新。如果想自己在
+IoTDB
中添加更多系统监控指标埋点，可以参考[IoTDB Metrics Framework](https://github.com/apache/iotdb/tree/master/metrics)使用说明。
### 4.1. Core 级别监控指标
@@ -84,275 +90,301 @@ Core 级别的监控指标在系统运行中默认开启，每一个 Core 级别
#### 4.1.1. 集群运行状态
-| Metric | Tags | Type | Description |
-| ----------- | ----------------------------------------------- | --------- | -------------------------------------- |
+| Metric | Tags | Type | Description |
+|-------------|-------------------------------------------------|-----------|----------------------------|
| config_node | name="total",status="Registered/Online/Unknown" | AutoGauge | 已注册/在线/离线 confignode 的节点数量 |
| data_node | name="total",status="Registered/Online/Unknown" | AutoGauge | 已注册/在线/离线 datanode 的节点数量 |
-| points | database="{{database}}", type="flush" | Gauge | 最新一个刷盘的memtale的点数 |
+| points | database="{{database}}", type="flush" | Gauge | 最新一个刷盘的memtale的点数 |
#### 4.1.2. IoTDB 进程运行状态
-| Metric | Tags | Type | Description |
-| ----------------- | -------------- | --------- | ----------------------------------- |
-| process_cpu_load | name="process" | AutoGauge | IoTDB 进程的 CPU 占用率，单位为% |
+
+| Metric | Tags | Type | Description |
+|-------------------|----------------|-----------|--------------------------|
+| process_cpu_load | name="process" | AutoGauge | IoTDB 进程的 CPU 占用率，单位为% |
| process_cpu_time | name="process" | AutoGauge | IoTDB 进程占用的 CPU 时间，单位为ns |
-| process_max_mem | name="memory" | AutoGauge | IoTDB 进程最大可用内存 |
-| process_total_mem | name="memory" | AutoGauge | IoTDB 进程当前已申请内存 |
-| process_free_mem | name="memory" | AutoGauge | IoTDB 进程当前剩余可用内存 |
+| process_max_mem | name="memory" | AutoGauge | IoTDB 进程最大可用内存 |
+| process_total_mem | name="memory" | AutoGauge | IoTDB 进程当前已申请内存 |
+| process_free_mem | name="memory" | AutoGauge | IoTDB 进程当前剩余可用内存 |
#### 4.1.3. 系统运行状态
-| Metric | Tags | Type | Description |
-| ------------------------------ | ------------- | --------- | ---------------------------------------- |
-| sys_cpu_load | name="system" | AutoGauge | 系统的 CPU 占用率，单位为% |
-| sys_cpu_cores | name="system" | Gauge | 系统的可用处理器数 |
-| sys_total_physical_memory_size | name="memory" | Gauge | 系统的最大物理内存 |
-| sys_free_physical_memory_size | name="memory" | AutoGauge | 系统的剩余可用内存 |
-| sys_total_swap_space_size | name="memory" | AutoGauge | 系统的交换区最大空间 |
-| sys_free_swap_space_size | name="memory" | AutoGauge | 系统的交换区剩余可用空间 |
+
+| Metric | Tags | Type | Description |
+|--------------------------------|---------------|-----------|----------------------|
+| sys_cpu_load | name="system" | AutoGauge | 系统的 CPU 占用率，单位为% |
+| sys_cpu_cores | name="system" | Gauge | 系统的可用处理器数 |
+| sys_total_physical_memory_size | name="memory" | Gauge | 系统的最大物理内存 |
+| sys_free_physical_memory_size | name="memory" | AutoGauge | 系统的剩余可用内存 |
+| sys_total_swap_space_size | name="memory" | AutoGauge | 系统的交换区最大空间 |
+| sys_free_swap_space_size | name="memory" | AutoGauge | 系统的交换区剩余可用空间 |
| sys_committed_vm_size | name="memory" | AutoGauge | 系统保证可用于正在运行的进程的虚拟内存量 |
-| sys_disk_total_space | name="disk" | AutoGauge | 系统磁盘总大小 |
-| sys_disk_free_space | name="disk" | AutoGauge | 系统磁盘可用大小 |
+| sys_disk_total_space | name="disk" | AutoGauge | 系统磁盘总大小 |
+| sys_disk_free_space | name="disk" | AutoGauge | 系统磁盘可用大小 |
### 4.2. Important 级别监控指标
目前 Important 级别的监控指标如下所述：
#### 4.2.1. 集群运行状态
-| Metric | Tags | Type | Description |
-| ------------------------- | ------------------------------------------------- | ----- | ------------------------------ |
-| cluster_node_leader_count | name="{{ip}}:{{port}}" | Gauge | 节点上共识组Leader的数量 |
+
+| Metric | Tags | Type | Description |
+|---------------------------|---------------------------------------------------|-------|--------------------------|
+| cluster_node_leader_count | name="{{ip}}:{{port}}" | Gauge | 节点上共识组Leader的数量 |
| cluster_node_status | name="{{ip}}:{{port}}",type="ConfigNode/DataNode" | Gauge | 节点的状态，0=Unkonwn 1=online |
#### 4.2.2. 节点统计
-| Metric | Tags | Type | Description |
-| -------- | ------------------------------------------ | --------- | ------------------------------------ |
-| quantity | name="database" | AutoGauge | 系统数据库数量 |
-| quantity | name="timeSeries" | AutoGauge | 系统时间序列数量 |
-| quantity | name="pointsIn" | Counter | 系统累计写入点数 |
-| region | name="total",type="SchemaRegion" | AutoGauge | 分区表中 SchemaRegion 总数量 |
-| region | name="total",type="DataRegion" | AutoGauge | 分区表中 DataRegion 总数量 |
+
+| Metric | Tags | Type | Description |
+|----------|--------------------------------------------|-----------|--------------------------|
+| quantity | name="database" | AutoGauge | 系统数据库数量 |
+| quantity | name="timeSeries" | AutoGauge | 系统时间序列数量 |
+| quantity | name="pointsIn" | Counter | 系统累计写入点数 |
+| region | name="total",type="SchemaRegion" | AutoGauge | 分区表中 SchemaRegion 总数量 |
+| region | name="total",type="DataRegion" | AutoGauge | 分区表中 DataRegion 总数量 |
| region | name="{{ip}}:{{port}}",type="SchemaRegion" | Gauge | 分区表中对应节点上 DataRegion 总数量 |
| region | name="{{ip}}:{{port}}",type="DataRegion" | Gauge | 分区表中对应节点上 DataRegion 总数量 |
#### 4.2.3. IoT共识协议统计
-| Metric | Tags | Type | Description |
-| ------------- | -------------------------------------------------------------------------------------------- | --------- | -------------------------------- |
-| iot_consensus | name="logDispatcher-{{IP}}:{{Port}}", region="{{region}}", type="currentSyncIndex" | AutoGauge | 副本组同步线程的当前同步进度 |
+
+| Metric | Tags | Type | Description |
+|---------------|----------------------------------------------------------------------------------------------|-----------|------------------|
+| iot_consensus | name="logDispatcher-{{IP}}:{{Port}}", region="{{region}}", type="currentSyncIndex" | AutoGauge | 副本组同步线程的当前同步进度 |
| iot_consensus | name="logDispatcher-{{IP}}:{{Port}}", region="{{region}}", type="cachedRequestInMemoryQueue" | AutoGauge | 副本组同步线程缓存队列请求总大小 |
-| iot_consensus | name="IoTConsensusServerImpl", region="{{region}}", type="searchIndex" | AutoGauge | 副本组主流程写入进度 |
-| iot_consensus | name="IoTConsensusServerImpl", region="{{region}}", type="safeIndex" | AutoGauge | 副本组同步进度 |
-| stage | name="iot_consensus", region="{{region}}", type="getStateMachineLock" | Histogram | 主流程获取状态机锁耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="checkingBeforeWrite" | Histogram | 主流程写入状态机检查耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="writeStateMachine" | Histogram | 主流程写入状态机耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="offerRequestToQueue" | Histogram | 主流程尝试添加队列耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="consensusWrite" | Histogram | 主流程全写入耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="constructBatch" | Histogram | 同步线程构造 Batch 耗时 |
-| stage | name="iot_consensus", region="{{region}}", type="syncLogTimePerRequest" | Histogram | 异步回调流程同步日志耗时 |
+| iot_consensus | name="IoTConsensusServerImpl", region="{{region}}", type="searchIndex" | AutoGauge | 副本组主流程写入进度 |
+| iot_consensus | name="IoTConsensusServerImpl", region="{{region}}", type="safeIndex" | AutoGauge | 副本组同步进度 |
+| stage | name="iot_consensus", region="{{region}}", type="getStateMachineLock" | Histogram | 主流程获取状态机锁耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="checkingBeforeWrite" | Histogram | 主流程写入状态机检查耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="writeStateMachine" | Histogram | 主流程写入状态机耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="offerRequestToQueue" | Histogram | 主流程尝试添加队列耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="consensusWrite" | Histogram | 主流程全写入耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="constructBatch" | Histogram | 同步线程构造 Batch 耗时 |
+| stage | name="iot_consensus", region="{{region}}", type="syncLogTimePerRequest" | Histogram | 异步回调流程同步日志耗时 |
#### 4.2.4. 缓存统计
-| Metric | Tags | Type | Description |
-| --------- | ---------------------------------- | --------- | ------------------------------------------------------- |
-| cache_hit | name="chunk" | AutoGauge | ChunkCache的命中率，单位为% |
-| cache_hit | name="schema" | AutoGauge | SchemaCache的命中率，单位为% |
-| cache_hit | name="timeSeriesMeta" | AutoGauge | TimeseriesMetadataCache的命中率，单位为% |
+
+| Metric | Tags | Type | Description |
+|-----------|------------------------------------|-----------|-----------------------------------------------|
+| cache_hit | name="chunk" | AutoGauge | ChunkCache的命中率，单位为% |
+| cache_hit | name="schema" | AutoGauge | SchemaCache的命中率，单位为% |
+| cache_hit | name="timeSeriesMeta" | AutoGauge | TimeseriesMetadataCache的命中率，单位为% |
| cache_hit | name="bloomFilter" | AutoGauge | TimeseriesMetadataCache中的bloomFilter的拦截率，单位为% |
-| cache | name="Database", type="hit" | Counter | Database Cache 的命中次数 |
-| cache | name="Database", type="all" | Counter | Database Cache 的访问次数 |
-| cache | name="SchemaPartition", type="hit" | Counter | SchemaPartition Cache 的命中次数 |
-| cache | name="SchemaPartition", type="all" | Counter | SchemaPartition Cache 的访问次数 |
-| cache | name="DataPartition", type="hit" | Counter | DataPartition Cache 的命中次数 |
-| cache | name="DataPartition", type="all" | Counter | DataPartition Cache 的访问次数 |
+| cache | name="Database", type="hit" | Counter | Database Cache 的命中次数 |
+| cache | name="Database", type="all" | Counter | Database Cache 的访问次数 |
+| cache | name="SchemaPartition", type="hit" | Counter | SchemaPartition Cache 的命中次数 |
+| cache | name="SchemaPartition", type="all" | Counter | SchemaPartition Cache 的访问次数 |
+| cache | name="DataPartition", type="hit" | Counter | DataPartition Cache 的命中次数 |
+| cache | name="DataPartition", type="all" | Counter | DataPartition Cache 的访问次数 |
#### 4.2.5. 接口层统计
-| Metric | Tags | Type | Description |
-| --------------------- | ---------------------------------------------------- | --------- | ----------------------------------- |
-| statement_execution | interface="{{interface}}", type="{{statement_type}}" | Timer | 客户端执行的操作的耗时情况 |
+
+| Metric | Tags | Type | Description |
+|-----------------------|------------------------------------------------------|-----------|-----------------------------|
+| statement_execution | interface="{{interface}}", type="{{statement_type}}" | Timer | 客户端执行的操作的耗时情况 |
| entry | name="{{interface}}" | Timer | Client 建立的 Thrift 的耗时情况 |
-| thrift_connections | name="ConfigNodeRPC" | AutoGauge | ConfigNode 的内部 Thrift 连接数 |
-| thrift_connections | name="Internal" | AutoGauge | DataNode 的内部 Thrift 连接数 |
+| thrift_connections | name="ConfigNodeRPC" | AutoGauge | ConfigNode 的内部 Thrift 连接数 |
+| thrift_connections | name="Internal" | AutoGauge | DataNode 的内部 Thrift 连接数 |
| thrift_connections | name="MPPDataExchange" | AutoGauge | MPP 框架的内部 Thrift 连接数 |
-| thrift_connections | name="RPC" | AutoGauge | Client 建立的 Thrift 连接数 |
+| thrift_connections | name="RPC" | AutoGauge | Client 建立的 Thrift 连接数 |
| thrift_active_threads | name="ConfigNodeRPC-Service" | AutoGauge | ConfigNode 的内部活跃 Thrift 连接数 |
| thrift_active_threads | name="DataNodeInternalRPC-Service" | AutoGauge | DataNode 的内部活跃 Thrift 连接数 |
-| thrift_active_threads | name="MPPDataExchangeRPC-Service" | AutoGauge | MPP 框架的内部活跃 Thrift 连接数 |
+| thrift_active_threads | name="MPPDataExchangeRPC-Service" | AutoGauge | MPP 框架的内部活跃 Thrift 连接数 |
| thrift_active_threads | name="ClientRPC-Service" | AutoGauge | Client 建立的活跃 Thrift 连接数 |
#### 4.2.6. 内存统计
-| Metric | Tags | Type | Description |
-| ------ | ------------------------------------ | --------- | ------------------------------------------------- |
-| mem | name="database_{{name}}" | AutoGauge | DataNode内对应DataRegion的内存占用，单位为byte |
+
+| Metric | Tags | Type | Description |
+|--------|--------------------------------------|-----------|--------------------------------------|
+| mem | name="database_{{name}}" | AutoGauge | DataNode内对应DataRegion的内存占用，单位为byte |
| mem | name="chunkMetaData_{{name}}" | AutoGauge | 写入TsFile时的ChunkMetaData的内存占用，单位为byte |
| mem | name="IoTConsensus" | AutoGauge | IoT共识协议的内存占用，单位为byte |
-| mem | name="schema_region_total_usage" | AutoGauge | 所有SchemaRegion的总内存占用，单位为byte |
-| mem | name="schema_region_total_remaining" | AutoGauge | 所有SchemaRegion的总内存剩余，单位为byte |
+| mem | name="schema_region_total_usage" | AutoGauge | 所有SchemaRegion的总内存占用，单位为byte |
+| mem | name="schema_region_total_remaining" | AutoGauge | 所有SchemaRegion的总内存剩余，单位为byte |
#### 4.2.7. 任务统计
-| Metric | Tags | Type | Description |
-| --------- | ------------------------------------------------- | --------- | ------------------ |
-| queue | name="compaction_inner", status="running/waiting" | Gauge | 空间内合并任务数 |
-| queue | name="compaction_cross", status="running/waiting" | Gauge | 跨空间合并任务数 |
-| cost_task | name="inner_compaction/cross_compaction/flush" | Gauge | 任务耗时情况 |
-| queue | name="flush",status="running/waiting" | AutoGauge | 刷盘任务数 |
+
+| Metric | Tags | Type | Description |
+|-----------|---------------------------------------------------|-----------|-------------|
+| queue | name="compaction_inner", status="running/waiting" | Gauge | 空间内合并任务数 |
+| queue | name="compaction_cross", status="running/waiting" | Gauge | 跨空间合并任务数 |
+| cost_task | name="inner_compaction/cross_compaction/flush" | Gauge | 任务耗时情况 |
+| queue | name="flush",status="running/waiting" | AutoGauge | 刷盘任务数 |
#### 4.2.8. 合并统计
-| Metric | Tags | Type | Description |
-| --------------------- | --------------------------------------------------- | ------- | ------------------ |
-| data_written | name="compaction", type="aligned/not-aligned/total" | Counter | 合并时写入量 |
+
+| Metric | Tags | Type | Description |
+|-----------------------|-----------------------------------------------------|---------|-------------|
+| data_written | name="compaction", type="aligned/not-aligned/total" | Counter | 合并时写入量 |
| data_read | name="compaction" | Counter | 合并时的读取量 |
-| compaction_task_count | name = "inner_compaction", type="sequence" | Counter | 顺序空间内合并次数 |
-| compaction_task_count | name = "inner_compaction", type="unsequence" | Counter | 乱序空间内合并次数 |
+| compaction_task_count | name = "inner_compaction", type="sequence" | Counter | 顺序空间内合并次数 |
+| compaction_task_count | name = "inner_compaction", type="unsequence" | Counter | 乱序空间内合并次数 |
| compaction_task_count | name = "cross_compaction", type="cross" | Counter | 跨空间合并次数 |
#### 4.2.9. 文件统计信息
-| Metric | Tags | Type | Description |
-| ---------- |---------------------------| --------- | ---------------------------------------- |
-| file_size | name="wal" | AutoGauge | 写前日志总大小，单位为byte |
-| file_size | name="seq" | AutoGauge | 顺序TsFile总大小，单位为byte |
-| file_size | name="unseq" | AutoGauge | 乱序TsFile总大小，单位为byte |
-| file_size | name="inner-seq-temp" | AutoGauge | 顺序空间内合并临时文件大小，单位为byte |
-| file_size | name="inner-unseq-temp" | AutoGauge | 乱序空间内合并临时文件大小，单位为byte |
-| file_size | name="cross-temp" | AutoGauge | 跨空间合并临时文件大小，单位为byte |
-| file_count | name="wal" | AutoGauge | 写前日志文件个数 |
-| file_count | name="seq" | AutoGauge | 顺序TsFile文件个数 |
-| file_count | name="unseq" | AutoGauge | 乱序TsFile文件个数 |
-| file_count | name="inner-seq-temp" | AutoGauge | 顺序空间内合并临时文件个数 |
-| file_count | name="inner-unseq-temp" | AutoGauge | 乱序空间内合并临时文件个数 |
-| file_count | name="cross-temp" | AutoGauge | 跨空间合并临时文件个数 |
+| Metric | Tags | Type | Description |
+|------------|---------------------------|-----------|------------------------------|
+| file_size | name="wal" | AutoGauge | 写前日志总大小，单位为byte |
+| file_size | name="seq" | AutoGauge | 顺序TsFile总大小，单位为byte |
+| file_size | name="unseq" | AutoGauge | 乱序TsFile总大小，单位为byte |
+| file_size | name="inner-seq-temp" | AutoGauge | 顺序空间内合并临时文件大小，单位为byte |
+| file_size | name="inner-unseq-temp" | AutoGauge | 乱序空间内合并临时文件大小，单位为byte |
+| file_size | name="cross-temp" | AutoGauge | 跨空间合并临时文件大小，单位为byte |
+| file_size | name="mods" | AutoGauge | Modification 文件的大小 |
+| file_count | name="wal" | AutoGauge | 写前日志文件个数 |
+| file_count | name="seq" | AutoGauge | 顺序TsFile文件个数 |
+| file_count | name="unseq" | AutoGauge | 乱序TsFile文件个数 |
+| file_count | name="inner-seq-temp" | AutoGauge | 顺序空间内合并临时文件个数 |
+| file_count | name="inner-unseq-temp" | AutoGauge | 乱序空间内合并临时文件个数 |
+| file_count | name="cross-temp" | AutoGauge | 跨空间合并临时文件个数 |
| file_count | name="open_file_handlers" | AutoGauge | IoTDB 进程打开文件数，仅支持Linux和MacOS |
+| file_count | name="mods | AutoGauge | Modification 文件的数目 |
#### 4.2.10. IoTDB 进程统计
-| Metric | Tags | Type | Description |
-| --------------------- | -------------- | --------- | ------------------------------------ |
-| process_used_mem | name="memory" | AutoGauge | IoTDB 进程当前使用内存 |
-| process_mem_ratio | name="memory" | AutoGauge | IoTDB 进程的内存占用比例 |
-| process_threads_count | name="process" | AutoGauge | IoTDB 进程当前线程数 |
+
+| Metric | Tags | Type | Description |
+|-----------------------|----------------|-----------|------------------------|
+| process_used_mem | name="memory" | AutoGauge | IoTDB 进程当前使用内存 |
+| process_mem_ratio | name="memory" | AutoGauge | IoTDB 进程的内存占用比例 |
+| process_threads_count | name="process" | AutoGauge | IoTDB 进程当前线程数 |
| process_status | name="process" | AutoGauge | IoTDB 进程存活状态，1为存活，0为终止 |
#### 4.2.11. IoTDB 日志统计
-| Metric | Tags | Type | Description |
-| -------------- | ----------------------------------- | ------- | ------------------ |
-| logback_events | level="trace/debug/info/warn/error" | Counter | 不同类型的日志个数 |
+
+| Metric | Tags | Type | Description |
+|----------------|-------------------------------------|---------|-------------|
+| logback_events | level="trace/debug/info/warn/error" | Counter | 不同类型的日志个数 |
#### 4.2.12. JVM 线程统计
-| Metric | Tags | Type | Description |
-| -------------------------- | ------------------------------------------------------------- | --------- | ------------------------ |
-| jvm_threads_live_threads | | AutoGauge | 当前线程数 |
-| jvm_threads_daemon_threads | | AutoGauge | 当前 Daemon 线程数 |
-| jvm_threads_peak_threads | | AutoGauge | 峰值线程数 |
-| jvm_threads_states_threads | state="runnable/blocked/waiting/timed-waiting/new/terminated" | AutoGauge | 当前处于各种状态的线程数 |
+| Metric | Tags | Type | Description |
+|----------------------------|---------------------------------------------------------------|-----------|---------------|
+| jvm_threads_live_threads | | AutoGauge | 当前线程数 |
+| jvm_threads_daemon_threads | | AutoGauge | 当前 Daemon 线程数 |
+| jvm_threads_peak_threads | | AutoGauge | 峰值线程数 |
+| jvm_threads_states_threads | state="runnable/blocked/waiting/timed-waiting/new/terminated" | AutoGauge | 当前处于各种状态的线程数 |
#### 4.2.13. JVM GC 统计
-| Metric | Tags | Type | Description |
-| ----------------------------- | ----------------------------------------------------- | --------- | -------------------------------------- |
+
+| Metric | Tags | Type | Description |
+|-------------------------------|-------------------------------------------------------|-----------|-----------------------------|
| jvm_gc_pause | action="end of major GC/end of minor GC",cause="xxxx" | Timer | 不同原因的Young GC/Full GC的次数与耗时 |
| |
| jvm_gc_concurrent_phase_time | action="{{action}}",cause="{{cause}}" | Timer | 不同原因的Young GC/Full GC的次数与耗时 |
| |
| jvm_gc_max_data_size_bytes | | AutoGauge | 老年代内存的历史最大值 |
-| jvm_gc_live_data_size_bytes | | AutoGauge | 老年代内存的使用值 |
-| jvm_gc_memory_promoted_bytes | | Counter | 老年代内存正向增长累计值 |
+| jvm_gc_live_data_size_bytes | | AutoGauge | 老年代内存的使用值 |
+| jvm_gc_memory_promoted_bytes | | Counter | 老年代内存正向增长累计值 |
| jvm_gc_memory_allocated_bytes | | Counter | GC分配内存正向增长累计值 |
#### 4.2.14. JVM 内存统计
-| Metric | Tags | Type | Description |
-| ------------------------------- | ------------------------------- | --------- | -------------------- |
-| jvm_buffer_memory_used_bytes | id="direct/mapped" | AutoGauge | 已经使用的缓冲区大小 |
-| jvm_buffer_total_capacity_bytes | id="direct/mapped" | AutoGauge | 最大缓冲区大小 |
-| jvm_buffer_count_buffers | id="direct/mapped" | AutoGauge | 当前缓冲区数量 |
+
+| Metric | Tags | Type | Description |
+|---------------------------------|---------------------------------|-----------|-------------|
+| jvm_buffer_memory_used_bytes | id="direct/mapped" | AutoGauge | 已经使用的缓冲区大小 |
+| jvm_buffer_total_capacity_bytes | id="direct/mapped" | AutoGauge | 最大缓冲区大小 |
+| jvm_buffer_count_buffers | id="direct/mapped" | AutoGauge | 当前缓冲区数量 |
| jvm_memory_committed_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | 当前申请的内存大小 |
-| jvm_memory_max_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | 最大内存 |
-| jvm_memory_used_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | 已使用内存大小 |
+| jvm_memory_max_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | 最大内存 |
+| jvm_memory_used_bytes | {area="heap/nonheap",id="xxx",} | AutoGauge | 已使用内存大小 |
#### 4.2.15. JVM 类加载统计
-| Metric | Tags | Type | Description |
-| ---------------------------- | ---- | --------- | ------------------- |
+
+| Metric | Tags | Type | Description |
+|------------------------------|------|-----------|--------------|
| jvm_classes_unloaded_classes | | AutoGauge | 累计卸载的class数量 |
| jvm_classes_loaded_classes | | AutoGauge | 累计加载的class数量 |
-#### 4.2.16. JVM 编译时间统计
-| Metric | Tags | Type | Description |
-| ----------------------- | --------------------------------------------- | --------- | ------------------ |
-| jvm_compilation_time_ms | {compiler="HotSpot 64-Bit Tiered Compilers",} | AutoGauge | 耗费在编译上的时间 |
+#### 4.2.16. JVM 编译时间统计
+
+| Metric | Tags | Type | Description |
+|-------------------------|-----------------------------------------------|-----------|-------------|
+| jvm_compilation_time_ms | {compiler="HotSpot 64-Bit Tiered Compilers",} | AutoGauge | 耗费在编译上的时间 |
#### 4.2.17. 查询规划耗时统计
-| Metric | Tags | Type | Description |
-|-----------------|------------------------------|-------|----------------------------|
-| query_plan_cost | stage="sql_parser" | Timer | SQL 解析耗时 |
-| query_plan_cost | stage="analyzer" | Timer | 查询语句分析耗时 |
-| query_plan_cost | stage="logical_planner" | Timer | 查询逻辑计划规划耗时 |
+
+| Metric | Tags | Type | Description |
+|-----------------|------------------------------|-------|---------------|
+| query_plan_cost | stage="sql_parser" | Timer | SQL 解析耗时 |
+| query_plan_cost | stage="analyzer" | Timer | 查询语句分析耗时 |
+| query_plan_cost | stage="logical_planner" | Timer | 查询逻辑计划规划耗时 |
| query_plan_cost | stage="distribution_planner" | Timer | 查询分布式执行计划规划耗时 |
-| query_plan_cost | stage="partition_fetcher" | Timer | 分区信息拉取耗时 |
-| query_plan_cost | stage="schema_fetcher" | Timer | 元数据信息拉取耗时 |
+| query_plan_cost | stage="partition_fetcher" | Timer | 分区信息拉取耗时 |
+| query_plan_cost | stage="schema_fetcher" | Timer | 元数据信息拉取耗时 |
#### 4.2.18. 执行计划分发耗时统计
-| Metric | Tags | Type | Description |
-|------------|---------------------------|-------|----------------------|
-| dispatcher | stage="wait_for_dispatch" | Timer | 分发执行计划耗时 |
-| dispatcher | stage="dispatch_read" | Timer | 查询执行计划发送耗时 |
+
+| Metric | Tags | Type | Description |
+|------------|---------------------------|-------|-------------|
+| dispatcher | stage="wait_for_dispatch" | Timer | 分发执行计划耗时 |
+| dispatcher | stage="dispatch_read" | Timer | 查询执行计划发送耗时 |
#### 4.2.19. 查询资源访问统计
-| Metric | Tags | Type | Description |
-|----------------|--------------------------|------|----------------------------|
-| query_resource | type="sequence_tsfile" | Rate | 顺序文件访问频率 |
-| query_resource | type="unsequence_tsfile" | Rate | 乱序文件访问频率 |
+
+| Metric | Tags | Type | Description |
+|----------------|--------------------------|------|------------------------|
+| query_resource | type="sequence_tsfile" | Rate | 顺序文件访问频率 |
+| query_resource | type="unsequence_tsfile" | Rate | 乱序文件访问频率 |
| query_resource | type="flushing_memtable" | Rate | flushing memtable 访问频率 |
| query_resource | type="working_memtable" | Rate | working memtable 访问频率 |
#### 4.2.20. 数据传输模块统计
-| Metric | Tags | Type | Description |
-|---------------------|------------------------------------------------------------------------|-----------|-----------------------------------------|
-| data_exchange_cost | operation="source_handle_get_tsblock", type="local/remote" | Timer | source handle 接收 TsBlock 耗时 |
+
+| Metric | Tags | Type | Description |
+|---------------------|------------------------------------------------------------------------|-----------|-----------------------------------|
+| data_exchange_cost | operation="source_handle_get_tsblock", type="local/remote" | Timer | source handle 接收 TsBlock 耗时 |
| data_exchange_cost | operation="source_handle_deserialize_tsblock", type="local/remote" | Timer | source handle 反序列化 TsBlock 耗时 |
-| data_exchange_cost | operation="sink_handle_send_tsblock", type="local/remote" | Timer | sink handle 发送 TsBlock 耗时 |
-| data_exchange_cost | operation="send_new_data_block_event_task", type="server/caller" | Timer | sink handle 发送 TsBlock RPC 耗时 |
-| data_exchange_cost | operation="get_data_block_task", type="server/caller" | Timer | source handle 接收 TsBlock RPC 耗时 |
+| data_exchange_cost | operation="sink_handle_send_tsblock", type="local/remote" | Timer | sink handle 发送 TsBlock 耗时 |
+| data_exchange_cost | operation="send_new_data_block_event_task", type="server/caller" | Timer | sink handle 发送 TsBlock RPC 耗时 |
+| data_exchange_cost | operation="get_data_block_task", type="server/caller" | Timer | source handle 接收 TsBlock RPC 耗时 |
| data_exchange_cost | operation="on_acknowledge_data_block_event_task", type="server/caller" | Timer | source handle 确认接收 TsBlock RPC 耗时 |
-| data_exchange_count | name="send_new_data_block_num", type="server/caller" | Histogram | sink handle 发送 TsBlock数量 |
-| data_exchange_count | name="get_data_block_num", type="server/caller" | Histogram | source handle 接收 TsBlock 数量 |
+| data_exchange_count | name="send_new_data_block_num", type="server/caller" | Histogram | sink handle 发送 TsBlock数量 |
+| data_exchange_count | name="get_data_block_num", type="server/caller" | Histogram | source handle 接收 TsBlock 数量 |
| data_exchange_count | name="on_acknowledge_data_block_num", type="server/caller" | Histogram | source handle 确认接收 TsBlock 数量 |
#### 4.2.21. 查询任务调度统计
-| Metric | Tags | Type | Description |
-|------------------|--------------------------------|-----------|--------------------|
-| driver_scheduler | name="ready_queued_time" | Timer | 就绪队列排队时间 |
-| driver_scheduler | name="block_queued_time" | Timer | 阻塞队列排队时间 |
-| driver_scheduler | name="ready_queue_task_count" | AutoGauge | 就绪队列排队任务数 |
-| driver_scheduler | name="block_queued_task_count" | AutoGauge | 阻塞队列排队任务数 |
+
+| Metric | Tags | Type | Description |
+|------------------|--------------------------------|-----------|-------------|
+| driver_scheduler | name="ready_queued_time" | Timer | 就绪队列排队时间 |
+| driver_scheduler | name="block_queued_time" | Timer | 阻塞队列排队时间 |
+| driver_scheduler | name="ready_queue_task_count" | AutoGauge | 就绪队列排队任务数 |
+| driver_scheduler | name="block_queued_task_count" | AutoGauge | 阻塞队列排队任务数 |
#### 4.2.22. 查询执行耗时统计
-| Metric | Tags | Type | Description |
-|--------------------------|-------------------------------------------------------------------------------------|---------|------------------------------------------------|
-| query_execution | stage="local_execution_planner" | Timer | 算子树构造耗时 |
-| query_execution | stage="query_resource_init" | Timer | 查询资源初始化耗时 |
-| query_execution | stage="get_query_resource_from_mem" | Timer | 查询资源内存查询与构造耗时 |
-| query_execution | stage="driver_internal_process" | Timer | Driver 执行耗时 |
+
+| Metric | Tags | Type | Description |
+|--------------------------|-------------------------------------------------------------------------------------|---------|------------------------------------|
+| query_execution | stage="local_execution_planner" | Timer | 算子树构造耗时 |
+| query_execution | stage="query_resource_init" | Timer | 查询资源初始化耗时 |
+| query_execution | stage="get_query_resource_from_mem" | Timer | 查询资源内存查询与构造耗时 |
+| query_execution | stage="driver_internal_process" | Timer | Driver 执行耗时 |
| query_execution | stage="wait_for_result" | Timer | 从resultHandle 获取一次查询结果的耗时 |
-| operator_execution_cost | name="{{operator_name}}" | Timer | 算子执行耗时 |
-| operator_execution_count | name="{{operator_name}}" | Counter | 算子调用次数（以 next 方法调用次数计算） |
-| aggregation | from="raw_data" | Timer | 从一批原始数据进行一次聚合计算的耗时 |
-| aggregation | from="statistics" | Timer | 使用统计信息更新一次聚合值的耗时 |
-| series_scan_cost | stage="load_timeseries_metadata", type="aligned/non_aligned", from="mem/disk" | Timer | 加载 TimeseriesMetadata 耗时 |
-| series_scan_cost | stage="read_timeseries_metadata", type="", from="cache/file" | Timer | 读取一个文件的 Metadata 耗时 |
-| series_scan_cost | stage="timeseries_metadata_modification", type="aligned/non_aligned", from="null" | Timer | 过滤删除的 TimeseriesMetadata 耗时 |
-| series_scan_cost | stage="load_chunk_metadata_list", type="aligned/non_aligned", from="mem/disk" | Timer | 加载 ChunkMetadata 列表耗时 |
-| series_scan_cost | stage="chunk_metadata_modification", type="aligned/non_aligned", from="mem/disk" | Timer | 过滤删除的 ChunkMetadata 耗时 |
+| operator_execution_cost | name="{{operator_name}}" | Timer | 算子执行耗时 |
+| operator_execution_count | name="{{operator_name}}" | Counter | 算子调用次数（以 next 方法调用次数计算） |
+| aggregation | from="raw_data" | Timer | 从一批原始数据进行一次聚合计算的耗时 |
+| aggregation | from="statistics" | Timer | 使用统计信息更新一次聚合值的耗时 |
+| series_scan_cost | stage="load_timeseries_metadata", type="aligned/non_aligned", from="mem/disk" | Timer | 加载 TimeseriesMetadata 耗时 |
+| series_scan_cost | stage="read_timeseries_metadata", type="", from="cache/file" | Timer | 读取一个文件的 Metadata 耗时 |
+| series_scan_cost | stage="timeseries_metadata_modification", type="aligned/non_aligned", from="null" | Timer | 过滤删除的 TimeseriesMetadata 耗时 |
+| series_scan_cost | stage="load_chunk_metadata_list", type="aligned/non_aligned", from="mem/disk" | Timer | 加载 ChunkMetadata 列表耗时 |
+| series_scan_cost | stage="chunk_metadata_modification", type="aligned/non_aligned", from="mem/disk" | Timer | 过滤删除的 ChunkMetadata 耗时 |
| series_scan_cost | stage="chunk_metadata_filter", type="aligned/non_aligned", from="mem/disk" | Timer | 根据查询过滤条件过滤 ChunkMetadata 耗时 |
-| series_scan_cost | stage="construct_chunk_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 构造 ChunkReader 耗时 |
-| series_scan_cost | stage="read_chunk", type="", from="cache/file" | Timer | 读取 Chunk 的耗时 |
-| series_scan_cost | stage="init_chunk_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 初始化 ChunkReader（构造 PageReader） 耗时 |
-| series_scan_cost | stage="build_tsblock_from_page_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 从 PageReader 构造 Tsblock 耗时 |
+| series_scan_cost | stage="construct_chunk_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 构造 ChunkReader 耗时 |
+| series_scan_cost | stage="read_chunk", type="", from="cache/file" | Timer | 读取 Chunk 的耗时 |
+| series_scan_cost | stage="init_chunk_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 初始化 ChunkReader（构造 PageReader） 耗时 |
+| series_scan_cost | stage="build_tsblock_from_page_reader", type="aligned/non_aligned", from="mem/disk" | Timer | 从 PageReader 构造 Tsblock 耗时 |
| series_scan_cost | stage="build_tsblock_from_merge_reader", type="aligned/non_aligned", from="null" | Timer | 从 MergeReader 构造 Tsblock （解乱序数据）耗时 |
### 4.3. Normal 级别监控指标
#### 4.3.1. 集群
-| Metric | Tags | Type | Description |
-| ------ | -------------------------------------------------------------- | --------- | ------------------------------------------------------- |
+
+| Metric | Tags | Type | Description |
+|--------|----------------------------------------------------------------|-----------|-----------------------------------------------|
| region | name="{{DatabaseName}}",type="SchemaRegion/DataRegion" | AutoGauge | 特定节点上不同 Database 的 DataRegion/SchemaRegion 个数 |
| slot | name="{{DatabaseName}}",type="schemaSlotNumber/dataSlotNumber" | AutoGauge | 特定节点上不同 Database 的 DataSlot/SchemaSlot 个数 |
### 4.4. All 级别监控指标
+
目前还没有All级别的监控指标，后续会持续添加。
## 5. 怎样获取这些系统监控？
@@ -360,36 +392,43 @@ Core 级别的监控指标在系统运行中默认开启，每一个 Core 级别
- 监控模块的相关配置均在`conf/iotdb-{datanode/confignode}.properties`中，所有配置项支持通过`load configuration`命令热加载。
### 5.1. 使用 JMX 方式
-对于使用 JMX 对外暴露的指标，可以通过 Jconsole 来进行查看。在进入 Jconsole 监控页面后，首先会看到 IoTDB 的各类运行情况的概览。在这里，您可以看到堆内存信息、线程信息、类信息以及服务器的 CPU 使用情况。
+
+对于使用 JMX 对外暴露的指标，可以通过 Jconsole 来进行查看。在进入 Jconsole 监控页面后，首先会看到 IoTDB
+的各类运行情况的概览。在这里，您可以看到堆内存信息、线程信息、类信息以及服务器的 CPU 使用情况。
#### 5.1.1. 获取监控指标数据
+
连接到 JMX 后，您可以通过 "MBeans" 标签找到名为 "org.apache.iotdb.metrics" 的 "MBean"，可以在侧边栏中查看所有监控指标的具体值。
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" alt="metric-jmx" src="https://user-images.githubusercontent.com/46039728/204018765-6fda9391-ebcf-4c80-98c5-26f34bd74df0.png">
#### 5.1.2. 获取其他相关数据
+
连接到 JMX 后，您可以通过 "MBeans" 标签找到名为 "org.apache.iotdb.service" 的 "MBean"，如下图所示，了解服务的基本状态
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/46039728/149951720-707f1ee8-32ee-4fde-9252-048caebd232e.png"> <br>
-为了提高查询性能，IOTDB 对 ChunkMetaData 和 TsFileMetaData 进行了缓存。用户可以使用 MXBean ，展开侧边栏`org.apache.iotdb.db.service`查看缓存命中率：
+为了提高查询性能，IOTDB 对 ChunkMetaData 和 TsFileMetaData 进行了缓存。用户可以使用 MXBean
+，展开侧边栏`org.apache.iotdb.db.service`查看缓存命中率：
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/112426760-73e3da80-8d73-11eb-9a8f-9232d1f2033b.png">
### 5.2. 使用 Prometheus 方式
#### 5.2.1. 监控指标的 Prometheus 映射关系
+
> 对于 Metric Name 为 name, Tags 为 K1=V1, ..., Kn=Vn 的监控指标有如下映射，其中 value 为具体值
-| 监控指标类型 | 映射关系 |
-| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Counter | name_total{k1="V1", ..., Kn="Vn"} value |
+| 监控指标类型 | 映射关系 |
+|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Counter | name_total{k1="V1", ..., Kn="Vn"} value |
| AutoGauge、Gauge | name{k1="V1", ..., Kn="Vn"} value |
-| Histogram | name_max{k1="V1", ..., Kn="Vn"} value <br> name_sum{k1="V1", ..., Kn="Vn"} value <br> name_count{k1="V1", ..., Kn="Vn"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
-| Rate | name_total{k1="V1", ..., Kn="Vn"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m1"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m5"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m15"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="mean"} value |
-| Timer | name_seconds_max{k1="V1", ..., Kn="Vn"} value <br> name_seconds_sum{k1="V1", ..., Kn="Vn"} value <br> name_seconds_count{k1="V1", ..., Kn="Vn"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
+| Histogram | name_max{k1="V1", ..., Kn="Vn"} value <br> name_sum{k1="V1", ..., Kn="Vn"} value <br> name_count{k1="V1", ..., Kn="Vn"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
+| Rate | name_total{k1="V1", ..., Kn="Vn"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m1"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m5"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="m15"} value <br> name_total{k1="V1", ..., Kn="Vn", rate="mean"} value |
+| Timer | name_seconds_max{k1="V1", ..., Kn="Vn"} value <br> name_seconds_sum{k1="V1", ..., Kn="Vn"} value <br> name_seconds_count{k1="V1", ..., Kn="Vn"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.0"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.25"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.5"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="0.75"} value <br> name_seconds{k1="V1", ..., Kn="Vn", quantile="1.0"} value |
#### 5.2.2. 修改配置文件
+
1) 以 DataNode 为例，修改 iotdb-datanode.properties 配置文件如下：
```properties
@@ -414,7 +453,8 @@ file_count{name="seq",} 2.0
#### 5.2.3. Prometheus + Grafana
-如上所示，IoTDB 对外暴露出标准的 Prometheus 格式的监控指标数据，可以使用 Prometheus 采集并存储监控指标，使用 Grafana 可视化监控指标。
+如上所示，IoTDB 对外暴露出标准的 Prometheus 格式的监控指标数据，可以使用 Prometheus 采集并存储监控指标，使用 Grafana
+可视化监控指标。
IoTDB、Prometheus、Grafana三者的关系如下图所示:
@@ -454,6 +494,7 @@ static_configs:
[Grafana从Prometheus查询数据并绘图的文档](https://prometheus.io/docs/visualization/grafana/#grafana-support-for-prometheus)
#### 5.2.4. Apache IoTDB Dashboard
+
我们提供了Apache IoTDB Dashboard，在Grafana中显示的效果图如下所示：
@@ -461,13 +502,16 @@ static_configs:
##### 5.2.4.1. 如何获取 Apache IoTDB Dashboard？
1. 您可以在 GitHub 上获取到Dashboard的json文件。
- 1. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-ConfigNode-Dashboard.json">Apache IoTDB ConfigNode Dashboard</a>
- 2. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-DataNode-Dashboard.json">Apache IoTDB DataNode Dashboard</a>
+ 1. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-ConfigNode-Dashboard.json">
+ Apache IoTDB ConfigNode Dashboard</a>
+ 2. <a href = "https://github.com/apache/iotdb/tree/master/docs/UserGuide/Monitor-Alert/Apache-IoTDB-DataNode-Dashboard.json">
+ Apache IoTDB DataNode Dashboard</a>
2. 您可以访问[Grafana Dashboard官网](https://grafana.com/grafana/dashboards/)搜索`Apache IoTDB Dashboard`并使用
在创建Grafana时，您可以选择Import刚刚下载的json文件，并为Apache IoTDB Dashboard选择对应目标数据源。
##### 5.2.4.2. Apache IoTDB ConfigNode Dashboard 说明
+
> 除特殊说明的监控项以外，以下监控项均保证在Important级别的监控框架中可用
- `Overview`：系统概述
@@ -497,8 +541,9 @@ static_configs:
- `Log Number Per Minute`：IoTDB 进程平均每分钟日志数
- `The Time Consumed of Compliation Per Minute`：平均每分钟编译耗时
- `The Number Of Class`：JVM 加载和卸载的类数量
-
+
##### 5.2.4.3. Apache IoTDB DataNode Dashboard 说明
+
> 除特殊说明的监控项以外，以下监控项均保证在Important级别的监控框架中可用。
- `Overview`：系统概述
@@ -543,8 +588,10 @@ static_configs:
- `The time consumed of construct chunk reader(avg\50%\75%\100%)`：查询构造不同类型Chunk读取器耗时的平均值\中位数\上四分位数\最大值
- `The time consumed of read chunk(avg\50%\75%\100%)`：查询读取不同类型Chunk耗时的平均值\中位数\上四分位数\最大值
- `The time consumed of init chunk reader(avg\50%\75%\100%)`：查询初始化不同类型Chunk读取器耗时的平均值\中位数\上四分位数\最大值
- - `The time consumed of build tsblock from page reader(avg\50%\75%\100%)`：查询从Page Reader构造TsBlock耗时的平均值\中位数\上四分位数\最大值
- - `The time consumed of build tsblock from merge reader(avg\50%\75%\100%)`：查询从Merge Reader构造TsBlock耗时的平均值\中位数\上四分位数\最大值
+ - `The time consumed of build tsblock from page reader(avg\50%\75%\100%)`：查询从Page
+ Reader构造TsBlock耗时的平均值\中位数\上四分位数\最大值
+ - `The time consumed of build tsblock from merge reader(avg\50%\75%\100%)`：查询从Merge
+ Reader构造TsBlock耗时的平均值\中位数\上四分位数\最大值
- `Query Data Exchange`：查询数据传输的具体耗时情况
- `The time consumed of source handle get tsblock(avg\50%\75%\100%)`：查询从不同来源获取TsBlock的耗时的平均值\中位数\上四分位数\最大值
- `The time consumed of source handle deserialize tsblock(avg\50%\75%\100%)`：查询从不同来源反序列化TsBlock的耗时的平均值\中位数\上四分位数\最大值
@@ -586,15 +633,17 @@ static_configs:
### 5.3. 使用 IoTDB 方式
#### 5.3.1. 监控指标的 IoTDB 映射关系
+
> 对于 Metric Name 为 name, Tags 为 K1=V1, ..., Kn=Vn 的监控指标有如下映射，以默认写到 root.__system.metric.`ip:port` 为例
-| 监控指标类型 | 映射关系 [...]
-| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
-| Counter | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
+| 监控指标类型 | 映射关系 [...]
+|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Counter | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
| AutoGauge、Gauge | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.value [...]
-| Histogram | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p75 <br> root.__system.metric.`ip:port [...]
-| Rate | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m1 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m5 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m15 [...]
-| Timer | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:por [...]
+| Histogram | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p75 <br> root.__system.metric.`ip:port` [...]
+| Rate | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m1 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m5 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.m15 [...]
+| Timer | root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.count <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.max <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.mean <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.sum <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p0 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p25 <br> root.__system.metric.`ip:port`.name.`K1=V1`...`Kn=Vn`.p50 <br> root.__system.metric.`ip:port [...]
#### 5.3.2. 获取监控指标
+
根据如上的映射关系，可以构成相关的 IoTDB 查询语句获取监控指标
\ No newline at end of file
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/TsFileMetricManager.java b/server/src/main/java/org/apache/iotdb/db/engine/TsFileMetricManager.java
index d5ccfb78ee..13a205a0ad 100644
--- a/server/src/main/java/org/apache/iotdb/db/engine/TsFileMetricManager.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/TsFileMetricManager.java
@@ -32,6 +32,10 @@ public class TsFileMetricManager {
private final AtomicInteger seqFileNum = new AtomicInteger(0);
private final AtomicInteger unseqFileNum = new AtomicInteger(0);
+ private final AtomicInteger modFileNum = new AtomicInteger(0);
+
+ private final AtomicLong modFileSize = new AtomicLong(0);
+
// compaction temporal files
private final AtomicLong innerSeqCompactionTempFileSize = new AtomicLong(0);
private final AtomicLong innerUnseqCompactionTempFileSize = new AtomicLong(0);
@@ -74,6 +78,30 @@ public class TsFileMetricManager {
return seq ? seqFileNum.get() : unseqFileNum.get();
}
+ public int getModFileNum() {
+ return modFileNum.get();
+ }
+
+ public long getModFileSize() {
+ return modFileSize.get();
+ }
+
+ public void increaseModFileNum(int num) {
+ modFileNum.addAndGet(num);
+ }
+
+ public void decreaseModFileNum(int num) {
+ modFileNum.addAndGet(-num);
+ }
+
+ public void increaseModFileSize(long size) {
+ modFileSize.addAndGet(size);
+ }
+
+ public void decreaseModFileSize(long size) {
+ modFileSize.addAndGet(-size);
+ }
+
public void addCompactionTempFileSize(boolean innerSpace, boolean seq, long delta) {
if (innerSpace) {
long unused =
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/task/CrossSpaceCompactionTask.java b/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/task/CrossSpaceCompactionTask.java
index e1333d15a7..7153f90057 100644
--- a/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/task/CrossSpaceCompactionTask.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/task/CrossSpaceCompactionTask.java
@@ -189,6 +189,22 @@ public class CrossSpaceCompactionTask extends AbstractCompactionTask {
releaseReadAndLockWrite(selectedSequenceFiles);
releaseReadAndLockWrite(selectedUnsequenceFiles);
+ for (TsFileResource sequenceResource : selectedSequenceFiles) {
+ if (sequenceResource.getModFile().exists()) {
+ TsFileMetricManager.getInstance().decreaseModFileNum(1);
+ TsFileMetricManager.getInstance()
+ .decreaseModFileSize(sequenceResource.getModFile().getSize());
+ }
+ }
+
+ for (TsFileResource unsequenceResource : selectedUnsequenceFiles) {
+ if (unsequenceResource.getModFile().exists()) {
+ TsFileMetricManager.getInstance().decreaseModFileNum(1);
+ TsFileMetricManager.getInstance()
+ .decreaseModFileSize(unsequenceResource.getModFile().getSize());
+ }
+ }
+
long sequenceFileSize = deleteOldFiles(selectedSequenceFiles);
long unsequenceFileSize = deleteOldFiles(selectedUnsequenceFiles);
CompactionUtils.deleteCompactionModsFile(selectedSequenceFiles, selectedUnsequenceFiles);
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/utils/CompactionUtils.java b/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/utils/CompactionUtils.java
index 697b4090ff..00bf68b85b 100644
--- a/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/utils/CompactionUtils.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/compaction/execute/utils/CompactionUtils.java
@@ -19,6 +19,7 @@
package org.apache.iotdb.db.engine.compaction.execute.utils;
import org.apache.iotdb.commons.conf.IoTDBConstant;
+import org.apache.iotdb.db.engine.TsFileMetricManager;
import org.apache.iotdb.db.engine.modification.Modification;
import org.apache.iotdb.db.engine.modification.ModificationFile;
import org.apache.iotdb.db.engine.storagegroup.TsFileManager;
@@ -119,6 +120,11 @@ public class CompactionUtils {
new HashSet<>(ModificationFile.getCompactionMods(seqResources.get(i)).getModifications());
modifications.addAll(seqModifications);
updateOneTargetMods(targetResource, modifications);
+ if (modifications.size() > 0) {
+ TsFileMetricManager.getInstance().increaseModFileNum(1);
+ TsFileMetricManager.getInstance()
+ .increaseModFileSize(targetResource.getModFile().getSize());
+ }
modifications.removeAll(seqModifications);
}
}
@@ -137,6 +143,10 @@ public class CompactionUtils {
}
}
updateOneTargetMods(targetTsFile, modifications);
+ if (modifications.size() > 0) {
+ TsFileMetricManager.getInstance().increaseModFileNum(1);
+ TsFileMetricManager.getInstance().increaseModFileSize(targetTsFile.getModFile().getSize());
+ }
}
private static void updateOneTargetMods(
@@ -198,6 +208,9 @@ public class CompactionUtils {
ModificationFile normalModification = ModificationFile.getNormalMods(tsFileResource);
if (normalModification.exists()) {
+ TsFileMetricManager.getInstance().decreaseModFileNum(1);
+ TsFileMetricManager.getInstance()
+ .decreaseModFileSize(tsFileResource.getModFile().getSize());
normalModification.remove();
}
}
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/modification/ModificationFile.java b/server/src/main/java/org/apache/iotdb/db/engine/modification/ModificationFile.java
index 77a8431118..54ddbc4acd 100644
--- a/server/src/main/java/org/apache/iotdb/db/engine/modification/ModificationFile.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/modification/ModificationFile.java
@@ -180,4 +180,13 @@ public class ModificationFile implements AutoCloseable {
return new ModificationFile(
tsFileResource.getTsFilePath() + ModificationFile.COMPACTION_FILE_SUFFIX);
}
+
+ public long getSize() {
+ File file = new File(filePath);
+ if (file.exists()) {
+ return file.length();
+ } else {
+ return 0;
+ }
+ }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/DataRegion.java b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/DataRegion.java
index 278edfd28a..18e66034ee 100644
--- a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/DataRegion.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/DataRegion.java
@@ -460,6 +460,11 @@ public class DataRegion implements IDataRegionForQuery {
for (TsFileResource resource : value) {
if (resource.resourceFileExists()) {
TsFileMetricManager.getInstance().addFile(resource.getTsFile().length(), true);
+ if (resource.getModFile().exists()) {
+ TsFileMetricManager.getInstance().increaseModFileNum(1);
+ TsFileMetricManager.getInstance()
+ .increaseModFileSize(resource.getModFile().getSize());
+ }
}
}
while (!value.isEmpty()) {
@@ -482,6 +487,10 @@ public class DataRegion implements IDataRegionForQuery {
if (resource.resourceFileExists()) {
TsFileMetricManager.getInstance().addFile(resource.getTsFile().length(), false);
}
+ if (resource.getModFile().exists()) {
+ TsFileMetricManager.getInstance().increaseModFileNum(1);
+ TsFileMetricManager.getInstance().increaseModFileSize(resource.getModFile().getSize());
+ }
}
while (!value.isEmpty()) {
TsFileResource tsFileResource = value.get(value.size() - 1);
@@ -1472,6 +1481,15 @@ public class DataRegion implements IDataRegionForQuery {
// normally, mergingModification is just need to be closed by after a merge task is finished.
// we close it here just for IT test.
closeAllResources();
+ List<TsFileResource> tsFileResourceList = tsFileManager.getTsFileList(true);
+ tsFileResourceList.addAll(tsFileManager.getTsFileList(false));
+ tsFileResourceList.forEach(
+ x -> {
+ if (x.getModFile().exists()) {
+ TsFileMetricManager.getInstance().decreaseModFileNum(1);
+ TsFileMetricManager.getInstance().decreaseModFileSize(x.getModFile().getSize());
+ }
+ });
deleteAllSGFolders(DirectoryManager.getInstance().getAllFilesFolders());
this.workSequenceTsFileProcessors.clear();
@@ -2004,9 +2022,16 @@ public class DataRegion implements IDataRegionForQuery {
} else {
deletion.setFileOffset(tsFileResource.getTsFileSize());
// write deletion into modification file
+ boolean modFileExists = tsFileResource.getModFile().exists();
+ long originSize = tsFileResource.getModFile().getSize();
tsFileResource.getModFile().write(deletion);
// remember to close mod file
tsFileResource.getModFile().close();
+ if (!modFileExists) {
+ TsFileMetricManager.getInstance().increaseModFileNum(1);
+ }
+ TsFileMetricManager.getInstance()
+ .increaseModFileSize(tsFileResource.getModFile().getSize() - originSize);
}
logger.info(
"[Deletion] Deletion with path:{}, time:{}-{} written into mods file:{}.",
diff --git a/server/src/main/java/org/apache/iotdb/db/service/metrics/FileMetrics.java b/server/src/main/java/org/apache/iotdb/db/service/metrics/FileMetrics.java
index 53573f7286..8d51fecb60 100644
--- a/server/src/main/java/org/apache/iotdb/db/service/metrics/FileMetrics.java
+++ b/server/src/main/java/org/apache/iotdb/db/service/metrics/FileMetrics.java
@@ -71,6 +71,13 @@ public class FileMetrics implements IMetricSet {
o -> o.getFileSize(false),
Tag.NAME.toString(),
"unseq");
+ metricService.createAutoGauge(
+ Metric.FILE_SIZE.toString(),
+ MetricLevel.IMPORTANT,
+ TS_FILE_METRIC_MANAGER,
+ TsFileMetricManager::getModFileSize,
+ Tag.NAME.toString(),
+ "mods");
metricService.createAutoGauge(
Metric.FILE_COUNT.toString(),
MetricLevel.IMPORTANT,
@@ -85,6 +92,13 @@ public class FileMetrics implements IMetricSet {
o -> o.getFileNum(false),
Tag.NAME.toString(),
"unseq");
+ metricService.createAutoGauge(
+ Metric.FILE_COUNT.toString(),
+ MetricLevel.IMPORTANT,
+ TS_FILE_METRIC_MANAGER,
+ TsFileMetricManager::getModFileNum,
+ Tag.NAME.toString(),
+ "mods");
}
private void bindWalFileMetrics(AbstractMetricService metricService) {
@@ -180,10 +194,14 @@ public class FileMetrics implements IMetricSet {
MetricType.AUTO_GAUGE, Metric.FILE_SIZE.toString(), Tag.NAME.toString(), "seq");
metricService.remove(
MetricType.AUTO_GAUGE, Metric.FILE_SIZE.toString(), Tag.NAME.toString(), "unseq");
+ metricService.remove(
+ MetricType.AUTO_GAUGE, Metric.FILE_SIZE.toString(), Tag.NAME.toString(), "mods");
metricService.remove(
MetricType.AUTO_GAUGE, Metric.FILE_COUNT.toString(), Tag.NAME.toString(), "seq");
metricService.remove(
MetricType.AUTO_GAUGE, Metric.FILE_COUNT.toString(), Tag.NAME.toString(), "unseq");
+ metricService.remove(
+ MetricType.AUTO_GAUGE, Metric.FILE_COUNT.toString(), Tag.NAME.toString(), "mods");
}
private void unbindWalMetrics(AbstractMetricService metricService) {