You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2021/06/02 07:32:30 UTC
[skywalking] branch master updated: Refine backend (#2) (#7056)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new da75a7f Refine backend (#2) (#7056)
da75a7f is described below
commit da75a7f23820ef57f5092056fd40e5f727b538af
Author: Wing <69...@users.noreply.github.com>
AuthorDate: Wed Jun 2 15:32:11 2021 +0800
Refine backend (#2) (#7056)
---
.../backend/backend-infrastructure-monitoring.md | 102 ++++++++++-----------
1 file changed, 51 insertions(+), 51 deletions(-)
diff --git a/docs/en/setup/backend/backend-infrastructure-monitoring.md b/docs/en/setup/backend/backend-infrastructure-monitoring.md
index 9b19e36..53163aa 100644
--- a/docs/en/setup/backend/backend-infrastructure-monitoring.md
+++ b/docs/en/setup/backend/backend-infrastructure-monitoring.md
@@ -1,11 +1,11 @@
# VMs monitoring
SkyWalking leverages Prometheus node-exporter for collecting metrics data from the VMs, and leverages OpenTelemetry Collector to transfer the metrics to
[OpenTelemetry receiver](backend-receivers.md#opentelemetry-receiver) and into the [Meter System](./../../concepts-and-designs/meter.md).
-We defined the VM entity as a `Service` in OAP, use `vm::` as a prefix to identify.
+We define the VM entity as a `Service` in OAP, and use `vm::` as a prefix to identify it.
## Data flow
-1. Prometheus node-exporter collects metrics data from the VMs.
-2. OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via the OpenCensus GRPC Exporter.
+1. The Prometheus node-exporter collects metrics data from the VMs.
+2. The OpenTelemetry Collector fetches metrics from the node-exporter via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via the OpenCensus gRPC Exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
@@ -17,59 +17,59 @@ We defined the VM entity as a `Service` in OAP, use `vm::` as a prefix to identi
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|-----|-----|-----|-----|-----|
-| CPU Usage | % | cpu_total_percentage | The CPU cores total used percentage, if there are 2 cores the max usage is 200% | Prometheus node-exporter |
-| Memory RAM Usage | MB | meter_vm_memory_used | The RAM total usage | Prometheus node-exporter |
-| Memory Swap Usage | % | meter_vm_memory_swap_percentage | The swap memory used percentage | Prometheus node-exporter |
-| CPU Average Used | % | meter_vm_cpu_average_used | The CPU cores used percentage in each mode | Prometheus node-exporter |
+| CPU Usage | % | cpu_total_percentage | The total percentage usage of the CPU core. If there are 2 cores, the maximum usage is 200%. | Prometheus node-exporter |
+| Memory RAM Usage | MB | meter_vm_memory_used | The total RAM usage | Prometheus node-exporter |
+| Memory Swap Usage | % | meter_vm_memory_swap_percentage | The percentage usage of swap memory | Prometheus node-exporter |
+| CPU Average Used | % | meter_vm_cpu_average_used | The percentage usage of the CPU core in each mode | Prometheus node-exporter |
| CPU Load | | meter_vm_cpu_load1<br />meter_vm_cpu_load5<br />meter_vm_cpu_load15 | The CPU 1m / 5m / 15m average load | Prometheus node-exporter |
-| Memory RAM | MB | meter_vm_memory_total<br />meter_vm_memory_available<br />meter_vm_memory_used | The RAM statistics, include Total / Available / Used | Prometheus node-exporter |
-| Memory Swap | MB | meter_vm_memory_swap_free<br />meter_vm_memory_swap_total | The Swap Memory statistics, include Free / Total | Prometheus node-exporter |
-| File System Mountpoint Usage | % | meter_vm_filesystem_percentage | The File System used percentage in each mount point | Prometheus node-exporter |
-| Disk R/W | KB/s | meter_vm_disk_read,meter_vm_disk_written | The Disk read and written | Prometheus node-exporter |
-| Network Bandwidth Usage | KB/s | meter_vm_network_receive<br />meter_vm_network_transmit | The Network receive and transmit | Prometheus node-exporter |
-| Network Status | | meter_vm_tcp_curr_estab<br />meter_vm_tcp_tw<br />meter_vm_tcp_alloc<br />meter_vm_sockets_used<br />meter_vm_udp_inuse | The number of the TCP establish / TCP time wait / TCP allocated / Sockets inuse / UDP inuse | Prometheus node-exporter |
-| Filefd Allocated | | meter_vm_filefd_allocated | The number of the File Descriptor allocated | Prometheus node-exporter |
+| Memory RAM | MB | meter_vm_memory_total<br />meter_vm_memory_available<br />meter_vm_memory_used | The RAM statistics, including Total / Available / Used | Prometheus node-exporter |
+| Memory Swap | MB | meter_vm_memory_swap_free<br />meter_vm_memory_swap_total | The swap memory statistics, including Free / Total | Prometheus node-exporter |
+| File System Mountpoint Usage | % | meter_vm_filesystem_percentage | The percentage usage of the file system at each mount point | Prometheus node-exporter |
+| Disk R/W | KB/s | meter_vm_disk_read,meter_vm_disk_written | The disk read and written | Prometheus node-exporter |
+| Network Bandwidth Usage | KB/s | meter_vm_network_receive<br />meter_vm_network_transmit | The network receive and transmit | Prometheus node-exporter |
+| Network Status | | meter_vm_tcp_curr_estab<br />meter_vm_tcp_tw<br />meter_vm_tcp_alloc<br />meter_vm_sockets_used<br />meter_vm_udp_inuse | The number of TCPs established / TCP time wait / TCPs allocated / sockets in use / UDPs in use | Prometheus node-exporter |
+| Filefd Allocated | | meter_vm_filefd_allocated | The number of file descriptors allocated | Prometheus node-exporter |
## Customizing
You can customize your own metrics/expression/dashboard panel.
-The metrics definition and expression rules are in `/config/otel-oc-rules/vm.yaml`.
-The dashboard panel confirmations are in `/config/ui-initialized-templates/vm.yml`.
+The metrics definition and expression rules are found in `/config/otel-oc-rules/vm.yaml`.
+The dashboard panel confirmations are found in `/config/ui-initialized-templates/vm.yml`.
## Blog
-A related blog can see: [SkyWalking 8.4 provides infrastructure monitoring](https://skywalking.apache.org/blog/2021-02-07-infrastructure-monitoring/)
+For more details, see blog article [SkyWalking 8.4 provides infrastructure monitoring](https://skywalking.apache.org/blog/2021-02-07-infrastructure-monitoring/).
# K8s monitoring
-SkyWalking leverages K8s kube-state-metrics and cAdvisor for collecting metrics data from the K8s, and leverages OpenTelemetry Collector to transfer the metrics to
+SkyWalking leverages K8s kube-state-metrics and cAdvisor for collecting metrics data from K8s, and leverages OpenTelemetry Collector to transfer the metrics to
[OpenTelemetry receiver](backend-receivers.md#opentelemetry-receiver) and into the [Meter System](./../../concepts-and-designs/meter.md). This feature requires authorizing the OAP Server to access K8s's `API Server`.
-We defined the k8s-cluster as a `Service` in OAP, use `k8s-cluster::` as a prefix to identify.
-Defined the k8s-node as an `Instance` in OAP, the name is k8s `node name`.
-Defined the k8s-service as an `Endpoint` in OAP, the name is `$serviceName.$namespace`.
+We define the k8s-cluster as a `Service` in the OAP, and use `k8s-cluster::` as a prefix to identify it.
+We define the k8s-node as an `Instance` in the OAP, and set its name as the K8s `node name`.
+We define the k8s-service as an `Endpoint` in the OAP, and set its name as `$serviceName.$namespace`.
## Data flow
-1. K8s kube-state-metrics and cAdvisor collects metrics data from the K8s.
+1. K8s kube-state-metrics and cAdvisor collect metrics data from K8s.
2. OpenTelemetry Collector fetches metrics from kube-state-metrics and cAdvisor via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via the OpenCensus GRPC Exporter.
3. The SkyWalking OAP Server access to K8s's `API Server` gets meta info and parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
1. Setup [kube-state-metric](https://github.com/kubernetes/kube-state-metrics#kubernetes-deployment).
2. cAdvisor is integrated into `kubelet` by default.
-3. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/getting-started/#kubernetes). Prometheus Receiver in OpenTelemetry Collector for K8s can reference [here](https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus-kubernetes.yml). For a quick start, we provided a full example for OpenTelemetry Collector configuration [otel-collector-config.yaml](otel-collector-config.yaml).
+3. Set up [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/getting-started/#kubernetes). For details on Prometheus Receiver in OpenTelemetry Collector for K8s, refer to [here](https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus-kubernetes.yml). For a quick start, we have provided a full example for OpenTelemetry Collector configuration [otel-collector-config.yaml](otel-collector-config.yaml).
4. Config SkyWalking [OpenTelemetry receiver](backend-receivers.md#opentelemetry-receiver).
## Supported Metrics
-From the different point of view to monitor the K8s, there are 3 kinds of metrics: [Cluster](#cluster) / [Node](#node) / [Service](#service)
+From the different points of view to monitor K8s, there are 3 kinds of metrics: [Cluster](#cluster) / [Node](#node) / [Service](#service)
### Cluster
-These metrics are related to the selected cluster(`Current Service in the dashboard`).
+These metrics are related to the selected cluster (`Current Service in the dashboard`).
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|-----|-----|-----|-----|-----|
-| Node Total | | k8s_cluster_node_total | The number of the nodes | K8s kube-state-metrics|
-| Namespace Total | | k8s_cluster_namespace_total | The number of the namespaces | K8s kube-state-metrics|
-| Deployment Total | | k8s_cluster_deployment_total | The number of the deployments | K8s kube-state-metrics|
-| Service Total | | k8s_cluster_service_total | The number of the services | K8s kube-state-metrics|
-| Pod Total | | k8s_cluster_pod_total | The number of the pods | K8s kube-state-metrics|
-| Container Total | | k8s_cluster_container_total | The number of the containers | K8s kube-state-metrics|
+| Node Total | | k8s_cluster_node_total | The number of nodes | K8s kube-state-metrics|
+| Namespace Total | | k8s_cluster_namespace_total | The number of namespaces | K8s kube-state-metrics|
+| Deployment Total | | k8s_cluster_deployment_total | The number of deployments | K8s kube-state-metrics|
+| Service Total | | k8s_cluster_service_total | The number of services | K8s kube-state-metrics|
+| Pod Total | | k8s_cluster_pod_total | The number of pods | K8s kube-state-metrics|
+| Container Total | | k8s_cluster_container_total | The number of containers | K8s kube-state-metrics|
| CPU Resources | m | k8s_cluster_cpu_cores<br />k8s_cluster_cpu_cores_requests<br />k8s_cluster_cpu_cores_limits<br />k8s_cluster_cpu_cores_allocatable | The capacity and the Requests / Limits / Allocatable of the CPU | K8s kube-state-metrics|
| Memory Resources | GB | k8s_cluster_memory_total<br />k8s_cluster_memory_requests<br />k8s_cluster_memory_limits<br />k8s_cluster_memory_allocatable | The capacity and the Requests / Limits / Allocatable of the memory | K8s kube-state-metrics|
| Storage Resources | GB | k8s_cluster_storage_total<br />k8s_cluster_storage_allocatable | The capacity and allocatable of the storage | K8s kube-state-metrics|
@@ -77,43 +77,43 @@ These metrics are related to the selected cluster(`Current Service in the dashbo
| Deployment Status | | k8s_cluster_deployment_status | The current status of the deployment | K8s kube-state-metrics|
| Deployment Spec Replicas | | k8s_cluster_deployment_spec_replicas | The number of desired pods for a deployment | K8s kube-state-metrics|
| Service Status | | k8s_cluster_service_pod_status | The services current status, depending on the related pods' status | K8s kube-state-metrics|
-| Pod Status Not Running | | k8s_cluster_pod_status_not_running | The pods which the current phase is not running | K8s kube-state-metrics|
-| Pod Status Waiting | | k8s_cluster_pod_status_waiting | The pods and containers which currently in the waiting status, and show the reason | K8s kube-state-metrics|
-| Pod Status Terminated | | k8s_cluster_container_status_terminated | The pods and containers which currently in the terminated status, and show the reason | K8s kube-state-metrics|
+| Pod Status Not Running | | k8s_cluster_pod_status_not_running | The pods which are not running in the current phase | K8s kube-state-metrics|
+| Pod Status Waiting | | k8s_cluster_pod_status_waiting | The pods and containers which are currently in the waiting status, with reasons shown | K8s kube-state-metrics|
+| Pod Status Terminated | | k8s_cluster_container_status_terminated | The pods and containers which are currently in the terminated status, with reasons shown | K8s kube-state-metrics|
### Node
These metrics are related to the selected node (`Current Instance in the dashboard`).
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|-----|-----|-----|-----|-----|
-| Pod Total | | k8s_node_pod_total | The number of the pods which in this node | K8s kube-state-metrics |
+| Pod Total | | k8s_node_pod_total | The number of pods in this node | K8s kube-state-metrics |
| Node Status | | k8s_node_node_status | The current status of this node | K8s kube-state-metrics |
-| CPU Resources | m | k8s_node_cpu_cores<br />k8s_node_cpu_cores_allocatable<br />k8s_node_cpu_cores_requests<br />k8s_node_cpu_cores_limits | The capacity and the Requests / Limits / Allocatable of the CPU | K8s kube-state-metrics |
-| Memory Resources | GB | k8s_node_memory_total<br />k8s_node_memory_allocatable<br />k8s_node_memory_requests<br />k8s_node_memory_limits | The capacity and the Requests / Limits / Allocatable of the memory | K8s kube-state-metrics |
+| CPU Resources | m | k8s_node_cpu_cores<br />k8s_node_cpu_cores_allocatable<br />k8s_node_cpu_cores_requests<br />k8s_node_cpu_cores_limits | The capacity and the requests / Limits / Allocatable of the CPU | K8s kube-state-metrics |
+| Memory Resources | GB | k8s_node_memory_total<br />k8s_node_memory_allocatable<br />k8s_node_memory_requests<br />k8s_node_memory_limits | The capacity and the requests / Limits / Allocatable of the memory | K8s kube-state-metrics |
| Storage Resources | GB | k8s_node_storage_total<br />k8s_node_storage_allocatable | The capacity and allocatable of the storage | K8s kube-state-metrics |
-| CPU Usage | m | k8s_node_cpu_usage | The CPU cores total usage, if there are 2 cores the max usage is 2000m | cAdvisor |
-| Memory Usage | GB | k8s_node_memory_usage | The memory total usage | cAdvisor |
-| Network I/O| KB/s | k8s_node_network_receive<br />k8s_node_network_transmit | The Network receive and transmit | cAdvisor |
+| CPU Usage | m | k8s_node_cpu_usage | The total usage of the CPU core, if there are 2 cores the maximum usage is 2000m | cAdvisor |
+| Memory Usage | GB | k8s_node_memory_usage | The totaly memory usage | cAdvisor |
+| Network I/O| KB/s | k8s_node_network_receive<br />k8s_node_network_transmit | The network receive and transmit | cAdvisor |
### Service
In these metrics, the pods are related to the selected service (`Current Endpoint in the dashboard`).
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|-----|-----|-----|-----|-----|
-| Service Pod Total | | k8s_service_pod_total | The number of the pods | K8s kube-state-metrics |
+| Service Pod Total | | k8s_service_pod_total | The number of pods | K8s kube-state-metrics |
| Service Pod Status | | k8s_service_pod_status | The current status of pods | K8s kube-state-metrics |
-| Service CPU Resources | m | k8s_service_cpu_cores_requests<br />k8s_service_cpu_cores_limits | The CPU resources Requests / Limits of this service | K8s kube-state-metrics |
-| Service Memory Resources | MB | k8s_service_memory_requests<br />k8s_service_memory_limits | The memory resources Requests / Limits of this service | K8s kube-state-metrics |
+| Service CPU Resources | m | k8s_service_cpu_cores_requests<br />k8s_service_cpu_cores_limits | The CPU resources requests / Limits of this service | K8s kube-state-metrics |
+| Service Memory Resources | MB | k8s_service_memory_requests<br />k8s_service_memory_limits | The memory resources requests / Limits of this service | K8s kube-state-metrics |
| Pod CPU Usage | m | k8s_service_pod_cpu_usage | The CPU resources total usage of pods | cAdvisor |
| Pod Memory Usage | MB | k8s_service_pod_memory_usage | The memory resources total usage of pods | cAdvisor |
-| Pod Waiting | | k8s_service_pod_status_waiting | The pods and containers which currently in the waiting status, and show the reason | K8s kube-state-metrics |
-| Pod Terminated | | k8s_service_pod_status_terminated | The pods and containers which currently in the terminated status, and show the reason | K8s kube-state-metrics |
-| Pod Restarts | | k8s_service_pod_status_restarts_total | The number of per container restarts that related to the pod | K8s kube-state-metrics |
-| Pod Network Receive | KB/s | k8s_service_pod_network_receive | The Network receive of the pods | cAdvisor |
-| Pod Network Transmit | KB/s | k8s_service_pod_network_transmit | The Network transmit of the pods | cAdvisor |
-| Pod Storage Usage | MB | k8s_service_pod_fs_usage | The storage resources total usage of pods which related to this service | cAdvisor |
+| Pod Waiting | | k8s_service_pod_status_waiting | The pods and containers which are currently in the waiting status, with reasons shown | K8s kube-state-metrics |
+| Pod Terminated | | k8s_service_pod_status_terminated | The pods and containers which are currently in the terminated status, with reasons shown | K8s kube-state-metrics |
+| Pod Restarts | | k8s_service_pod_status_restarts_total | The number of per container restarts related to the pods | K8s kube-state-metrics |
+| Pod Network Receive | KB/s | k8s_service_pod_network_receive | The network receive of the pods | cAdvisor |
+| Pod Network Transmit | KB/s | k8s_service_pod_network_transmit | The network transmit of the pods | cAdvisor |
+| Pod Storage Usage | MB | k8s_service_pod_fs_usage | The storage resources total usage of pods related to this service | cAdvisor |
## Customizing
You can customize your own metrics/expression/dashboard panel.
-The metrics definition and expression rules are in `/config/otel-oc-rules/k8s-cluster.yaml，/config/otel-oc-rules/k8s-node.yaml, /config/otel-oc-rules/k8s-service.yaml`.
-The dashboard panel configurations are in `/config/ui-initialized-templates/k8s.yml`.
\ No newline at end of file
+The metrics definition and expression rules are found in `/config/otel-oc-rules/k8s-cluster.yaml，/config/otel-oc-rules/k8s-node.yaml, /config/otel-oc-rules/k8s-service.yaml`.
+The dashboard panel configurations are found in `/config/ui-initialized-templates/k8s.yml`.