You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@skywalking.apache.org by wu...@apache.org on 2018/09/12 09:37:32 UTC
[incubator-skywalking] branch docs created (now 06e7130)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch docs
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git.
at 06e7130 Change oal doc.
This branch includes the following new commits:
new 06e7130 Change oal doc.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[incubator-skywalking] 01/01: Change oal doc.
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch docs
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git
commit 06e7130383ecd60bd635bb33d53b79027aef8d9d
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Wed Sep 12 17:37:27 2018 +0800
Change oal doc.
---
docs/en/concepts-and-designs/oal.md | 199 +++------------------
.../{oal.md => scope-definitions.md} | 82 +--------
2 files changed, 28 insertions(+), 253 deletions(-)
diff --git a/docs/en/concepts-and-designs/oal.md b/docs/en/concepts-and-designs/oal.md
index d764d8f..eb88a22 100644
--- a/docs/en/concepts-and-designs/oal.md
+++ b/docs/en/concepts-and-designs/oal.md
@@ -20,171 +20,8 @@ METRIC_NAME = from(SCOPE.(* | [FIELD][,FIELD ...]))
Primary **SCOPE**s are `All`, `Service`, `ServiceInstance`, `Endpoint`, `ServiceRelation`, `ServiceInstanceRelation`, `EndpointRelation`.
Also there are some secondary scopes, which belongs to one primary scope.
-## Field
-By using Aggregation Function, the requests will group by time and **Group Key(s)** in each scope.
-
-- SCOPE `All`
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| endpoint | Represent the endpoint path of each request. | | string |
-| latency | Represent how much time of each request. | | int(in ms) |
-| status | Represent whether success or fail of the request. | | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. e.g. 200, 404, 302| | int |
-
-
-### SCOPE `Service`
-
-Calculate the metric data from each request of the service.
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service | yes | int |
-| name | Represent the name of the service | | string |
-| serviceInstanceName | Represent the name of the service instance id referred | | string |
-| endpointName | Represent the name of the endpoint, such a full path of HTTP URI | | string |
-| latency | Represent how much time of each request. | | int |
-| status | Represent whether success or fail of the request. | | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call | | int|
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-
-### SCOPE `ServiceInstance`
-
-Calculate the metric data from each request of the service instance.
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service instance, usually a number. | yes | int |
-| name | Represent the name of the service instance. Such as `ip:port@Service Name`. **Notice**: current native agent uses `processId@Service name` as instance name, which is useless when you want to setup a filter in aggregation. | | string|
-| serviceName | Represent the name of the service. | | string |
-| endpointName | Represent the name of the endpoint, such a full path of HTTP URI. | | string|
-| latency | Represent how much time of each request. | | int |
-| status | Represent whether success or fail of the request. | | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-
-#### Secondary scopes of `ServiceInstance`
-
-Calculate the metric data if the service instance is a JVM and collected by javaagent.
-
-1. SCOPE `ServiceInstanceJVMCPU`
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service instance, usually a number. | yes | int |
-| name | Represent the name of the service instance. Such as `ip:port@Service Name`. **Notice**: current native agent uses `processId@Service name` as instance name, which is useless when you want to setup a filter in aggregation. | | string|
-| serviceName | Represent the name of the service. | | string |
-| usePercent | Represent how much percent of cpu time cost| | double|
-
-2. SCOPE `ServiceInstanceJVMMemory`
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service instance, usually a number. | yes | int |
-| name | Represent the name of the service instance. Such as `ip:port@Service Name`. **Notice**: current native agent uses `processId@Service name` as instance name, which is useless when you want to setup a filter in aggregation. | | string|
-| serviceName | Represent the name of the service. | | string |
-| isHeap | Represent this value the memory metric values are heap or not | | bool |
-| init | See JVM document | | long |
-| max | See JVM document | | long |
-| used | See JVM document | | long |
-| committed | See JVM document | | long |
-
-3. SCOPE `ServiceInstanceJVMMemoryPool`
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service instance, usually a number. | yes | int |
-| name | Represent the name of the service instance. Such as `ip:port@Service Name`. **Notice**: current native agent uses `processId@Service name` as instance name, which is useless when you want to setup a filter in aggregation. | | string|
-| serviceName | Represent the name of the service. | | string |
-| poolType | Include CODE_CACHE_USAGE, NEWGEN_USAGE, OLDGEN_USAGE, SURVIVOR_USAGE, PERMGEN_USAGE, METASPACE_USAGE based on different version of JVM. | | enum |
-| init | See JVM document | | long |
-| max | See JVM document | | long |
-| used | See JVM document | | long |
-| committed | See JVM document | | long |
-
-4. SCOPE `ServiceInstanceJVMGC`
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the service instance, usually a number. | yes | int |
-| name | Represent the name of the service instance. Such as `ip:port@Service Name`. **Notice**: current native agent uses `processId@Service name` as instance name, which is useless when you want to setup a filter in aggregation. | | string|
-| serviceName | Represent the name of the service. | | string |
-| phrase | Include NEW and OLD | | Enum |
-| time | GC time cost | | long |
-| count | Count of GC op | | long |
-
-### SCOPE `Endpoint`
-
-Calculate the metric data from each request of the endpoint in the service.
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| id | Represent the unique id of the endpoint, usually a number. | yes | int |
-| name | Represent the name of the endpoint, such a full path of HTTP URI. | | string |
-| serviceName | Represent the name of the service. | | string |
-| serviceInstanceName | Represent the name of the service instance id referred. | | string |
-| latency | Represent how much time of each request. | | int |
-| status | Represent whether success or fail of the request.| | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-
-### SCOPE `ServiceRelation`
-
-Calculate the metric data from each request between one service and the other service
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| sourceServiceId | Represent the id of the source service. | yes | int |
-| sourceServiceName | Represent the name of the source service. | | string |
-| sourceServiceInstanceName | Represent the name of the source service instance. | | string |
-| destServiceId | Represent the id of the destination service. | yes | string |
-| destServiceName | Represent the name of the destination service. | | string |
-| destServiceInstanceName | Represent the name of the destination service instance.| | string|
-| endpoint | Represent the endpoint used in this call. | | string
-| latency | Represent how much time of each request. | | int |
-| status | Represent whether success or fail of the request.| | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-| detectPoint | Represent where is the relation detected. Values: client, server, proxy. | yes | enum|
-
-
-### SCOPE `ServiceInstanceRelation`
-
-Calculate the metric data from each request between one service instance and the other service instance
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| sourceServiceInstanceId | Represent the id of the source service instance. | yes | int|
-| sourceServiceName | Represent the name of the source service. | | string |
-| sourceServiceInstanceName | Represent the name of the source service instance. | | string |
-| destServiceName | Represent the name of the destination service. | | |
-| destServiceInstanceId | Represent the id of the destination service instance. | yes | int|
-| destServiceInstanceName | Represent the name of the destination service instance. | | string |
-| endpoint | Represent the endpoint used in this call. | | string
-| latency | Represent how much time of each request. | | int |
-| status | Represent whether success or fail of the request.| | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-| detectPoint | Represent where is the relation detected. Values: client, server, proxy. | yes | enum|
-
-### SCOPE `EndpointRelation`
-
-Calculate the metric data of the dependency between one endpoint and the other endpoint.
-This relation is hard to detect, also depends on tracing lib to propagate the prev endpoint.
-So `EndpointRelation` scope aggregation effects only in service under tracing by SkyWalking native agents,
-including auto instrument agents(like Java, .NET), OpenCensus SkyWalking exporter implementation or others propagate tracing context in SkyWalking spec.
-
-| Name | Remarks | Group Key | Type |
-|---|---|---|---|
-| endpointId | Represent the id of the endpoint as parent in the dependency. | yes | int |
-| endpoint | Represent the endpoint as parent in the dependency.| | string|
-| childEndpointId | Represent the id of the endpoint being used by the parent endpoint in row(1) | yes | int|
-| childEndpoint| Represent the endpoint being used by the parent endpoint in row(2) | | string |
-| rpcLatency | Represent the latency of the RPC from some codes in the endpoint to the childEndpoint. Exclude the latency caused by the endpoint(1) itself.
-| status | Represent whether success or fail of the request.| | bool(true for success) |
-| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
-| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
-| detectPoint | Represent where is the relation detected. Values: client, server, proxy. | yes | enum|
+Read [Scope Definitions](scope-definitions.md), you can find all existing Scopes and Fields.
+
## Filter
Use filter to build the conditions for the value of fields, by using field name and expression.
@@ -197,14 +34,30 @@ The OPs support `=`, `!=`, `>`, `<`, `in (v1, v2, ...`, `like "%..."`, with type
The default functions are provided by SkyWalking OAP core, and could implement more.
Provided functions
-- `avg()`. The average value. The field type must be number.
-- `p99()`. The 99% of the given values should be greater or equal. The field type must be number.
-- `p90()`. The 90% of the given values should be greater or equal. The field type must be number.
-- `p75()`. The 75% of the given values should be greater or equal. The field type must be number.
-- `p50()`. The 75% of the given values should be greater or equal. The field type must be number.
-- `percent()`. The percentage of selected by filter in the whole given data. No type requirement.
-- `histogram(start, step)`. Group the given value by the given step, begin with the start value.
-- `sum()`. The sum number of selected by filter. No type requirement.
+- `longAvg`. The avg of all input per scope entity. The input field must be a long.
+> instance_jvm_memory_max = from(ServiceInstanceJVMMemory.max).longAvg();
+
+In this case, input are request of each ServiceInstanceJVMMemory scope, avg is based on field `max`.
+- `doubleAvg`. The avg of all input per scope entity. The input field must be a double.
+> instance_jvm_cpu = from(ServiceInstanceJVMCPU.usePercent).doubleAvg();
+
+In this case, input are request of each ServiceInstanceJVMCPU scope, avg is based on field `usePercent`.
+- `percent`. The number or ratio expressed as a fraction of 100, for the condition matched input.
+> endpoint_percent = from(Endpoint.*).percent(status == true);
+
+In this case, all input are requests of each endpoint, condition is `endpoint.status == true`.
+- `sum`. The sum calls per scope entity.
+> Service_Calls_Sum = from(Service.*).sum();
+
+In this case, calls of each service.
+- `p99`, `p95`, `p90`, `p75`, `p50`. Read [p99 in WIKI](https://en.wikipedia.org/wiki/Percentile)
+> All_p99 = from(All.latency).p99(10);
+
+In this case, p99 value of all incoming requests.
+- `thermodynamic`. Read [Headmap in WIKI](https://en.wikipedia.org/wiki/Heat_map))
+> All_heatmap = from(All.latency).thermodynamic(100, 20);
+
+In this case, thermodynamic headmap of all incoming requests.
## Metric name
The metric name for storage implementor, alarm and query modules. The type inference supported by core.
diff --git a/docs/en/concepts-and-designs/oal.md b/docs/en/concepts-and-designs/scope-definitions.md
similarity index 74%
copy from docs/en/concepts-and-designs/oal.md
copy to docs/en/concepts-and-designs/scope-definitions.md
index d764d8f..25ab6be 100644
--- a/docs/en/concepts-and-designs/oal.md
+++ b/docs/en/concepts-and-designs/scope-definitions.md
@@ -1,29 +1,7 @@
-# Observability Analysis Language
-Provide OAL(Observability Analysis Language) to analysis incoming data in streaming mode.
-
-OAL focuses on metric in Service, Service Instance and Endpoint. Because of that, the language is easy to
-learn and use.
-
-Considering performance, reading and debugging, OAL is defined as a compile language.
-The OAL scrips will be compiled to normal Java codes in package stage.
-
-## Grammar
-Scripts should be named as `*.oal`
-```
-
-METRIC_NAME = from(SCOPE.(* | [FIELD][,FIELD ...]))
-[.filter(FIELD OP [INT | STRING])]
-.FUNCTION([PARAM][, PARAM ...])
-```
-
-## Scope
-Primary **SCOPE**s are `All`, `Service`, `ServiceInstance`, `Endpoint`, `ServiceRelation`, `ServiceInstanceRelation`, `EndpointRelation`.
-Also there are some secondary scopes, which belongs to one primary scope.
-
-## Field
+# Scopes and Fields
By using Aggregation Function, the requests will group by time and **Group Key(s)** in each scope.
-- SCOPE `All`
+### SCOPE `All`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
@@ -185,59 +163,3 @@ including auto instrument agents(like Java, .NET), OpenCensus SkyWalking exporte
| responseCode | Represent the response code of HTTP response, if this request is the HTTP call. | | int |
| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
| detectPoint | Represent where is the relation detected. Values: client, server, proxy. | yes | enum|
-
-## Filter
-Use filter to build the conditions for the value of fields, by using field name and expression.
-
-The expressions support to link by `and`, `or` and `(...)`.
-The OPs support `=`, `!=`, `>`, `<`, `in (v1, v2, ...`, `like "%..."`, with type detection based of field type. Trigger compile
- or code generation error if incompatible.
-
-## Aggregation Function
-The default functions are provided by SkyWalking OAP core, and could implement more.
-
-Provided functions
-- `avg()`. The average value. The field type must be number.
-- `p99()`. The 99% of the given values should be greater or equal. The field type must be number.
-- `p90()`. The 90% of the given values should be greater or equal. The field type must be number.
-- `p75()`. The 75% of the given values should be greater or equal. The field type must be number.
-- `p50()`. The 75% of the given values should be greater or equal. The field type must be number.
-- `percent()`. The percentage of selected by filter in the whole given data. No type requirement.
-- `histogram(start, step)`. Group the given value by the given step, begin with the start value.
-- `sum()`. The sum number of selected by filter. No type requirement.
-
-## Metric name
-The metric name for storage implementor, alarm and query modules. The type inference supported by core.
-
-## Group
-All metric data will be grouped by Scope.ID and min-level TimeBucket.
-
-- In `Endpoint` scope, the Scope.ID = Endpoint id (the unique id based on service and its Endpoint)
-
-## Examples
-```
-// Caculate p99 of both Endpoint1 and Endpoint2
-Endpoint_p99 = from(Endpoint.latency).filter(name in ("Endpoint1", "Endpoint2")).summary(0.99)
-
-// Caculate p99 of Endpoint name started with `serv`
-serv_Endpoint_p99 = from(Endpoint.latency).filter(name like ("serv%")).summary(0.99)
-
-// Caculate the avg response time of each Endpoint
-Endpoint_avg = from(Endpoint.latency).avg()
-
-// Caculate the histogram of each Endpoint by 50 ms steps.
-// Always thermodynamic diagram in UI matches this metric.
-Endpoint_histogram = from(Endpoint.latency).histogram(50)
-
-// Caculate the percent of response status is true, for each service.
-Endpoint_success = from(Endpoint.*).filter(status = "true").percent()
-
-// Caculate the percent of response code in [200, 299], for each service.
-Endpoint_200 = from(Endpoint.*).filter(responseCode like "2%").percent()
-
-// Caculate the percent of response code in [500, 599], for each service.
-Endpoint_500 = from(Endpoint.*).filter(responseCode like "5%").percent()
-
-// Caculate the sum of calls for each service.
-EndpointCalls = from(Endpoint.*).sum()
-```
\ No newline at end of file