You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2019/02/24 13:07:12 UTC
[incubator-skywalking] branch scope-refactor updated: Provide new
document and `generate-tool-grammer` module.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch scope-refactor
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git
The following commit(s) were added to refs/heads/scope-refactor by this push:
new eb43f43 Provide new document and `generate-tool-grammer` module.
eb43f43 is described below
commit eb43f433cad8091aa823f87694259412af982219
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Sun Feb 24 21:07:03 2019 +0800
Provide new document and `generate-tool-grammer` module.
---
docs/en/guides/How-to-build.md | 2 +-
docs/en/guides/README.md | 7 +-
docs/en/guides/source-extension.md | 84 ++++++++++++++++++++++
docs/others/cn/guides/How-to-build.md | 2 +-
.../pom.xml | 26 ++-----
.../apache/skywalking/oal/tool/grammar/OALLexer.g4 | 0
.../skywalking/oal/tool/grammar/OALParser.g4 | 0
oap-server/generate-tool/pom.xml | 23 ++----
oap-server/pom.xml | 1 +
9 files changed, 102 insertions(+), 43 deletions(-)
diff --git a/docs/en/guides/How-to-build.md b/docs/en/guides/How-to-build.md
index 27f8612..80c8ccb 100644
--- a/docs/en/guides/How-to-build.md
+++ b/docs/en/guides/How-to-build.md
@@ -30,7 +30,7 @@ For each official Apache release, there is a complete and independent source cod
* `grpc-java` and `java` folders in **apm-protocol/apm-network/target/generated-sources/protobuf**
* `grpc-java` and `java` folders in **oap-server/server-core/target/generated-sources/protobuf**
* `grpc-java` and `java` folders in **oap-server/server-receiver-plugin/skywalking-istio-telemetry-receiver-plugin/target/generated-sources/protobuf**
- * `antlr4` folder in **oap-server/generate-tool/target/generated-sources**
+ * `antlr4` folder in **oap-server/generate-tool-grammar/target/generated-sources**
* `oal` folder in **oap-server/generated-analysis/target/generated-sources**
## Setup your Eclipse IDE
diff --git a/docs/en/guides/README.md b/docs/en/guides/README.md
index af3c192..7d406e8 100644
--- a/docs/en/guides/README.md
+++ b/docs/en/guides/README.md
@@ -27,7 +27,7 @@ All the following channels are open to the community, you could choose the way y
As a develop, first step, read [Compiling Guide](How-to-build.md). It teaches developer how to build the project in local.
### Project Extensions
-SkyWalking project supports many ways to extends existing features. If you are interesting in these ways,
+SkyWalking project supports many ways to extend existing features. If you are interesting in these ways,
read the following guides.
- [Java agent plugin development guide](Java-Plugin-Development-Guide.md).
@@ -37,6 +37,11 @@ and private plugin developer should read this.
- [Storage extension development guide](storage-extention.md). Help potential contributors to build a new
storage implementor besides the official.
- [Customize analysis by oal script](write-oal.md). Guide you to use oal script to make your own metric available.
+- [Source and scope extension for new metric](source-extension.md). If you want to analysis a new metric, which SkyWalking
+haven't provide. You need to
+add a new receiver rather than choosing [existed receiver](../setup/backend/backend-receivers.md).
+At that moment,
+you most likely need to add a new source and scope. This document will teach you how to do.
- [Backend Inventory entity extension](inventory-extension.md). If you want to extend SkyWalking inventory entities, and
want to push upstream back to our Apache OSS repo, please read these principles.
diff --git a/docs/en/guides/source-extension.md b/docs/en/guides/source-extension.md
new file mode 100644
index 0000000..c25e2b8
--- /dev/null
+++ b/docs/en/guides/source-extension.md
@@ -0,0 +1,84 @@
+# Source and Scope extension for new metric
+From [OAL scope introduction](../concepts-and-designs/oal.md#scope), you should already have understood what the scope is.
+At here, as you want to do more extension, you need understand deeper, which is the **Source**.
+
+**Source** and **Scope** are binding concepts. **Scope** declare the id(int) and name, **Source** declare the attributes.
+Please follow these steps to create a new Source and Scope.
+
+1. In the OAP core module, it provide **SourceReceiver** internal service.
+```java
+public interface SourceReceiver extends Service {
+ void receive(Source source);
+}
+```
+
+2. All analysis data must be a **org.apache.skywalking.oap.server.core.source.Source**,
+tagged by `@SourceType` annotation,
+so it could be supported by OAL script and OAP core.
+
+Such as existed source, **Service**.
+```java
+@SourceType
+public class Service extends Source {
+ @Override public int scope() {
+ return DefaultScopeDefine.SERVICE;
+ }
+
+ @Override public String getEntityId() {
+ return String.valueOf(id);
+ }
+
+ @Getter @Setter private int id;
+ @Getter @Setter private String name;
+ @Getter @Setter private String serviceInstanceName;
+ @Getter @Setter private String endpointName;
+ @Getter @Setter private int latency;
+ @Getter @Setter private boolean status;
+ @Getter @Setter private int responseCode;
+ @Getter @Setter private RequestType type;
+}
+```
+
+3. The `scope()` method in Source, returns an ID, which is not a random number. This ID need to be declared through
+`@ScopeDeclaration` annotation. This annotation could be used in any class in `org.apache.skywalking` package.
+But just for code style, if this scope is provided in our ASF official repo, we recommend and ask you to add it at
+`org.apache.skywalking.oap.server.core.source.DefaultScopeDefine`, like the existing ones.
+```java
+@ScopeDeclaration(id = ALL, name = "All")
+@ScopeDeclaration(id = SERVICE, name = "Service")
+@ScopeDeclaration(id = SERVICE_INSTANCE, name = "ServiceInstance")
+@ScopeDeclaration(id = ENDPOINT, name = "Endpoint")
+@ScopeDeclaration(id = SERVICE_RELATION, name = "ServiceRelation")
+@ScopeDeclaration(id = SERVICE_INSTANCE_RELATION, name = "ServiceInstanceRelation")
+@ScopeDeclaration(id = ENDPOINT_RELATION, name = "EndpointRelation")
+@ScopeDeclaration(id = NETWORK_ADDRESS, name = "NetworkAddress")
+@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_CPU, name = "ServiceInstanceJVMCPU")
+@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_MEMORY, name = "ServiceInstanceJVMMemory")
+@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_MEMORY_POOL, name = "ServiceInstanceJVMMemoryPool")
+@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_GC, name = "ServiceInstanceJVMGC")
+@ScopeDeclaration(id = SEGMENT, name = "Segment")
+@ScopeDeclaration(id = ALARM, name = "Alarm")
+@ScopeDeclaration(id = SERVICE_INVENTORY, name = "ServiceInventory")
+@ScopeDeclaration(id = SERVICE_INSTANCE_INVENTORY, name = "ServiceInstanceInventory")
+@ScopeDeclaration(id = ENDPOINT_INVENTORY, name = "EndpointInventory")
+@ScopeDeclaration(id = DATABASE_ACCESS, name = "DatabaseAccess")
+@ScopeDeclaration(id = DATABASE_SLOW_STATEMENT, name = "DatabaseSlowStatement")
+public class DefaultScopeDefine {
+ ...
+}
+```
+
+4. The `String getEntityId()` method in Source, request the return value representing unique entity which the scope related.
+Such as,
+in this Service scope, the id is service id, which is used in [OAL group mechanism](../concepts-and-designs/oal.md#group).
+
+5. Add scope name as keyword to oal grammar definition file, `OALLexer.g4`, which is at `antlr4` folder of `generate-tool-grammar` module.
+
+6. Add scope name keyword as source in parser definition file, `OALParser.g4`, which is at same fold of `OALLexer.g4`.
+
+___
+After you done all of these, you could build a receiver, which do
+1. Get the original data of the metric,
+1. Build the source, send into `SourceReceiver`.
+1. Write your whole OAL scripts.
+1. Repackage the project.
\ No newline at end of file
diff --git a/docs/others/cn/guides/How-to-build.md b/docs/others/cn/guides/How-to-build.md
index 65cb6e0..b501e69 100644
--- a/docs/others/cn/guides/How-to-build.md
+++ b/docs/others/cn/guides/How-to-build.md
@@ -30,7 +30,7 @@
* **apm-protocol/apm-network/target/generated-sources/protobuf** 文件夹下的`grpc-java` 和 `java`
* **oap-server/server-core/target/generated-sources/protobuf** 文件夹下的`grpc-java` 和 `java`
* **oap-server/server-receiver-plugin/skywalking-istio-telemetry-receiver-plugin/target/generated-sources/protobuf** 文件夹下的`grpc-java` 和 `java`
- * **oap-server/generate-tool/target/generated-sources** 文件夹下的 `antlr4`
+ * **oap-server/generate-tool-grammar/target/generated-sources** 文件夹下的 `antlr4`
* **oap-server/generated-analysis/target/generated-sources** 文件夹下的 `oal`
## 设置Eclipse IDE
diff --git a/oap-server/generate-tool/pom.xml b/oap-server/generate-tool-grammar/pom.xml
similarity index 64%
copy from oap-server/generate-tool/pom.xml
copy to oap-server/generate-tool-grammar/pom.xml
index e562a41..72a3537 100644
--- a/oap-server/generate-tool/pom.xml
+++ b/oap-server/generate-tool-grammar/pom.xml
@@ -17,7 +17,9 @@
~
-->
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>oap-server</artifactId>
<groupId>org.apache.skywalking</groupId>
@@ -25,34 +27,14 @@
</parent>
<modelVersion>4.0.0</modelVersion>
- <artifactId>generate-tool</artifactId>
+ <artifactId>generate-tool-grammar</artifactId>
<dependencies>
<dependency>
- <groupId>org.apache.skywalking</groupId>
- <artifactId>server-core</artifactId>
- <version>${project.version}</version>
- </dependency>
- <dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4</artifactId>
<version>4.7.1</version>
</dependency>
- <dependency>
- <groupId>commons-cli</groupId>
- <artifactId>commons-cli</artifactId>
- <version>1.4</version>
- </dependency>
- <dependency>
- <groupId>org.projectlombok</groupId>
- <artifactId>lombok</artifactId>
- <version>1.18.0</version>
- </dependency>
- <dependency>
- <groupId>org.freemarker</groupId>
- <artifactId>freemarker</artifactId>
- <version>2.3.28</version>
- </dependency>
</dependencies>
<build>
diff --git a/oap-server/generate-tool/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALLexer.g4 b/oap-server/generate-tool-grammar/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALLexer.g4
similarity index 100%
rename from oap-server/generate-tool/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALLexer.g4
rename to oap-server/generate-tool-grammar/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALLexer.g4
diff --git a/oap-server/generate-tool/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALParser.g4 b/oap-server/generate-tool-grammar/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALParser.g4
similarity index 100%
rename from oap-server/generate-tool/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALParser.g4
rename to oap-server/generate-tool-grammar/src/main/antlr4/org/apache/skywalking/oal/tool/grammar/OALParser.g4
diff --git a/oap-server/generate-tool/pom.xml b/oap-server/generate-tool/pom.xml
index e562a41..538c296 100644
--- a/oap-server/generate-tool/pom.xml
+++ b/oap-server/generate-tool/pom.xml
@@ -34,6 +34,11 @@
<version>${project.version}</version>
</dependency>
<dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>generate-tool-grammar</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4</artifactId>
<version>4.7.1</version>
@@ -54,22 +59,4 @@
<version>2.3.28</version>
</dependency>
</dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.antlr</groupId>
- <artifactId>antlr4-maven-plugin</artifactId>
- <version>4.7.1</version>
- <executions>
- <execution>
- <id>antlr</id>
- <goals>
- <goal>antlr4</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
</project>
\ No newline at end of file
diff --git a/oap-server/pom.xml b/oap-server/pom.xml
index f0869ae..77bc8a4 100644
--- a/oap-server/pom.xml
+++ b/oap-server/pom.xml
@@ -40,6 +40,7 @@
<module>generated-analysis</module>
<module>generate-tool</module>
<module>server-telemetry</module>
+ <module>generate-tool-grammar</module>
</modules>
<properties>