You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@skywalking.apache.org by wu...@apache.org on 2018/03/07 02:55:46 UTC
[incubator-skywalking] branch doc/protocol-en-sync created (now
dee25f7)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch doc/protocol-en-sync
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git.
at dee25f7 Add new document of protocol in en.
This branch includes the following new commits:
new dee25f7 Add new document of protocol in en.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
--
To stop receiving notification emails like this one, please contact
wusheng@apache.org.
[incubator-skywalking] 01/01: Add new document of protocol in en.
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch doc/protocol-en-sync
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git
commit dee25f78c06f8520106190a516928804ebdcae3e
Author: wu-sheng <wu...@foxmail.com>
AuthorDate: Wed Mar 7 10:55:08 2018 +0800
Add new document of protocol in en.
---
docs/README.md | 1 +
docs/cn/Trace-Data-Protocol-CN.md | 7 +-
docs/en/Trace-Data-Protocol.md | 231 ++++++++++++++++++++++++++++++++++++++
3 files changed, 234 insertions(+), 5 deletions(-)
diff --git a/docs/README.md b/docs/README.md
index f7c6bed..3bdfa68 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -21,6 +21,7 @@
* [Plugin development guide](en/Plugin-Development-Guide.md)
* Protocol
* [Cross Process Propagation Headers Protocol, v1.0](en/Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md)
+ * [SkyWalking Trace Data Protocol](en/Trace-Data-Protocol.md)
* [Roadmap](ROADMAP.md)
* Resources provided by community
* [Public speakings](https://github.com/OpenSkywalking/Community#public-speakings)
diff --git a/docs/cn/Trace-Data-Protocol-CN.md b/docs/cn/Trace-Data-Protocol-CN.md
index 32b649a..d9b8ab4 100644
--- a/docs/cn/Trace-Data-Protocol-CN.md
+++ b/docs/cn/Trace-Data-Protocol-CN.md
@@ -40,11 +40,8 @@ JSON数组，数组的每个元素，为一个有效的gRPC服务地址。
### 协议内容
[gRPC service define](../..apm-protocol/apm-network/src/main/proto/ApplicationRegisterService.proto)
-- 首次调用时，applicationCode为客户端设置的应用名（显示在拓扑图和应用列表上的名字）。之后随着追踪过程，会上报此应用相关的周边服务的`ip:port`地址列表
-- KeyWithIntegerValue 返回，key为上报的applicationCode或ip:port地址，value为对应的id。applicationCode对应的返回id,在后续协议中，被称为applicationId。
-- 此服务按需调用，本地无法找到ip:port对应的id时，可异步发起调用。
-- 获取applicationId的操作是必选。后续追踪数据依赖此id
-- 获取ip:port对应的id是可选，使用id，会有效提高collector处理效率，降低网络消耗。
+- applicationCode为客户端设置的应用名.
+- applicationCode对应的返回id,在后续协议中，被称为applicationId。
## 应用实例发现服务
### 简介
diff --git a/docs/en/Trace-Data-Protocol.md b/docs/en/Trace-Data-Protocol.md
new file mode 100644
index 0000000..ee796cb
--- /dev/null
+++ b/docs/en/Trace-Data-Protocol.md
@@ -0,0 +1,231 @@
+# Trace Data Protocol
+Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend.
+
+## Abstract
+This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to
+uplink data to the SkyWalking backend.
+
+- Discovery service provided by HTTP only
+- Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both.
+
+### Version
+v1.1
+
+#### gRPC proto files
+[gRPC proto files](../../apm-protocol/apm-network/src/main/proto)
+
+## Collector discovery service
+### Abstract
+**Collector discovery service should be the first service after agent started**
+Through this service, get the gRPC service list. The agent can choose any one of them for uplink data. Recommend to
+acquire the list periodically.
+
+### HTTP GET
+- Input
+GET：http://collectorIp:port/agent/grpc
+
+- Output
+JSON Array. Each element in the array ia a valid gRPC service address.
+```json
+["ip address1:port1","ip address2:port2","ip address3:port3"]
+```
+
+## Application Register Service
+### Abstract
+Register Application Code to the backend, and receive an integer represents the application.
+
+[gRPC service define](../..apm-protocol/apm-network/src/main/proto/ApplicationRegisterService.proto)
+
+- applicationCode is the config in your `agent.config`.
+- The return id is **ApplicationId** as the value in `KeyWithIntegerValue`, which will be used in further data uplink.
+
+## Discovery Service
+
+### Register Instance Service
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/DiscoveryService.proto#L11-L12)
+
+- agentUUID generated by agent, should be unique. Stay same before reboot, at least.
+- **ApplicationInstanceId** will be used in further data uplink.
+
+### Heart beat service
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/DiscoveryService.proto#L14-L15)
+
+- Recommend to report heart beat every 20-60 seconds.
+- Java agent don't use this, because JVM metrics upstream replace the capabilities of this.
+
+## Service Name Discovery Service
+### Abstract
+Replace the literal String service(operation) name by an id(integer)
+
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/DiscoveryService.proto#L53-L74)
+
+- Optional service, reduce the network cost but use more memory as a buffer mapping.
+
+## Network Address Register Service
+### Abstract
+Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc.
+
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/NetworkAddressRegisterService.proto)
+
+- Optional service, reduce the network cost but use more memory as a buffer mapping.
+
+## JVM Metrics Service
+### Abstract
+Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every second.
+
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/JVMMetricsService.proto)
+
+## Trace Segment Service
+[gRPC service define](../../apm-protocol/apm-network/src/main/proto/TraceSegmentService.proto)
+
+- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
+- Span data please refs to [Plugin Development Guide](Plugin-Development-Guide.md)
+- Id and name both exist, please use id if possible.
+ - operationNameId/operationName
+ - networkAddress/networkAddressId
+ - entryServiceName/entryServiceId
+ - parentServiceName/parentServiceId
+ - peerId/peer
+- componentIds are defined in backend, [here](../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)
+
+# HTTP JSON Services
+All HTTP Services match the gRPC services, just adjust use short keys.
+
+## Instance Register Service
+- http://ip:port/instance/register(default: localhost:12800)
+
+Input:
+```
+{
+ ai: x, #applicationId
+ au: "", #agentUUID
+ rt: x, #registerTime
+ oi: "", #osinfo
+}
+```
+
+Output:
+```
+{
+ ai: x, #applicationId
+ ii: x, #applicationInstanceId
+}
+```
+
+## Heart beat service
+- http://ip:port/instance/heartbeat(default: localhost:12800)
+
+Input:
+```
+{
+ "ii": x, #applicationInstanceId
+ "ht": x #heartbeatTime, java timestamp format
+}
+```
+
+Output: empty
+
+## Service Name Discovery
+- http://ip:port/servicename/discovery(default: localhost:12800)
+
+Input：
+```
+{
+ ai: x, #applicationId
+ sn: "", #serviceName
+ st: x, #srcSpanType
+}
+```
+
+Output：
+```
+{
+ si: x, #osinfo
+ el: { #element
+ ai: x, #applicationId
+ sn: "", #serviceName
+ st: x, #srcSpanType
+ }
+}
+```
+
+## Trace Segment Service
+Input：
+```
+[
+ {
+ "gt": [[230150, 185809, 24040000]],
+ "sg": { //TraceSegmentObject
+ "ts": [137150, 185809, 48780000],
+ "ai": 2, //applicationId
+ "ii": 3, //applicationInstanceId
+ "ss": [ //SpanObject
+ {
+ "si": 0, //spanId
+ "tv": 0, //SpanType
+ "lv": 2, //SpanLayer
+ "ps": -1, //parentSpanId
+ "st": 1501858094726, //startTime
+ "et": 1501858096804, //endTime
+ "ci": 3, //componentId
+ "cn": "", //component
+ "oi": 0, //operationNameId
+ "on": "org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()", //operationName
+ "pi": 0, //peerId
+ "pn": "", //peer
+ "ie": false, //isError
+ "rs": [ //TraceSegmentReference
+ {
+ "pts": [230150, 185809, 24040000], //parentTraceSegmentId
+ "pii": 2, //parentApplicationInstanceId
+ "psp": 1, //parentSpanId
+ "psi": 0, //parentServiceId
+ "psn": "/dubbox-case/case/dubbox-rest", //parentServiceName
+ "ni": 0, //networkAddressId
+ "nn": "172.25.0.4:20880", //networkAddress
+ "eii": 2, //entryApplicationInstanceId
+ "esi": 0, //entryServiceId
+ "esn": "/dubbox-case/case/dubbox-rest", //entryServiceName
+ "rn": 0 //RefType
+ }
+ ],
+ "to": [ //KeyWithStringValue
+ {
+ "k": "url", //key
+ "v": "rest://172.25.0.4:20880/org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()" //value
+ },
+ {
+ "k": "http.method",
+ "v": "GET"
+ }
+ ],
+ "lo": { //LogMessage
+ "t": 1501858094726,
+ "d": [
+ "k": "NullPointException",
+ "v": "Error Stack"
+ }
+ }
+ },
+ {
+ "si": 1,
+ "tv": 1,
+ "lv": 1,
+ "ps": 0,
+ "st": 1501858094726,
+ "et": 1501858095804,
+ "ci": 9,
+ "cn": "",
+ "oi": 0,
+ "on": "mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]",
+ "pi": 0,
+ "pn": "localhost:27017",
+ "ie": false,
+ "to": [],
+ "lo": []
+ }
+ ]
+ }
+ }
+]
+```
--
To stop receiving notification emails like this one, please contact
wusheng@apache.org.