You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@skywalking.apache.org by wu...@apache.org on 2018/09/02 15:12:06 UTC
[incubator-skywalking] branch 6.0 updated: Add many new chapters of
v6 document. Make the document more clear.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch 6.0
in repository https://gitbox.apache.org/repos/asf/incubator-skywalking.git
The following commit(s) were added to refs/heads/6.0 by this push:
new f80b031 Add many new chapters of v6 document. Make the document more clear.
f80b031 is described below
commit f80b03125000a2a7c8feea338fefea88e4722db6
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Sun Sep 2 23:11:56 2018 +0800
Add many new chapters of v6 document. Make the document more clear.
---
.../concepts-and-designs/OAP-Cluster-Management.md | 43 ----------------------
.../Java-Plugin-Development-Guide.md} | 2 +-
docs/en/guides/README.md | 5 ++-
docs/en/guides/storage-extention.md | 3 ++
docs/en/protocols/Trace-Data-Protocol.md | 2 +-
docs/en/setup/backend/backend-alarm.md | 1 +
docs/en/setup/backend/backend-cluster.md | 3 ++
docs/en/setup/backend/backend-oal-scripts.md | 1 +
docs/en/setup/backend/backend-receivers.md | 8 ++++
docs/en/setup/backend/backend-setup.md | 33 ++++++++++++++---
docs/en/setup/backend/backend-storage.md | 42 +++++++++++++++++++++
docs/en/setup/backend/backend-ui-setup.md | 4 +-
docs/en/setup/service-agent/java-agent/README.md | 2 +-
13 files changed, 93 insertions(+), 56 deletions(-)
diff --git a/docs/en/concepts-and-designs/OAP-Cluster-Management.md b/docs/en/concepts-and-designs/OAP-Cluster-Management.md
deleted file mode 100644
index 8a27d54..0000000
--- a/docs/en/concepts-and-designs/OAP-Cluster-Management.md
+++ /dev/null
@@ -1,43 +0,0 @@
-# Observability Analysis Platform Cluster Management
-OAP(Observability Analysis Platform) backend is a distributed system, services need to find each
-other. i.e. a web service needs to find query service, level 1 aggregate service need to find
-level 2 aggregate service. Cluster management is just a client-side implementation. It must work
-together with a distributed coordination service, (i.e. Zookeeper, Consul, Kubernetes.) unless
-running in standalone mode.
-
-## Architecture overview
-<img src="https://skywalkingtest.github.io/page-resources/6.0.0-alpha/cluster_management.png"/>
-
-### Cluster Management Plugins
-By default, OAP backend provides two implementations for cluster management, which are standalone
-and zookeeper. When the scale of services, which are under monitoring, is small, you can choose
-the standalone mode. Otherwise, you must choose the cluster mode by zookeeper plugin, or you can
-implement a new service discovery plugin for your own scene.
-
-### Cluster Management Interface
-There are two interfaces defined beforehand in the OAP server core, which are Module register and
-module query, all the cluster management plugins must implement those two interfaces.
-
-* Module Register: When any modules need to provide services for others, the module must do register
-through this interface.
-* Module Query: When any module needs to find other services, it can use this interface to retrieve
-the service instance list in the certain order.
-
-### Process Flow Between Client and Cluster Management
-The client has two ways to connect the backend, one, use the direct link by a set of backend instance
-endpoint list, or you can use naming service, which considers your given list is just the seed nodes
-of the whole cluster. The following graph is showing you how naming service works. You need to check
-the probe documents to know which way(s) is(are) supported.
-```
- Client lib Collector1 Collector2 Collector3
- (Set collector.servers=Collector2) (Collector 1,2,3 constitute the cluster)
- |
- +-----------> naming service ---------------------------->|
- |
- |<------- receive gRPC IP:Port(s) of Collector 1,2,3---<--|
- |
- |Select a random gRPC service
- |For example collector 3
- |
- |------------------------->Uplink gRPC service----------------------------------->|
-```
\ No newline at end of file
diff --git a/docs/en/setup/service-agent/java-agent/Plugin-Development-Guide.md b/docs/en/guides/Java-Plugin-Development-Guide.md
similarity index 99%
rename from docs/en/setup/service-agent/java-agent/Plugin-Development-Guide.md
rename to docs/en/guides/Java-Plugin-Development-Guide.md
index e493ac6..9d9cf4b 100644
--- a/docs/en/setup/service-agent/java-agent/Plugin-Development-Guide.md
+++ b/docs/en/guides/Java-Plugin-Development-Guide.md
@@ -158,7 +158,7 @@ SpanLayer is the catalog of span. Here are 5 values:
1. MQ
Component IDs are defined and reserved by SkyWalking project.
-For component name/ID extension, please follow [component library setting document](../../backend/Component-library-settings.md).
+For component name/ID extension, please follow [component library setting document](../setup/backend/Component-library-settings.md).
## Develop a plugin
### Abstract
diff --git a/docs/en/guides/README.md b/docs/en/guides/README.md
index b4e954c..b9e882f 100644
--- a/docs/en/guides/README.md
+++ b/docs/en/guides/README.md
@@ -11,6 +11,7 @@ breaking any Apache rule.
SkyWalking project supports many ways to extends existing features. If you are interesting in these ways,
read the following guides.
-- [Java agent plugin development guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md).
+- [Java agent plugin development guide](Java-Plugin-Development-Guide.md).
This guide helps you to develop SkyWalking agent plugin to support more frameworks. Both open source plugin
-and private plugin developer should read this.
\ No newline at end of file
+and private plugin developer should read this.
+- [Storage extension development guide](storage-extention.md)
diff --git a/docs/en/guides/storage-extention.md b/docs/en/guides/storage-extention.md
new file mode 100644
index 0000000..66c6fbf
--- /dev/null
+++ b/docs/en/guides/storage-extention.md
@@ -0,0 +1,3 @@
+# Extend storage
+SkyWalking has already provided H2 and ElasticSearch as storage solutions. In this document, you could
+learn how to implement a new storage easily.
\ No newline at end of file
diff --git a/docs/en/protocols/Trace-Data-Protocol.md b/docs/en/protocols/Trace-Data-Protocol.md
index 54652fb..0222a22 100644
--- a/docs/en/protocols/Trace-Data-Protocol.md
+++ b/docs/en/protocols/Trace-Data-Protocol.md
@@ -18,7 +18,7 @@ v2.0
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/TraceSegmentService.proto)
- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
-- Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md)
+- Span data please refs to [Plugin Development Guide](../guides/Java-Plugin-Development-Guide.md)
- Id and name both exist, please use id if possible.
- operationNameId/operationName
- networkAddress/networkAddressId
diff --git a/docs/en/setup/backend/backend-alarm.md b/docs/en/setup/backend/backend-alarm.md
new file mode 100644
index 0000000..497e256
--- /dev/null
+++ b/docs/en/setup/backend/backend-alarm.md
@@ -0,0 +1 @@
+# Alarm
\ No newline at end of file
diff --git a/docs/en/setup/backend/backend-cluster.md b/docs/en/setup/backend/backend-cluster.md
index cb037dc..cf4cd60 100644
--- a/docs/en/setup/backend/backend-cluster.md
+++ b/docs/en/setup/backend/backend-cluster.md
@@ -1,4 +1,7 @@
# Cluster Management
+In many product environments, backend need to support high throughputs and provide HA to keep robustness,
+so you should need cluster management always in product env.
+
Backend provides several ways to do cluster management. Choose the one you need/want.
- [Zookeeper coordinator](#zookeeper-coordinator). Use Zookeeper to let backend detects and communicates
diff --git a/docs/en/setup/backend/backend-oal-scripts.md b/docs/en/setup/backend/backend-oal-scripts.md
new file mode 100644
index 0000000..5945ee4
--- /dev/null
+++ b/docs/en/setup/backend/backend-oal-scripts.md
@@ -0,0 +1 @@
+# Official OAL scripts
\ No newline at end of file
diff --git a/docs/en/setup/backend/backend-receivers.md b/docs/en/setup/backend/backend-receivers.md
new file mode 100644
index 0000000..056a255
--- /dev/null
+++ b/docs/en/setup/backend/backend-receivers.md
@@ -0,0 +1,8 @@
+# Choose receiver
+Receiver is a concept in SkyWalking backend. All modules, which are responsible for receiving telemetry
+or tracing data from other being monitored system, are all being called **Receiver**. Although today, most of
+receivers are using gRPC or HTTPRestful to provide service, actually, whether listening mode or pull mode
+could be receiver. Such as a receiver could base on pull data from remote, like Kakfa MQ.
+
+We have following receivers
+1.
\ No newline at end of file
diff --git a/docs/en/setup/backend/backend-setup.md b/docs/en/setup/backend/backend-setup.md
index 4318f4e..429cadd 100644
--- a/docs/en/setup/backend/backend-setup.md
+++ b/docs/en/setup/backend/backend-setup.md
@@ -27,17 +27,38 @@ core:
At the same time, modules includes required and optional, the required modules provide the skeleton of backend,
even modulization supported pluggable, remove those modules are meanless. We highly recommend you don't try to
-switch those modules, unless you are well known SkyWalking project and its codes.
+change APIs of those modules, unless you understand SkyWalking project and its codes very well.
List the required modules here
-1. **core**
-1. **cluster**
-1. **storage**
-1. **query**
+1. **Core**. Do basic and major skeleton of all data analysis and stream dispatch.
+1. **Cluster**. Manage multiple backend instances in a cluster, which could provide high throughputs process
+capabilities.
+1. **Storage**. Make the analysis result persistence.
+1. **Query**. Provide query interfaces to UI.
+For **Cluster** and **Storage** have provided multiple implementors(providers), see **Cluster management**
+and **Choose storage** documents in the [link list](#advanced-feature-document-link-list).
+
+Also, several **receiver** modules are provided.
+Receiver is the module in charge of accepting incoming data requests to backend. Most(all) provide
+service by some network(RPC) protocol, such as gRPC, HTTPRestful.
+The receivers have many different module names, you could
+read **Set receivers** document in the [link list](#advanced-feature-document-link-list).
+
+## Advanced feature document link list
After understand the setting file structure, you could choose your interesting feature document.
-We recommend you to read the feature documents in our following list.
+We recommend you to read the feature documents in our following order.
1. [IP and port setting](backend-ip-port.md). Introduce how IP and port set and be used.
1. [Cluster management](backend-cluster.md). Guide you to set backend server in cluster mode.
1. [Deploy in kubernetes](backend-k8s.md). Guide you to build and use SkyWalking image, and deploy in k8s.
+1. [Choose storage](backend-storage.md). As we know, in default quick start, backend is running with H2
+DB. But clearly, it doesn't fit the product env. In here, you could find what other choices do you have.
+Choose the one you like, we are also welcome anyone to contribute new storage implementor,
+1. [Set receivers](backend-receivers.md). You could choose receivers by your requirements, most receivers
+are harmless, at least our default receivers are. You would set and active all receivers provided.
+1. Official [OAL scripts](backend-oal-scripts.md). As you known from our [OAL introduction](../../concepts-and-designs/oal.md),
+most of backend analysis capabilities based on the scripts. Here is the description of official scripts,
+which helps you to understand which metric data are in process, also could be used in alarm.
+1. [Alarm](backend-alarm.md). Alarm provides a time-series based check mechanism. You could set alarm
+rules targeting the analysis oal metric objects.
diff --git a/docs/en/setup/backend/backend-storage.md b/docs/en/setup/backend/backend-storage.md
new file mode 100644
index 0000000..b45e6a4
--- /dev/null
+++ b/docs/en/setup/backend/backend-storage.md
@@ -0,0 +1,42 @@
+# Backend storage
+SkyWalking storage is pluggable, we have provided the following storage solutions, you could easily
+use is by changing the `application.yml`
+
+- [**H2**](#h2)
+- [**ElasticSearch 6**](#elasticsearch-6)
+
+## H2
+Active ElasticSearch 6 as storage, set storage provider to **H2**.
+
+Setting fragment example
+```yaml
+
+```
+
+## ElasticSearch 6
+Active ElasticSearch 6 as storage, set storage provider to **elasticsearch**.
+
+Setting fragment example
+```yaml
+storage:
+ elasticsearch:
+ clusterNodes: localhost:9200
+ indexShardsNumber: 2
+ indexReplicasNumber: 0
+ # Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html
+ bulkActions: 2000 # Execute the bulk every 2000 requests
+ bulkSize: 20 # flush the bulk every 20mb
+ flushInterval: 10 # flush the bulk every 10 seconds whatever the number of requests
+ concurrentRequests: 2 # the number of concurrent requests
+ # Set a timeout on metric data. After the timeout has expired, the metric data will automatically be deleted.
+ traceDataTTL: 90 # Unit is minute
+ minuteMetricDataTTL: 90 # Unit is minute
+ hourMetricDataTTL: 36 # Unit is hour
+ dayMetricDataTTL: 45 # Unit is day
+ monthMetricDataTTL: 18 # Unit is month
+```
+
+
+## More storage solution extension
+Follow [Storage extension development guide](../../guides/storage-extention.md)
+in [Project Extensions document](../../guides/README.md#project-extensions) in development guide.
\ No newline at end of file
diff --git a/docs/en/setup/backend/backend-ui-setup.md b/docs/en/setup/backend/backend-ui-setup.md
index 081a1d9..cd04b81 100644
--- a/docs/en/setup/backend/backend-ui-setup.md
+++ b/docs/en/setup/backend/backend-ui-setup.md
@@ -35,7 +35,7 @@ data to backend. Also, jetty service supported in JSON format.
- UI uses GraphQL(HTTP) query to access backend also in Jetty service(`core/default/rest*` in `application.yml`).
Now, let's go the backend and UI setting documents.
-- [Backend setting document](backend-setup.md)
-- [UI setting document](ui-setup.md)
+- [Backend setup document](backend-setup.md)
+- [UI setup document](ui-setup.md)
\ No newline at end of file
diff --git a/docs/en/setup/service-agent/java-agent/README.md b/docs/en/setup/service-agent/java-agent/README.md
index dc048e7..3d8641d 100644
--- a/docs/en/setup/service-agent/java-agent/README.md
+++ b/docs/en/setup/service-agent/java-agent/README.md
@@ -70,7 +70,7 @@ Now, we have the following known plugins.
## Plugin Development Guide
SkyWalking java agent supports plugin to extend [the supported list](Supported-list.md). Please follow
-our [Plugin Development Guide](Plugin-Development-Guide.md).
+our [Plugin Development Guide](../../../guides/Java-Plugin-Development-Guide.md).
# Test
If you are interested in plugin compatible tests or agent performance, see the following reports.