You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2022/08/10 13:45:02 UTC
[skywalking] branch doc created (now f50c9b2fcb)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch doc
in repository https://gitbox.apache.org/repos/asf/skywalking.git
at f50c9b2fcb Polish docs for no sharding mode.
This branch includes the following new commits:
new f50c9b2fcb Polish docs for no sharding mode.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[skywalking] 01/01: Polish docs for no sharding mode.
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch doc
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit f50c9b2fcb36549224ee373b31a3b9f5410ab856
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Wed Aug 10 21:44:49 2022 +0800
Polish docs for no sharding mode.
---
...icSearch-storage-option-explanation-in-9.2.0.md | 32 ++++++++++++----------
docs/en/setup/backend/backend-storage.md | 19 +++++++++----
2 files changed, 31 insertions(+), 20 deletions(-)
diff --git a/docs/en/FAQ/New-ElasticSearch-storage-option-explanation-in-9.2.0.md b/docs/en/FAQ/New-ElasticSearch-storage-option-explanation-in-9.2.0.md
index 11a2f7b91b..bf9095be34 100644
--- a/docs/en/FAQ/New-ElasticSearch-storage-option-explanation-in-9.2.0.md
+++ b/docs/en/FAQ/New-ElasticSearch-storage-option-explanation-in-9.2.0.md
@@ -1,19 +1,23 @@
## New ElasticSearch storage option explanation in 9.2.0
-Since v9.2.0, SkyWalking OAP provide 2 storage options for metrics/meter and records,
-system environment variable is (`SW_STORAGE_ES_LOGIC_SHARDING`):
+Since v9.2.0, SkyWalking OAP provides 2 storage options for all data, including metadata, metrics, traces, logs, events, profiling data, etc.. OAP exposes a system environment variable (`SW_STORAGE_ES_LOGIC_SHARDING`) to control the running mode.
-### No-Sharding Model (OAP default setting, `SW_STORAGE_ES_LOGIC_SHARDING = false`)
-1. OAP merge all metrics/meter and records(without super datasets, such as segments) indices into one physical
-index template `metrics-all` and `records-all`.
-2. The logic index name would present in column `metric_table` and `record_table`.
-3. If logic column name has alias (configured by `@ElasticSearch.Column()`), the alias would be the real physical column name.
+### No-Sharding Mode (OAP default setting, `SW_STORAGE_ES_LOGIC_SHARDING = false`)
+This is the new mode introduced in 9.2.0. It prefers to keep data with similar properties in one index template, such as all metrics and metadata.
-### No-Sharding Model (`SW_STORAGE_ES_LOGIC_SHARDING = true `)
-1. OAP shard metrics/meter indices into multi-physical indices as the previous versions(one index template per metric/meter aggregation function).
-2. Records and metrics without configure aggregation function in `@MetricsFunction` and `@MeterFunction` would not be sharded.
+1. OAP merges all metrics/meter and records(without super datasets, such as segments) indices into one physical
+ index template `metrics-all` and `records-all`.
+2. The logic index name would be present in columns `metric_table` or `record_table`.
+3. If the logic column name has an alias (configured through `@ElasticSearch.Column()`), the alias would be the real physical column name.
+
+The super dataset would not be affected by this, such as traces and logs.
+
+### Sharding Mode (`SW_STORAGE_ES_LOGIC_SHARDING = true `)
+1. OAP shard metrics/meter indices into multi-physical indices as in the previous versions(one index template per metric/meter aggregation function).
+2. Records and metrics without configuring aggregation functions with `@MetricsFunction` or `@MeterFunction` annotation would not be merged. They would be kept in a separate index template.
3. The shard template name would be `metrics-aggregation function name` or `meter-aggregation function name` such as `metrics-count`,
-and the logic index name would present in column `metric_table`.
-4. The OAP **would not** use the column alias, the logic column name would be the real physical column name.
+ and the logic index name would be present in column `metric_table`.
+5. The OAP **would not** use the column alias, the logic column name would be the real physical column name.
-**Notice**:
-Users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out no matter the option is.
+___
+**Notice**:
+Users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out in either mode.
diff --git a/docs/en/setup/backend/backend-storage.md b/docs/en/setup/backend/backend-storage.md
index 02fba2ef18..5d09ed0822 100644
--- a/docs/en/setup/backend/backend-storage.md
+++ b/docs/en/setup/backend/backend-storage.md
@@ -36,6 +36,7 @@ storage:
## OpenSearch
+OpenSearch is a fork from ElasticSearch 7.11 but licensed in Apache 2.0.
OpenSearch storage shares the same configurations as ElasticSearch.
In order to activate OpenSearch as storage, set the storage provider to **elasticsearch**.
@@ -44,13 +45,10 @@ In order to activate OpenSearch as storage, set the storage provider to **elasti
**NOTE:** Elastic announced through their blog that Elasticsearch will be moving over to a Server Side Public
License (SSPL), which is incompatible with Apache License 2.0. This license change is effective from Elasticsearch
version 7.11. So please choose the suitable ElasticSearch version according to your usage.
+If you have concerns about SSPL, choose the versions before 7.11 or switch to OpenSearch.
-Since 8.8.0, SkyWalking rebuilds the ElasticSearch client on top of ElasticSearch REST API and automatically picks up
-correct request formats according to the server-side version, hence you don't need to download different binaries
-and don't need to configure different storage selectors for different ElasticSearch server-side versions anymore.
-
-Since 9.2.0, SkyWalking merges all metrics/meter and records(without super datasets) indices into one physical index template `metrics-all` and `records-all` on the default setting.
-Provide system environment variable(`SW_STORAGE_ES_LOGIC_SHARDING`) to shard metrics indices into multi-physical indices as the previous versions(one index template per metric/meter aggregation function).
+Since 9.2.0, SkyWalking provides no-sharding/one-index mode to merge all metrics/meter and records(without super datasets)
+indices into one physical index template `metrics-all` and `records-all` on the default setting.
In the current one index mode, users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out.
After merge all indices, the following indices are available:
@@ -62,6 +60,15 @@ After merge all indices, the following indices are available:
* sw_zipkin_span-`${day-format}`
* sw_records-all-`${day-format}`
+___
+Provide system environment variable(`SW_STORAGE_ES_LOGIC_SHARDING`). Set it to `true` could shard metrics indices into multi-physical indices
+as same as the versions(one index template per metric/meter aggregation function) before 9.2.0.
+___
+
+Since 8.8.0, SkyWalking rebuilds the ElasticSearch client on top of ElasticSearch REST API and automatically picks up
+correct request formats according to the server-side version, hence you don't need to download different binaries
+and don't need to configure different storage selectors for different ElasticSearch server-side versions anymore.
+
For now, SkyWalking supports ElasticSearch 6.x, ElasticSearch 7.x, ElasticSearch 8.x, and OpenSearch 1.x, their
configurations are as follows: