You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2021/05/09 02:09:35 UTC
[skywalking] branch master updated: Add OpenSearch as a storage
option (#6916)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new c5e0c01 Add OpenSearch as a storage option (#6916)
c5e0c01 is described below
commit c5e0c01f0655ef1dd62bbf67d094f97b31d1c142
Author: Zhenxu Ke <ke...@apache.org>
AuthorDate: Sun May 9 10:09:15 2021 +0800
Add OpenSearch as a storage option (#6916)
---
.github/workflows/e2e.storages.yaml | 7 +++-
CHANGES.md | 2 +-
docs/en/setup/backend/backend-storage.md | 16 +++++++-
test/e2e/e2e-test/docker/build-opensearch.sh | 27 +++++++++++++
.../docker/storage/docker-compose.opensearch.yml | 47 ++++++++++++++++++++++
5 files changed, 95 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/e2e.storages.yaml b/.github/workflows/e2e.storages.yaml
index 0aec03a..ba3a798 100644
--- a/.github/workflows/e2e.storages.yaml
+++ b/.github/workflows/e2e.storages.yaml
@@ -31,7 +31,7 @@ jobs:
timeout-minutes: 90
strategy:
matrix:
- storage: ['mysql', 'es6', 'es7.0', 'es7.10', 'influxdb', 'tidb', 'postgresql']
+ storage: ['mysql', 'es6', 'es7.0', 'es7.10', 'opensearch', 'influxdb', 'tidb', 'postgresql']
env:
SW_STORAGE: ${{ matrix.storage }}
steps:
@@ -48,6 +48,11 @@ jobs:
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
+
+ - name: Build OpenSearch
+ if: matrix.storage == 'opensearch'
+ run: bash test/e2e/e2e-test/docker/build-opensearch.sh
+
- name: Run E2E Test
if: env.SKIP_CI != 'true'
uses: ./.github/actions/e2e-test
diff --git a/CHANGES.md b/CHANGES.md
index d4d12cd..edcc8d9 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -5,7 +5,7 @@ Release Notes.
8.6.0
------------------
#### Project
-
+* Add OpenSearch as storage option.
#### Java Agent
* Add `trace_segment_ref_limit_per_span` configuration mechanism to avoid OOM.
diff --git a/docs/en/setup/backend/backend-storage.md b/docs/en/setup/backend/backend-storage.md
index 0d0d282..4be32ca 100644
--- a/docs/en/setup/backend/backend-storage.md
+++ b/docs/en/setup/backend/backend-storage.md
@@ -9,6 +9,7 @@ storage:
Native supported storage
- H2
+- OpenSearch
- ElasticSearch 6, 7
- MySQL
- TiDB
@@ -31,14 +32,25 @@ storage:
user: sa
```
+## OpenSearch
+
+OpenSearch storage shares the same configurations as ElasticSearch 7.
+In order to activate ElasticSearch 7 as storage, set storage provider to **elasticsearch7**.
+Please download the `apache-skywalking-bin-es7.tar.gz` if you want to use OpenSearch as storage.
+
## ElasticSearch
+
+**NOTICE:** Elastic announced through their blog that Elasticsearch will be moving over to a Server Side Public
+License (SSPL), which is incompatible with Apache License 2.0. This license change is effective from Elasticsearch
+version 7.11. So please choose the suitable ElasticSearch version according to your usage.
+
- In order to activate ElasticSearch 6 as storage, set storage provider to **elasticsearch**
- In order to activate ElasticSearch 7 as storage, set storage provider to **elasticsearch7**
**Required ElasticSearch 6.3.2 or higher. HTTP RestHighLevelClient is used to connect server.**
-- For ElasticSearch 6.3.2 ~ 7.0.0 (excluded), please download the `apache-skywalking-bin.tar.gz` or `apache-skywalking-bin.zip`,
-- For ElasticSearch 7.0.0 ~ 8.0.0 (excluded), please download the `apache-skywalking-bin-es7.tar.gz` or `apache-skywalking-bin-es7.zip`.
+- For ElasticSearch 6.3.2 ~ 7.0.0 (excluded), please download the `apache-skywalking-bin.tar.gz`,
+- For ElasticSearch 7.0.0 ~ 8.0.0 (excluded), please download the `apache-skywalking-bin-es7.tar.gz`.
For now, ElasticSearch 6 and ElasticSearch 7 share the same configurations, as follows:
diff --git a/test/e2e/e2e-test/docker/build-opensearch.sh b/test/e2e/e2e-test/docker/build-opensearch.sh
new file mode 100644
index 0000000..6db86ce
--- /dev/null
+++ b/test/e2e/e2e-test/docker/build-opensearch.sh
@@ -0,0 +1,27 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+
+set -ex
+
+wd=$(mktemp -d -t opensearch-XXXXXXXXXX) && cd "$wd"
+
+curl -L https://github.com/opensearch-project/OpenSearch/archive/1.0.0-beta1.tar.gz -o opensearch.tgz
+
+tar -zxf opensearch.tgz --strip-components=1 -C .
+
+./gradlew clean distribution:docker:assemble -x test -x javadoc
diff --git a/test/e2e/e2e-test/docker/storage/docker-compose.opensearch.yml b/test/e2e/e2e-test/docker/storage/docker-compose.opensearch.yml
new file mode 100644
index 0000000..2724412
--- /dev/null
+++ b/test/e2e/e2e-test/docker/storage/docker-compose.opensearch.yml
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: '2.1'
+
+services:
+ es:
+ image: opensearch:test
+ expose:
+ - 9200
+ networks:
+ - e2e
+ environment:
+ - discovery.type=single-node
+ - cluster.routing.allocation.disk.threshold_enabled=false
+ healthcheck:
+ test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9200"]
+ interval: 5s
+ timeout: 60s
+ retries: 120
+
+ oap:
+ extends:
+ file: ../base-compose.yml
+ service: oap-es7
+ environment:
+ SW_STORAGE: elasticsearch7
+ SW_PROMETHEUS_FETCHER: "default"
+ SW_TELEMETRY: prometheus
+ depends_on:
+ es:
+ condition: service_healthy
+
+networks:
+ e2e: