You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2024/12/12 01:51:53 UTC
(skywalking) branch master updated: Support `Get Effective TTL Configurations` API. (#12847)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new 841a92b621 Support `Get Effective TTL Configurations` API. (#12847)
841a92b621 is described below
commit 841a92b621307b3738c701fbade45c83166f9c08
Author: 吴晟 Wu Sheng <wu...@foxmail.com>
AuthorDate: Thu Dec 12 09:51:47 2024 +0800
Support `Get Effective TTL Configurations` API. (#12847)
* [Breaking Change] Rename `debugging-query` module to `status-query` module. Relative exposed APIs are **UNCHANGED**.
* Support `Get Effective TTL Configurations` API.
---
docs/en/changes/changes.md | 3 ++
docs/en/debugging/config_dump.md | 13 ++---
docs/en/setup/backend/configuration-vocabulary.md | 2 +-
docs/en/setup/backend/ttl.md | 11 ++--
docs/en/status/query_ttl_setup.md | 48 +++++++++++++++++
docs/menu.yml | 6 ++-
.../skywalking/oap/server/core/CoreModule.java | 6 ++-
.../oap/server/core/CoreModuleProvider.java | 8 +++
.../oap/server/core/query/TTLStatusQuery.java | 60 ++++++++++++++++++++++
.../oap/server/core/storage/StorageModule.java | 2 +
.../storage/ttl/DefaultStorageTTLStatusQuery.java} | 20 ++++----
.../oap/server/core/storage/ttl/MetricsTTL.java} | 22 ++++----
.../oap/server/core/storage/ttl/RecordsTTL.java} | 20 ++++----
.../core/storage/ttl/StorageTTLStatusQuery.java} | 22 ++++----
.../oap/server/core/storage/ttl/TTLDefinition.java | 56 ++++++++++++++++++++
.../skywalking/oap/server/core/CoreModuleTest.java | 2 +-
oap-server/server-query-plugin/pom.xml | 2 +-
.../pom.xml | 2 +-
.../oap/query/debug/DebuggingHTTPHandler.java | 6 +--
.../oap/query/debug/DebuggingSpanRsp.java | 0
.../oap/query/debug/DebuggingTraceRsp.java | 0
.../oap/query/debug/StatusQueryConfig.java} | 2 +-
.../query/debug/StatusQueryExceptionHandler.java} | 2 +-
.../oap/query/debug/StatusQueryModule.java} | 6 +--
.../oap/query/debug/StatusQueryProvider.java} | 18 ++++---
.../oap/query/debug/TTLConfigQueryHandler.java | 58 +++++++++++++++++++++
.../oap/query/debug/log/DebuggingQueryLogsRsp.java | 0
.../oap/query/debug/mqe/DebuggingMQERsp.java | 0
.../DebuggingQueryEndpointTopologyRsp.java | 0
.../DebuggingQueryInstanceTopologyRsp.java | 0
.../topology/DebuggingQueryProcessTopologyRsp.java | 0
.../topology/DebuggingQueryServiceTopologyRsp.java | 0
.../debug/trace/DebuggingQueryTraceBriefRsp.java | 0
.../query/debug/trace/DebuggingQueryTraceRsp.java | 0
.../trace/zipkin/DebuggingZipkinQueryTraceRsp.java | 0
.../zipkin/DebuggingZipkinQueryTracesRsp.java | 0
...ywalking.oap.server.library.module.ModuleDefine | 2 +-
...alking.oap.server.library.module.ModuleProvider | 2 +-
oap-server/server-starter/pom.xml | 2 +-
.../src/main/resources/application.yml | 4 +-
.../plugin/banyandb/BanyanDBStorageConfig.java | 24 +++++++--
.../plugin/banyandb/BanyanDBStorageProvider.java | 6 ++-
.../plugin/banyandb/BanyanDBTTLStatusQuery.java | 43 ++++++++++++++++
.../StorageModuleElasticsearchProvider.java | 6 +++
.../plugin/jdbc/common/JDBCStorageProvider.java | 51 ++++++++++++------
45 files changed, 439 insertions(+), 98 deletions(-)
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 3254ee8abf..d83c74f369 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -6,6 +6,7 @@
* This change would not impact the existing deployment and its feature for our official release users.
* **Warning** If there are custom query plugins for our Elasticsearch indices, this change could break them as
sort queries and aggregation queries which used the unexpected fields are being blocked.
+* [Breaking Change] Rename `debugging-query` module to `status-query` module. Relative exposed APIs are **UNCHANGED**.
#### OAP Server
@@ -34,6 +35,7 @@
* BanyanDB: Support update the Schema when OAP starting.
* BanyanDB: Speed up OAP booting while initializing BanyanDB.
* BanyanDB: Support `@EnableSort` on the column to enable sorting for `IndexRule` and set the default to false.
+* Support `Get Effective TTL Configurations` API.
#### UI
@@ -61,5 +63,6 @@
* Improve BanyanDB documentation.
* Improve component-libraries documentation.
* Improve configuration-vocabulary documentation.
+* Add `Get Effective TTL Configurations` API documentation.
All issues and pull requests are [here](https://github.com/apache/skywalking/milestone/224?closed=1)
diff --git a/docs/en/debugging/config_dump.md b/docs/en/debugging/config_dump.md
index eed2af175b..51ffdad918 100644
--- a/docs/en/debugging/config_dump.md
+++ b/docs/en/debugging/config_dump.md
@@ -1,14 +1,15 @@
-# Scratch The OAP Config Dump
+# Dump Effective Initial Configurations
SkyWalking OAP behaviors could be controlled through hundreds of configurations. It is hard to know what is the final
-configuration as all the configurations could be overrided by system environments.
+configuration as all the configurations could be override by system environments.
-The core config file [application.yml](../../../oap-server/server-starter/src/main/resources/application.yml) lists all
-the configurations
+The core config file [application.yml](../../../oap-server/server-starter/src/main/resources/application.yml) lists all the configurations
and their default values. However, it is still hard to know the runtime value.
-Scratch is a tool to dump the final configuration. It is provided within OAP rest server, which could be accessed
-through HTTP GET `http://{core restHost}:{core restPort}/debugging/config/dump`.
+Dump Effective Initial Configurations API is designed to help users to understand the effective configurations, no matter
+they are initialized in the `application.yml`, or override through system environments.
+- URL, `http://{core restHost}:{core restPort}/debugging/config/dump`
+- HTTP GET method.
```shell
> curl http://127.0.0.1:12800/debugging/config/dump
diff --git a/docs/en/setup/backend/configuration-vocabulary.md b/docs/en/setup/backend/configuration-vocabulary.md
index 22f849300d..c0e08ec239 100644
--- a/docs/en/setup/backend/configuration-vocabulary.md
+++ b/docs/en/setup/backend/configuration-vocabulary.md
@@ -344,7 +344,7 @@ The Configuration Vocabulary lists all available configurations provided by `app
| - | - | kafkaTopicLog | Kafka topic name for log. [...]
| - | - | exportErrorStatusTraceOnly | Export error status trace segments through the Kafka channel. [...]
| health-checker | default | checkIntervalSeconds | The period of checking OAP internal health status (in seconds). [...]
-| debugging-query | default | | [...]
+| status-query | default | | [...]
| - | - | keywords4MaskingSecretsOfConfig | Include the list of keywords to filter configurations including secrets. Separate keywords by a comma. [...]
| configuration-discovery | default | disableMessageDigest | If true, agent receives the latest configuration every time, even without making any changes. By default, OAP uses the SHA512 message digest mechanism to detect changes in configuration. [...]
| receiver-event | default | gRPC services that handle events data. | - [...]
diff --git a/docs/en/setup/backend/ttl.md b/docs/en/setup/backend/ttl.md
index c0c7a9b197..1513880a78 100644
--- a/docs/en/setup/backend/ttl.md
+++ b/docs/en/setup/backend/ttl.md
@@ -1,4 +1,6 @@
-# TTL
+# Data Lifecycle. Time To Live (TTL)
+Time To Live (TTL) is a mechanism to automatically delete data that is older than the specified time.
+
In SkyWalking, there are two types of observability data:
1. Records include traces, logs, topN sampled statements and alarm. `recordDataTTL` applies to **record** data.
1. Metrics include all metrics for service, instance, endpoint, and topology map. Metadata(lists of services, instances, or endpoints) also belongs to metrics. `metricsDataTTL` applies to **Metrics** data.
@@ -12,6 +14,9 @@ These are the settings for the different types:
## BanyanDB TTL
-BanyanDB has a TTL mechanism to automatically delete data that is older than the specified time. When you use BanyanDB as the storage backend, `recordDataTTL` and `metricsDataTTL` are not used. Instead, you should configure the TTL settings in `storage.banyandb`.
+BanyanDB has a TTL mechanism to automatically delete data that is older than the specified time. When you use BanyanDB
+as the storage backend, `recordDataTTL` and `metricsDataTTL` are not used. Instead, you should configure the TTL
+settings in `storage.banyandb`.
-Please refer to the [Storage BanyanDB](storages/banyandb.md) document for more information.
+Please refer to the [Storage BanyanDB](storages/banyandb.md) and [BanyanDB's Progressive TTL](../../banyandb/ttl.md)
+documents for more information.
diff --git a/docs/en/status/query_ttl_setup.md b/docs/en/status/query_ttl_setup.md
new file mode 100644
index 0000000000..3a9c67d960
--- /dev/null
+++ b/docs/en/status/query_ttl_setup.md
@@ -0,0 +1,48 @@
+# Get Effective TTL Configurations
+
+Time To Live (TTL) mechanism has different behaviors according to different storage implementations. By default, the
+core module provides two TTL configurations: [`recordDataTTL` and `metricsDataTTL`](../setup/backend/ttl.md).
+But some storage implementations could override these settings and provide its own TTL configurations, for example,
+BanyanDB provides its native TTL mechanism to support [progressive TTL](../banyandb/ttl.md) feature.
+
+This API is used to get the unified and effective TTL configurations.
+- URL, `http://{core restHost}:{core restPort}/status/config/ttl`
+- HTTP GET method.
+
+```shell
+> curl -X GET "http://127.0.0.1:12800/status/config/ttl"
+# Metrics TTL includes the definition of the TTL of the metrics-ish data in the storage,
+# e.g.
+# 1. The metadata of the service, instance, endpoint, topology map, etc.
+# 2. Generated metrics data from OAL and MAL engines.
+#
+# TTLs for each granularity metrics are listed separately.
+metrics.minute=7
+metrics.hour=7
+metrics.day=7
+
+# Records TTL includes the definition of the TTL of the records data in the storage,
+# Records include traces, logs, sampled slow SQL statements, HTTP requests(by Rover), alarms, etc.
+# Super dataset of records are traces and logs, which volume should be much larger.
+records.default=3
+records.superDataset=3
+```
+
+This API also provides the response in JSON format, which is more friendly for programmatic usage.
+
+```shell
+> curl -X GET "http://127.0.0.1:12800/status/config/ttl" \
+ -H "Accept: application/json"
+
+{
+ "metrics": {
+ "minute": 7,
+ "hour": 7,
+ "day": 7
+ },
+ "records": {
+ "default": 3,
+ "superDataset": 3
+ }
+}
+```
\ No newline at end of file
diff --git a/docs/menu.yml b/docs/menu.yml
index 39b0d7d052..04a5b2322b 100644
--- a/docs/menu.yml
+++ b/docs/menu.yml
@@ -352,12 +352,14 @@ catalog:
path: "/en/concepts-and-designs/metrics-attributes"
- name: "Security Notice"
path: "/en/security/readme"
- - name: "Debugging"
+ - name: "Status APIs"
catalog:
- - name: "Scratch The OAP Config Dump"
+ - name: "Dump Effective Initial Configurations"
path: "/en/debugging/config_dump"
- name: "Tracing Query Execution"
path: "/en/debugging/query-tracing"
+ - name: "Get Effective TTL Configurations"
+ path: "/en/status/query_ttl_setup"
- name: "Academy"
catalog:
- name: "Scaling SkyWalking server automatically in kubernetes"
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModule.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModule.java
index b2c6071f38..7e1cc8a3fa 100755
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModule.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModule.java
@@ -52,6 +52,7 @@ import org.apache.skywalking.oap.server.core.query.MetadataQueryService;
import org.apache.skywalking.oap.server.core.query.MetricsMetadataQueryService;
import org.apache.skywalking.oap.server.core.query.MetricsQueryService;
import org.apache.skywalking.oap.server.core.query.RecordQueryService;
+import org.apache.skywalking.oap.server.core.query.TTLStatusQuery;
import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService;
import org.apache.skywalking.oap.server.core.query.TopNRecordsQueryService;
import org.apache.skywalking.oap.server.core.query.TopologyQueryService;
@@ -96,7 +97,7 @@ public class CoreModule extends ModuleDefine {
addServerInterface(classes);
addReceiverInterface(classes);
- addInsideService(classes);
+ addInternalServices(classes);
addCacheService(classes);
addQueryService(classes);
addProfileService(classes);
@@ -153,6 +154,7 @@ public class CoreModule extends ModuleDefine {
classes.add(TagAutoCompleteQueryService.class);
classes.add(RecordQueryService.class);
classes.add(HierarchyQueryService.class);
+ classes.add(TTLStatusQuery.class);
}
private void addServerInterface(List<Class> classes) {
@@ -160,7 +162,7 @@ public class CoreModule extends ModuleDefine {
classes.add(HTTPHandlerRegister.class);
}
- private void addInsideService(List<Class> classes) {
+ private void addInternalServices(List<Class> classes) {
classes.add(ModelCreator.class);
classes.add(IModelManager.class);
classes.add(ModelManipulator.class);
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
index d0c5bd6156..6370102079 100755
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
@@ -77,6 +77,7 @@ import org.apache.skywalking.oap.server.core.query.MetadataQueryService;
import org.apache.skywalking.oap.server.core.query.MetricsMetadataQueryService;
import org.apache.skywalking.oap.server.core.query.MetricsQueryService;
import org.apache.skywalking.oap.server.core.query.RecordQueryService;
+import org.apache.skywalking.oap.server.core.query.TTLStatusQuery;
import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService;
import org.apache.skywalking.oap.server.core.query.TopNRecordsQueryService;
import org.apache.skywalking.oap.server.core.query.TopologyQueryService;
@@ -304,6 +305,13 @@ public class CoreModuleProvider extends ModuleProvider {
TagAutoCompleteQueryService.class, new TagAutoCompleteQueryService(getManager(), moduleConfig));
this.registerServiceImplementation(RecordQueryService.class, new RecordQueryService(getManager()));
this.registerServiceImplementation(HierarchyQueryService.class, new HierarchyQueryService(getManager(), moduleConfig));
+ this.registerServiceImplementation(
+ TTLStatusQuery.class, new TTLStatusQuery(
+ getManager(),
+ moduleConfig.getMetricsDataTTL(),
+ moduleConfig.getRecordDataTTL()
+ )
+ );
// add profile service implementations
this.registerServiceImplementation(
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/TTLStatusQuery.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/TTLStatusQuery.java
new file mode 100644
index 0000000000..60ee8995e2
--- /dev/null
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/TTLStatusQuery.java
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.query;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.oap.server.core.storage.StorageModule;
+import org.apache.skywalking.oap.server.core.storage.ttl.MetricsTTL;
+import org.apache.skywalking.oap.server.core.storage.ttl.RecordsTTL;
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
+import org.apache.skywalking.oap.server.core.storage.ttl.TTLDefinition;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.library.module.Service;
+
+@RequiredArgsConstructor
+public class TTLStatusQuery implements Service {
+ private final ModuleManager moduleManager;
+ private final int coreMetricsDataTTL;
+ private final int coreRecordDataTTL;
+
+ private StorageTTLStatusQuery storageTTLStatusQuery;
+
+ private StorageTTLStatusQuery getStorageTTLStatusQuery() {
+ if (storageTTLStatusQuery == null) {
+ storageTTLStatusQuery = moduleManager.find(StorageModule.NAME)
+ .provider()
+ .getService(StorageTTLStatusQuery.class);
+ }
+ return storageTTLStatusQuery;
+ }
+
+ /**
+ * @return effective TTL configuration values.
+ */
+ public TTLDefinition getTTL() {
+ TTLDefinition ttlDefinition = getStorageTTLStatusQuery().getTTL();
+ if (ttlDefinition == null) {
+ ttlDefinition = new TTLDefinition(
+ new MetricsTTL(coreMetricsDataTTL, coreMetricsDataTTL, coreMetricsDataTTL),
+ new RecordsTTL(coreRecordDataTTL, coreRecordDataTTL)
+ );
+ }
+ return ttlDefinition;
+ }
+}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/StorageModule.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/StorageModule.java
index 95b623d6e3..89ac015cd0 100644
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/StorageModule.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/StorageModule.java
@@ -46,6 +46,7 @@ import org.apache.skywalking.oap.server.core.storage.query.IRecordsQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IZipkinQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
/**
@@ -95,6 +96,7 @@ public class StorageModule extends ModuleDefine {
IAsyncProfilerTaskQueryDAO.class,
IAsyncProfilerTaskLogQueryDAO.class,
IJFRDataQueryDAO.class,
+ StorageTTLStatusQuery.class
};
}
}
diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/DefaultStorageTTLStatusQuery.java
similarity index 62%
copy from oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java
copy to oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/DefaultStorageTTLStatusQuery.java
index 76942345d6..4ab904cdf9 100644
--- a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/DefaultStorageTTLStatusQuery.java
@@ -16,16 +16,14 @@
*
*/
-package org.apache.skywalking.oap.server.core;
+package org.apache.skywalking.oap.server.core.storage.ttl;
-import org.junit.jupiter.api.Assertions;
-import org.junit.jupiter.api.Test;
-
-public class CoreModuleTest {
- @Test
- public void testOpenServiceList() {
- CoreModule coreModule = new CoreModule();
-
- Assertions.assertEquals(47, coreModule.services().length);
- }
+/**
+ * The default implementation of {@link StorageTTLStatusQuery}.
+ * This is used when the storage doesn't provide the TTL customization.
+ * Typically, BanyanDB provides advanced progressive TTL configurations.
+ * For more details, visit <a href="https://skywalking.apache.org/docs/main/next/en/banyandb/ttl">Progressive TTL
+ * Documentation</a>.
+ */
+public class DefaultStorageTTLStatusQuery implements StorageTTLStatusQuery {
}
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/MetricsTTL.java
similarity index 64%
copy from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
copy to oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/MetricsTTL.java
index da17e16d98..e3a72dd328 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/MetricsTTL.java
@@ -16,17 +16,21 @@
*
*/
-package org.apache.skywalking.oap.query.debug.trace.zipkin;
+package org.apache.skywalking.oap.server.core.storage.ttl;
-import java.util.List;
import lombok.Data;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.oap.query.debug.DebuggingTraceRsp;
-import zipkin2.Span;
-@RequiredArgsConstructor
+/**
+ * Metrics TTL includes the definition of the TTL of the metrics-ish data in the storage,
+ * e.g.
+ * 1. The metadata of the service, instance, endpoint, topology map, etc.
+ * 2. Generated metrics data from OAL and MAL engines.
+ *
+ * TTLs for ach granularity metrics are listed separately.
+ */
@Data
-public class DebuggingZipkinQueryTraceRsp {
- private final List<Span> trace;
- private final DebuggingTraceRsp debuggingTrace;
+public class MetricsTTL {
+ private final int minute;
+ private final int hour;
+ private final int day;
}
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/RecordsTTL.java
similarity index 62%
copy from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
copy to oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/RecordsTTL.java
index da17e16d98..09484f936b 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/RecordsTTL.java
@@ -16,17 +16,19 @@
*
*/
-package org.apache.skywalking.oap.query.debug.trace.zipkin;
+package org.apache.skywalking.oap.server.core.storage.ttl;
-import java.util.List;
+import com.google.gson.annotations.SerializedName;
import lombok.Data;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.oap.query.debug.DebuggingTraceRsp;
-import zipkin2.Span;
-@RequiredArgsConstructor
+/**
+ * RecordsTTL includes the definition of the TTL of the records data in the storage,
+ * Records include traces, logs, sampled slow SQL statements, HTTP requests(by Rover), alarms, etc.
+ * Super dataset of records are traces and logs, which volume should be much larger.
+ */
@Data
-public class DebuggingZipkinQueryTraceRsp {
- private final List<Span> trace;
- private final DebuggingTraceRsp debuggingTrace;
+public class RecordsTTL {
+ @SerializedName("default")
+ private final int value;
+ private final int superDataset;
}
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/StorageTTLStatusQuery.java
similarity index 65%
copy from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java
copy to oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/StorageTTLStatusQuery.java
index 02b5fd5991..1053b9daa9 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/StorageTTLStatusQuery.java
@@ -16,18 +16,18 @@
*
*/
-package org.apache.skywalking.oap.query.debug;
+package org.apache.skywalking.oap.server.core.storage.ttl;
-import org.apache.skywalking.oap.server.library.module.ModuleDefine;
+import org.apache.skywalking.oap.server.library.module.Service;
-public class DebuggingQueryModule extends ModuleDefine {
- public static final String NAME = "debugging-query";
-
- public DebuggingQueryModule() {
- super(NAME);
- }
-
- public Class[] services() {
- return new Class[0];
+public interface StorageTTLStatusQuery extends Service {
+ /**
+ * Get the TTL of the metrics and records data from the selected storage.
+ *
+ * @return null if the storage doesn't support TTL customization. Or return the TTL definition from specific storage
+ * implementation.
+ */
+ default TTLDefinition getTTL() {
+ return null;
}
}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/TTLDefinition.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/TTLDefinition.java
new file mode 100644
index 0000000000..5ef4b90ff6
--- /dev/null
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/ttl/TTLDefinition.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.storage.ttl;
+
+import com.google.gson.Gson;
+import lombok.Data;
+
+/**
+ * TTLDefinition defines the TTL of the data in the storage.
+ */
+@Data
+public class TTLDefinition {
+ private final static Gson GSON = new Gson();
+ private final MetricsTTL metrics;
+ private final RecordsTTL records;
+
+ public String generateTTLDefinition() {
+ StringBuilder ttlDefinition = new StringBuilder();
+ ttlDefinition.append("# Metrics TTL includes the definition of the TTL of the metrics-ish data in the storage,\n");
+ ttlDefinition.append("# e.g.\n");
+ ttlDefinition.append("# 1. The metadata of the service, instance, endpoint, topology map, etc.\n");
+ ttlDefinition.append("# 2. Generated metrics data from OAL and MAL engines.\n");
+ ttlDefinition.append("#\n");
+ ttlDefinition.append("# TTLs for each granularity metrics are listed separately.\n");
+ ttlDefinition.append("metrics.minute=").append(metrics.getMinute()).append("\n");
+ ttlDefinition.append("metrics.hour=").append(metrics.getHour()).append("\n");
+ ttlDefinition.append("metrics.day=").append(metrics.getDay()).append("\n");
+ ttlDefinition.append("\n");
+ ttlDefinition.append("# Records TTL includes the definition of the TTL of the records data in the storage,\n");
+ ttlDefinition.append("# Records include traces, logs, sampled slow SQL statements, HTTP requests(by Rover), alarms, etc.\n");
+ ttlDefinition.append("# Super dataset of records are traces and logs, which volume should be much larger.\n");
+ ttlDefinition.append("records.default=").append(records.getValue()).append("\n");
+ ttlDefinition.append("records.superDataset=").append(records.getSuperDataset()).append("\n");
+ return ttlDefinition.toString();
+ }
+
+ public String generateTTLDefinitionAsJSONStr() {
+ return GSON.toJson(this);
+ }
+}
diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java
index 76942345d6..4f6276d97a 100644
--- a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java
+++ b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/CoreModuleTest.java
@@ -26,6 +26,6 @@ public class CoreModuleTest {
public void testOpenServiceList() {
CoreModule coreModule = new CoreModule();
- Assertions.assertEquals(47, coreModule.services().length);
+ Assertions.assertEquals(48, coreModule.services().length);
}
}
diff --git a/oap-server/server-query-plugin/pom.xml b/oap-server/server-query-plugin/pom.xml
index 6ef535e84a..87bc68a8ba 100644
--- a/oap-server/server-query-plugin/pom.xml
+++ b/oap-server/server-query-plugin/pom.xml
@@ -46,6 +46,6 @@
<module>zipkin-query-plugin</module>
<module>promql-plugin</module>
<module>logql-plugin</module>
- <module>debugging-query-plugin</module>
+ <module>status-query-plugin</module>
</modules>
</project>
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/pom.xml b/oap-server/server-query-plugin/status-query-plugin/pom.xml
similarity index 97%
rename from oap-server/server-query-plugin/debugging-query-plugin/pom.xml
rename to oap-server/server-query-plugin/status-query-plugin/pom.xml
index 1e2226180e..67c410d184 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/pom.xml
+++ b/oap-server/server-query-plugin/status-query-plugin/pom.xml
@@ -25,7 +25,7 @@
</parent>
<modelVersion>4.0.0</modelVersion>
- <artifactId>debugging-query-plugin</artifactId>
+ <artifactId>status-query-plugin</artifactId>
<packaging>jar</packaging>
<dependencies>
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java
similarity index 99%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java
index 8b52f37075..564f7c4354 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingHTTPHandler.java
@@ -85,7 +85,7 @@ import org.apache.skywalking.oap.server.library.module.ModuleManager;
import zipkin2.Span;
@Slf4j
-@ExceptionHandler(DebuggingQueryExceptionHandler.class)
+@ExceptionHandler(StatusQueryExceptionHandler.class)
public class DebuggingHTTPHandler {
private final ServerStatusService serverStatusService;
private final MetricsExpressionQuery mqeQuery;
@@ -93,9 +93,9 @@ public class DebuggingHTTPHandler {
private final ZipkinQueryHandler zipkinQueryHandler;
private final TopologyQuery topologyQuery;
private final LogQuery logQuery;
- final DebuggingQueryConfig config;
+ final StatusQueryConfig config;
- public DebuggingHTTPHandler(final ModuleManager manager, final DebuggingQueryConfig config) {
+ public DebuggingHTTPHandler(final ModuleManager manager, final StatusQueryConfig config) {
serverStatusService = manager.find(CoreModule.NAME)
.provider()
.getService(ServerStatusService.class);
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingSpanRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingSpanRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingSpanRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingSpanRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingTraceRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingTraceRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingTraceRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingTraceRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryConfig.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryConfig.java
similarity index 95%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryConfig.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryConfig.java
index 217ced2427..4be0a9c8a4 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryConfig.java
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryConfig.java
@@ -22,7 +22,7 @@ import lombok.Getter;
import org.apache.skywalking.oap.server.library.module.ModuleConfig;
@Getter
-public class DebuggingQueryConfig extends ModuleConfig {
+public class StatusQueryConfig extends ModuleConfig {
/**
* Include the list of keywords to filter configurations including secrets. Separate keywords by a comma.
*
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryExceptionHandler.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryExceptionHandler.java
similarity index 96%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryExceptionHandler.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryExceptionHandler.java
index 5bede1f430..28d4037a54 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryExceptionHandler.java
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryExceptionHandler.java
@@ -29,7 +29,7 @@ import static com.linecorp.armeria.common.HttpStatus.INTERNAL_SERVER_ERROR;
import static com.linecorp.armeria.common.MediaType.ANY_TEXT_TYPE;
@Slf4j
-public class DebuggingQueryExceptionHandler implements ExceptionHandlerFunction {
+public class StatusQueryExceptionHandler implements ExceptionHandlerFunction {
@Override
public HttpResponse handleException(final ServiceRequestContext ctx, final HttpRequest req, final Throwable cause) {
String rspMsg = cause.getMessage() != null ? cause.getMessage() : cause.getClass().getSimpleName();
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryModule.java
similarity index 87%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryModule.java
index 02b5fd5991..6375966d8d 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryModule.java
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryModule.java
@@ -20,10 +20,10 @@ package org.apache.skywalking.oap.query.debug;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
-public class DebuggingQueryModule extends ModuleDefine {
- public static final String NAME = "debugging-query";
+public class StatusQueryModule extends ModuleDefine {
+ public static final String NAME = "status-query";
- public DebuggingQueryModule() {
+ public StatusQueryModule() {
super(NAME);
}
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryProvider.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryProvider.java
similarity index 82%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryProvider.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryProvider.java
index a3618feffb..635a01d1ea 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/DebuggingQueryProvider.java
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/StatusQueryProvider.java
@@ -28,28 +28,28 @@ import org.apache.skywalking.oap.server.library.module.ModuleProvider;
import org.apache.skywalking.oap.server.library.module.ModuleStartException;
import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
-public class DebuggingQueryProvider extends ModuleProvider {
+public class StatusQueryProvider extends ModuleProvider {
public static final String NAME = "default";
- private DebuggingQueryConfig config;
+ private StatusQueryConfig config;
public String name() {
return NAME;
}
public Class<? extends ModuleDefine> module() {
- return DebuggingQueryModule.class;
+ return StatusQueryModule.class;
}
public ConfigCreator<? extends ModuleConfig> newConfigCreator() {
- return new ConfigCreator<DebuggingQueryConfig>() {
+ return new ConfigCreator<StatusQueryConfig>() {
@Override
- public Class<DebuggingQueryConfig> type() {
- return DebuggingQueryConfig.class;
+ public Class<StatusQueryConfig> type() {
+ return StatusQueryConfig.class;
}
@Override
- public void onInitialized(final DebuggingQueryConfig initialized) {
+ public void onInitialized(final StatusQueryConfig initialized) {
config = initialized;
}
};
@@ -67,6 +67,10 @@ public class DebuggingQueryProvider extends ModuleProvider {
new DebuggingHTTPHandler(getManager(), config),
Collections.singletonList(HttpMethod.GET)
);
+ service.addHandler(
+ new TTLConfigQueryHandler(getManager()),
+ Collections.singletonList(HttpMethod.GET)
+ );
}
public void notifyAfterCompleted() throws ServiceNotProvidedException, ModuleStartException {
diff --git a/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/TTLConfigQueryHandler.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/TTLConfigQueryHandler.java
new file mode 100644
index 0000000000..05ade021ee
--- /dev/null
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/TTLConfigQueryHandler.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.debug;
+
+import com.linecorp.armeria.common.HttpHeaderNames;
+import com.linecorp.armeria.common.HttpRequest;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.server.annotation.ExceptionHandler;
+import com.linecorp.armeria.server.annotation.Get;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.query.TTLStatusQuery;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+
+@Slf4j
+@ExceptionHandler(StatusQueryExceptionHandler.class)
+public class TTLConfigQueryHandler {
+ private final ModuleManager moduleManager;
+ private TTLStatusQuery ttlStatusQuery;
+
+ public TTLConfigQueryHandler(final ModuleManager manager) {
+ this.moduleManager = manager;
+ }
+
+ private TTLStatusQuery getTTLStatusQuery() {
+ if (ttlStatusQuery == null) {
+ ttlStatusQuery = moduleManager.find(CoreModule.NAME)
+ .provider()
+ .getService(TTLStatusQuery.class);
+ }
+ return ttlStatusQuery;
+ }
+
+ @Get("/status/config/ttl")
+ public HttpResponse affectedTTLConfigurations(HttpRequest request) {
+ if ("application/json".equalsIgnoreCase(request.headers().get(HttpHeaderNames.ACCEPT))) {
+ return HttpResponse.of(MediaType.JSON_UTF_8, getTTLStatusQuery().getTTL().generateTTLDefinitionAsJSONStr());
+ }
+ return HttpResponse.of(MediaType.PLAIN_TEXT_UTF_8, getTTLStatusQuery().getTTL().generateTTLDefinition());
+ }
+}
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/log/DebuggingQueryLogsRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/log/DebuggingQueryLogsRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/log/DebuggingQueryLogsRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/log/DebuggingQueryLogsRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/mqe/DebuggingMQERsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/mqe/DebuggingMQERsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/mqe/DebuggingMQERsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/mqe/DebuggingMQERsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryEndpointTopologyRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryEndpointTopologyRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryEndpointTopologyRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryEndpointTopologyRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryInstanceTopologyRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryInstanceTopologyRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryInstanceTopologyRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryInstanceTopologyRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryProcessTopologyRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryProcessTopologyRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryProcessTopologyRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryProcessTopologyRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryServiceTopologyRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryServiceTopologyRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryServiceTopologyRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/topology/DebuggingQueryServiceTopologyRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceBriefRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceBriefRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceBriefRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceBriefRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/DebuggingQueryTraceRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTraceRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTracesRsp.java b/oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTracesRsp.java
similarity index 100%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTracesRsp.java
rename to oap-server/server-query-plugin/status-query-plugin/src/main/java/org/apache/skywalking/oap/query/debug/trace/zipkin/DebuggingZipkinQueryTracesRsp.java
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine b/oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
similarity index 93%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
rename to oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
index 0a5b82263a..e63b6e2066 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine
@@ -16,4 +16,4 @@
#
#
-org.apache.skywalking.oap.query.debug.DebuggingQueryModule
+org.apache.skywalking.oap.query.debug.StatusQueryModule
diff --git a/oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider b/oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
similarity index 92%
rename from oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
rename to oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
index 32aee0d7b1..94780edbbf 100644
--- a/oap-server/server-query-plugin/debugging-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
+++ b/oap-server/server-query-plugin/status-query-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
@@ -16,4 +16,4 @@
#
#
-org.apache.skywalking.oap.query.debug.DebuggingQueryProvider
\ No newline at end of file
+org.apache.skywalking.oap.query.debug.StatusQueryProvider
\ No newline at end of file
diff --git a/oap-server/server-starter/pom.xml b/oap-server/server-starter/pom.xml
index d74a4ae02b..5eeefb808e 100644
--- a/oap-server/server-starter/pom.xml
+++ b/oap-server/server-starter/pom.xml
@@ -113,7 +113,7 @@
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
- <artifactId>debugging-query-plugin</artifactId>
+ <artifactId>status-query-plugin</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
diff --git a/oap-server/server-starter/src/main/resources/application.yml b/oap-server/server-starter/src/main/resources/application.yml
index 038477fc33..ee396483ea 100644
--- a/oap-server/server-starter/src/main/resources/application.yml
+++ b/oap-server/server-starter/src/main/resources/application.yml
@@ -641,8 +641,8 @@ health-checker:
default:
checkIntervalSeconds: ${SW_HEALTH_CHECKER_INTERVAL_SECONDS:5}
-debugging-query:
- selector: ${SW_DEBUGGING_QUERY:default}
+status-query:
+ selector: ${SW_STATUS_QUERY:default}
default:
# Include the list of keywords to filter configurations including secrets. Separate keywords by a comma.
keywords4MaskingSecretsOfConfig: ${SW_DEBUGGING_QUERY_KEYWORDS_FOR_MASKING_SECRETS:user,password,token,accessKey,secretKey,authentication}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
index 0d968237bb..9ad6c03d79 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
@@ -70,10 +70,26 @@ public class BanyanDBStorageConfig extends ModuleConfig {
private int segmentQueryMaxSize = 200;
private int profileDataQueryBatchSize = 100;
- /**
- * The configuration of the groups.
- * @since 10.2.0
- */
+ // ----------------------------------------
+ // The configuration of the groups.
+ // since 10.2.0
+ // ----------------------------------------
+ // The group settings of record.
+ // `gr` is the short name of the group settings of record.
+ //
+ // The "normal"(`gr...`) section defines settings for datasets not specified in "super".
+ // Each dataset will be grouped under a single group named "normal".
+ // "super"(`grSuper...`) is a special dataset designed to store trace or log data that is too large for normal datasets.
+ // # Each super dataset will be a separate group in BanyanDB, following the settings defined in the "super" section.
+ // ----------------------------------------
+ // The group settings of metrics.
+ // `gm` is the short name of the group settings of metrics.
+ //
+ // OAP stores metrics based its granularity.
+ // Valid values are "day", "hour", and "minute". That means metrics will be stored in the three separate groups.
+ // Non-"minute" are governed by the "core.downsampling" setting.
+ // For example, if "core.downsampling" is set to "hour", the "hour" will be used, while "day" are ignored.
+
private int grNormalShardNum = 1;
private int grNormalSIDays = 1;
private int grNormalTTLDays = 3;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 58e6e314b1..0d777ba2da 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -55,6 +55,7 @@ import org.apache.skywalking.oap.server.core.storage.query.ITagAutoCompleteQuery
import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IZipkinQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
import org.apache.skywalking.oap.server.library.module.ModuleStartException;
@@ -186,8 +187,11 @@ public class BanyanDBStorageProvider extends ModuleProvider {
IAsyncProfilerTaskLogQueryDAO.class, new BanyanDBAsyncProfilerTaskLogQueryDAO(client,
this.config.getAsyncProfilerTaskQueryMaxSize()
));
+ this.registerServiceImplementation(IJFRDataQueryDAO.class, new BanyanDBJFRDataQueryDAO(client));
this.registerServiceImplementation(
- IJFRDataQueryDAO.class, new BanyanDBJFRDataQueryDAO(client));
+ StorageTTLStatusQuery.class,
+ new BanyanDBTTLStatusQuery(config)
+ );
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTTLStatusQuery.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTTLStatusQuery.java
new file mode 100644
index 0000000000..9ad975c5f3
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTTLStatusQuery.java
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
+import org.apache.skywalking.oap.server.core.storage.ttl.TTLDefinition;
+
+public class BanyanDBTTLStatusQuery implements StorageTTLStatusQuery {
+ private final int grNormalTTLDays;
+ private final int grSuperTTLDays;
+ private final int gmMinuteTTLDays;
+ private final int gmHourTTLDays;
+ private final int gmDayTTLDays;
+
+ public BanyanDBTTLStatusQuery(BanyanDBStorageConfig config) {
+ grNormalTTLDays = config.getGrNormalTTLDays();
+ grSuperTTLDays = config.getGrSuperTTLDays();
+ gmMinuteTTLDays = config.getGmMinuteTTLDays();
+ gmHourTTLDays = config.getGmHourTTLDays();
+ gmDayTTLDays = config.getGmDayTTLDays();
+ }
+
+ @Override
+ public TTLDefinition getTTL() {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchProvider.java b/oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchProvider.java
index 0302452553..cd255d99b6 100644
--- a/oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchProvider.java
+++ b/oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchProvider.java
@@ -61,6 +61,8 @@ import org.apache.skywalking.oap.server.core.storage.query.ITagAutoCompleteQuery
import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IZipkinQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.ttl.DefaultStorageTTLStatusQuery;
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
import org.apache.skywalking.oap.server.library.client.elasticsearch.ElasticSearchClient;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
@@ -273,6 +275,10 @@ public class StorageModuleElasticsearchProvider extends ModuleProvider {
IJFRDataQueryDAO.class,
new JFRDataQueryEsDAO(elasticSearchClient)
);
+ this.registerServiceImplementation(
+ StorageTTLStatusQuery.class,
+ new DefaultStorageTTLStatusQuery()
+ );
IndexController.INSTANCE.setLogicSharding(config.isLogicSharding());
IndexController.INSTANCE.setEnableCustomRouting(config.isEnableCustomRouting());
this.registerServiceImplementation(IHierarchyQueryDAO.class, new HierarchyQueryEsDAO(elasticSearchClient, config));
diff --git a/oap-server/server-storage-plugin/storage-jdbc-hikaricp-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/jdbc/common/JDBCStorageProvider.java b/oap-server/server-storage-plugin/storage-jdbc-hikaricp-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/jdbc/common/JDBCStorageProvider.java
index a46d0a6a3f..2fd5cbcb49 100644
--- a/oap-server/server-storage-plugin/storage-jdbc-hikaricp-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/jdbc/common/JDBCStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-jdbc-hikaricp-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/jdbc/common/JDBCStorageProvider.java
@@ -55,6 +55,8 @@ import org.apache.skywalking.oap.server.core.storage.query.ITagAutoCompleteQuery
import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IZipkinQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.ttl.DefaultStorageTTLStatusQuery;
+import org.apache.skywalking.oap.server.core.storage.ttl.StorageTTLStatusQuery;
import org.apache.skywalking.oap.server.library.client.jdbc.hikaricp.JDBCClient;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
@@ -207,39 +209,56 @@ public abstract class JDBCStorageProvider extends ModuleProvider {
this.registerServiceImplementation(
IEBPFProfilingTaskDAO.class,
- new JDBCEBPFProfilingTaskDAO(jdbcClient, tableHelper));
+ new JDBCEBPFProfilingTaskDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
IEBPFProfilingScheduleDAO.class,
- new JDBCEBPFProfilingScheduleDAO(jdbcClient, tableHelper));
+ new JDBCEBPFProfilingScheduleDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
IEBPFProfilingDataDAO.class,
- new JDBCEBPFProfilingDataDAO(jdbcClient, tableHelper));
+ new JDBCEBPFProfilingDataDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
IContinuousProfilingPolicyDAO.class,
- new JDBCContinuousProfilingPolicyDAO(jdbcClient, tableHelper));
+ new JDBCContinuousProfilingPolicyDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
IServiceLabelDAO.class,
- new JDBCServiceLabelQueryDAO(jdbcClient, tableHelper));
+ new JDBCServiceLabelQueryDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
ITagAutoCompleteQueryDAO.class,
- new JDBCTagAutoCompleteQueryDAO(jdbcClient, tableHelper));
+ new JDBCTagAutoCompleteQueryDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
IZipkinQueryDAO.class,
- new JDBCZipkinQueryDAO(jdbcClient, tableHelper));
+ new JDBCZipkinQueryDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
ISpanAttachedEventQueryDAO.class,
- new JDBCSpanAttachedEventQueryDAO(jdbcClient, tableHelper));
- this.registerServiceImplementation(IHierarchyQueryDAO.class,
- new JDBCHierarchyQueryDAO(jdbcClient, config.getMetadataQueryMaxSize(), tableHelper));
+ new JDBCSpanAttachedEventQueryDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
- IAsyncProfilerTaskQueryDAO.class,
- new JDBCAsyncProfilerTaskQueryDAO(jdbcClient, tableHelper));
+ IHierarchyQueryDAO.class,
+ new JDBCHierarchyQueryDAO(jdbcClient, config.getMetadataQueryMaxSize(), tableHelper)
+ );
this.registerServiceImplementation(
- IAsyncProfilerTaskLogQueryDAO.class,
- new JDBCAsyncProfilerTaskLogQueryDAO(jdbcClient, tableHelper));
+ IAsyncProfilerTaskQueryDAO.class,
+ new JDBCAsyncProfilerTaskQueryDAO(jdbcClient, tableHelper)
+ );
this.registerServiceImplementation(
- IJFRDataQueryDAO.class,
- new JDBCJFRDataQueryDAO(jdbcClient, tableHelper));
+ IAsyncProfilerTaskLogQueryDAO.class,
+ new JDBCAsyncProfilerTaskLogQueryDAO(jdbcClient, tableHelper)
+ );
+ this.registerServiceImplementation(
+ IJFRDataQueryDAO.class,
+ new JDBCJFRDataQueryDAO(jdbcClient, tableHelper)
+ );
+ this.registerServiceImplementation(
+ StorageTTLStatusQuery.class,
+ new DefaultStorageTTLStatusQuery()
+ );
}
@Override