You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by lu...@apache.org on 2022/04/30 09:03:19 UTC
[skywalking] branch banyandb-integration-stream updated (b043af38d5 -> cd13a7372a)
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a change to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
discard b043af38d5 add measure id and check sharding keys
discard 058c3a859a refactor converter and measure reigstration
omit f54cafb0b6 support UI Template
omit 6aac11e47c make code compiled
omit aff43b1be2 remove all hardcoded builder
omit 84637bd15d fix styles
omit a82c6e1f5a support metadata, segment and topo
omit e5a25364c9 add metadata json
omit 1dc57cee0c support events, profile_task and instance_traffic
omit fa92429ed8 support metadata-v2
omit 840141d110 add banyandb plugin to pom
omit 837f9373bc use unified tag family name
omit 0a93ed6601 fix license headers
omit 560f3980f0 bump version to 9.0.0-SNAPSHOT
omit 8d3ea5795a add installer
omit f8b7633800 refactor deserializer
omit 1634de9ea6 fix pom and fix style issues
omit fe8b642ccd register necessary modules
omit 7c7140e535 complete basic logic
omit 2733b18c35 refactor deser
omit 401857ead8 add alarm, networkAddressAlias and logs
omit a363ba045b complete UITemplate, ProfileTaskRecord and ProfileThreasSnapshot query
omit ca33c8bb71 abstract row entity mapper
omit c0ecd62f3f start from new code base and copy all previous stream impl
add 70560e3b2d HTTPServer support the handler register with allowed HTTP methods (#8917)
add 05afa982f4 ADD kafka consumers number default = 1,enhance consumption capacity (#8828)
add 023a2d3165 Revert "Enhance DataCarrier#MultipleChannelsConsumer to add priority" (#8924)
add 6a6f89e2c3 Fix the kafka consumer configuration override bug. (#8929)
add 210f26b7df Remove build time from the OAP version (#8934)
add d2fcd83018 Documentation grammar enhancements (#8935)
add 3256b6c9d9 Add data-generator module to run OAP in testing mode, generating mock data for testing (#8725)
add 92ce13c9c4 Publish snapshot Docker image for data generator (#8939)
add 5c247b8d7f sync ui: Implement the eBPF profiling widget on dashboard (#8943)
add 4e29ba5e54 Chore: polish and add missing MAL test case (#8947)
add a93667184f Support receive Kubernetes process from the Rover (#8960)
add 8170177900 fix bug: Even after a rerun of init mode job, TimeSeriesTable ES index not created. (#8958)
add 12ba7aa533 Support autocomplete tags in traces query (backend side), Replace all configurations **_JETTY_** to **_REST_** (#8971)
add 211efc0562 Further remove jetty references and config (#8974)
new 896223ab37 start from new code base and copy all previous stream impl
new c0bbcf1f68 abstract row entity mapper
new b89c8ced7a complete UITemplate, ProfileTaskRecord and ProfileThreasSnapshot query
new cb705ed0cc add alarm, networkAddressAlias and logs
new 7fb9a6ad37 refactor deser
new 799608dd59 complete basic logic
new 32c0b82a59 register necessary modules
new 4f41af3368 fix pom and fix style issues
new a3bba672ec refactor deserializer
new a8e161f033 add installer
new 773307e9a8 bump version to 9.0.0-SNAPSHOT
new bddab99719 fix license headers
new f32fd8d6f4 use unified tag family name
new 16325317dc add banyandb plugin to pom
new fa24b07b79 support metadata-v2
new a4a29bf4df support events, profile_task and instance_traffic
new 9a8979a879 add metadata json
new cfa04fdb69 support metadata, segment and topo
new d567b1f143 fix styles
new bd50d4afd0 remove all hardcoded builder
new 05569c0693 make code compiled
new 6afdd8b398 support UI Template
new e078a3bbc0 refactor converter and measure reigstration
new cd13a7372a add measure id and check sharding keys
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b043af38d5)
\
N -- N -- N refs/heads/banyandb-integration-stream (cd13a7372a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 24 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/publish-docker.yaml | 2 +-
.github/workflows/skywalking.yaml | 4 +
Makefile | 4 +-
apm-dist/src/main/assembly/binary.xml | 25 +++
apm-protocol/apm-network/src/main/proto | 2 +-
dist-material/log4j2.xml | 1 -
dist-material/release-docs/LICENSE | 2 +
.../LICENSE-d3-flame-graph} | 26 +--
.../LICENSE-vis-timeline} | 27 +--
docker/{oap => data-generator}/Dockerfile | 0
.../{oap => data-generator}/docker-entrypoint.sh | 8 +-
docker/{oap => data-generator}/log4j2.xml | 0
docs/en/changes/changes-9.0.0.md | 2 +-
docs/en/changes/changes.md | 19 +-
docs/en/concepts-and-designs/mal.md | 4 +-
docs/en/setup/backend/backend-alarm.md | 82 ++++----
docs/en/setup/backend/backend-cluster.md | 18 +-
docs/en/setup/backend/backend-data-generator.md | 207 +++++++++++++++++++++
docs/en/setup/backend/backend-docker.md | 6 +-
docs/en/setup/backend/backend-expose.md | 9 +-
docs/en/setup/backend/backend-health-check.md | 2 +-
docs/en/setup/backend/backend-init-mode.md | 6 +-
docs/en/setup/backend/backend-ip-port.md | 6 +-
docs/en/setup/backend/backend-k8s-monitoring.md | 11 +-
docs/en/setup/backend/backend-k8s.md | 10 +-
docs/en/setup/backend/backend-load-balancer.md | 14 +-
docs/en/setup/backend/backend-meter.md | 4 +-
docs/en/setup/backend/backend-setting-override.md | 6 +-
docs/en/setup/backend/backend-setup.md | 44 +++--
docs/en/setup/backend/backend-start-up-mode.md | 14 +-
docs/en/setup/backend/backend-storage.md | 46 ++---
docs/en/setup/backend/backend-telemetry.md | 20 +-
docs/en/setup/backend/backend-token-auth.md | 12 +-
docs/en/setup/backend/backend-vm-monitoring.md | 6 +-
docs/en/setup/backend/backend-zabbix.md | 2 +-
docs/en/setup/backend/configuration-vocabulary.md | 28 +--
docs/en/setup/backend/dynamic-config-apollo.md | 10 +-
docs/en/setup/backend/dynamic-config-configmap.md | 5 +-
docs/en/setup/backend/dynamic-config-consul.md | 4 +-
docs/en/setup/backend/dynamic-config-etcd.md | 8 +-
docs/en/setup/backend/dynamic-config-nacos.md | 14 +-
docs/en/setup/backend/dynamic-config-service.md | 6 +-
docs/en/setup/backend/dynamic-config-zookeeper.md | 2 +-
docs/en/setup/backend/dynamic-config.md | 8 +-
docs/en/setup/backend/dynamical-logging.md | 8 +-
docs/en/setup/backend/endpoint-grouping-rules.md | 18 +-
docs/en/setup/backend/grpc-security.md | 12 +-
docs/en/setup/backend/kafka-fetcher.md | 15 +-
docs/en/setup/backend/log-analyzer.md | 16 +-
docs/en/setup/backend/logs-template.json | 83 +++++++++
docs/en/setup/backend/metrics-exporter.md | 6 +-
docs/en/setup/backend/mq.md | 6 +-
docs/en/setup/backend/opentelemetry-receiver.md | 2 +-
docs/en/setup/backend/prometheus-metrics.md | 8 +-
docs/en/setup/backend/segment-template.json | 145 +++++++++++++++
docs/en/setup/backend/service-auto-grouping.md | 2 +-
docs/en/setup/backend/slow-db-statement.md | 4 +-
docs/en/setup/backend/trace-sampling.md | 19 +-
docs/en/setup/backend/ui-setup.md | 6 +-
docs/en/setup/backend/uninstrumented-gateways.md | 12 +-
docs/en/setup/backend/zipkin-trace.md | 22 +--
docs/en/setup/envoy/als_setting.md | 10 +-
docs/en/setup/envoy/examples/metrics/README.md | 5 +-
docs/en/setup/envoy/examples/metrics/log4j2.xml | 1 -
docs/en/setup/envoy/metrics_service_setting.md | 14 +-
docs/en/setup/istio/README.md | 14 +-
docs/en/setup/service-agent/agent-compatibility.md | 8 +-
docs/en/setup/service-agent/browser-agent.md | 2 +-
docs/en/setup/service-agent/server-agents.md | 30 ++-
docs/en/setup/service-agent/virtual-database.md | 9 +-
docs/en/ui/README.md | 27 ++-
docs/menu.yml | 6 +-
.../parser/listener/SegmentAnalysisListener.java | 13 ++
.../oap/meter/analyzer/dsl/ArithmeticTest.java | 62 +++---
.../oap/meter/analyzer/dsl/FunctionTest.java | 72 +++----
.../oap/meter/analyzer/dsl/IncreaseTest.java | 30 +--
.../oap/meter/analyzer/dsl/TagFilterTest.java | 86 +++++++--
.../manual/searchtag/TagAutocompleteData.java} | 55 ++++--
.../manual/segment/TraceTagAutocompleteData.java | 55 ++++++
.../TraceTagAutocompleteDispatcher.java} | 20 +-
.../ebpf/EBPFProfilingMutationService.java | 9 +-
.../ebpf/analyze/EBPFProfilingAnalyzer.java | 4 +-
.../oap/server/core/query/TraceQueryService.java | 13 ++
.../server/core/server/HTTPHandlerRegister.java | 4 +-
.../core/server/HTTPHandlerRegisterImpl.java | 6 +-
.../oap/server/core/source/DefaultScopeDefine.java | 1 +
.../skywalking/oap/server/core/source/Segment.java | 2 +
.../core/source/{Log.java => TagAutocomplete.java} | 27 ++-
.../{Source.java => TraceTagAutocomplete.java} | 9 +-
.../server/core/storage/query/ITraceQueryDAO.java | 9 +
.../oap/server/core/version/Version.java | 7 +-
.../oap/server/core/remote/HTTPServerTest.java | 4 +-
.../agent/kafka/KafkaFetcherHandlerRegister.java | 70 ++++---
.../agent/kafka/module/KafkaFetcherConfig.java | 6 +-
.../datacarrier/consumer/BulkConsumePool.java | 4 +-
.../consumer/MultipleChannelsConsumer.java | 112 +++--------
.../oap/server/library/server/http/HTTPServer.java | 15 +-
.../library/server/http/HTTPServerConfig.java | 2 -
.../oap/query/graphql/GraphQLQueryProvider.java | 6 +-
.../oap/query/graphql/resolver/TraceQuery.java | 10 +
.../src/main/resources/query-protocol | 2 +-
.../browser/provider/BrowserModuleProvider.java | 4 +-
.../handler/EBPFProcessServiceHandler.java | 47 ++++-
.../server/receiver/event/EventModuleProvider.java | 6 +-
.../recevier/log/provider/LogModuleProvider.java | 6 +-
.../register/provider/RegisterModuleProvider.java | 6 +-
.../server/ReceiverHTTPHandlerRegister.java | 6 +-
.../sharing/server/SharingServerConfig.java | 2 +-
.../sharing/server/SharingServerModule.java | 4 +-
.../trace/provider/TraceModuleProvider.java | 6 +-
.../receiver/zipkin/ZipkinReceiverConfig.java | 20 +-
.../receiver/zipkin/ZipkinReceiverProvider.java | 20 +-
.../server/receiver/zipkin/trace/SpanForward.java | 19 +-
oap-server/server-starter/pom.xml | 1 -
.../src/main/resources/application.yml | 37 ++--
.../server-starter/src/main/resources/log4j2.xml | 1 -
.../config/ApplicationConfigLoaderTestCase.java | 4 +-
.../elasticsearch/base/StorageEsInstaller.java | 36 ++--
.../elasticsearch/query/TraceQueryEsDAO.java | 70 +++++++
.../storage/plugin/influxdb/InfluxConstants.java | 4 +
.../storage/plugin/influxdb/TableMetaInfo.java | 11 ++
.../storage/plugin/influxdb/query/TraceQuery.java | 70 +++++++
.../server/storage/plugin/iotdb/IoTDBIndexes.java | 3 +-
.../storage/plugin/iotdb/IoTDBTableMetaInfo.java | 3 +
.../plugin/iotdb/query/IoTDBTraceQueryDAO.java | 87 +++++++++
.../plugin/jdbc/h2/dao/H2TraceQueryDAO.java | 70 +++++++
.../elasticsearch/ZipkinTraceQueryEsDAO.java | 73 +++++++-
oap-server/server-tools/data-generator/pom.xml | 55 ++++++
.../data-generator/src/main/assembly/bin/start.sh | 6 +-
.../apache/skywalking/generator/BoolGenerator.java | 54 ++++++
.../generator/FixedStringGenerator.java} | 46 +++--
.../org/apache/skywalking/generator/Generator.java | 44 +++++
.../apache/skywalking/generator/IntGenerator.java | 102 ++++++++++
.../apache/skywalking/generator/ListGenerator.java | 60 ++++++
.../skywalking/generator/SequenceGenerator.java | 118 ++++++++++++
.../skywalking/generator/StringGenerator.java | 90 +++++++++
.../apache/skywalking/generator/UUIDGenerator.java | 70 +++++++
.../skywalking/module/DataGeneratorModule.java} | 24 ++-
.../module/DataGeneratorModuleProvider.java | 80 ++++++++
.../skywalking/restapi/LogGeneratorHandler.java | 133 +++++++++++++
.../org/apache/skywalking/restapi/LogRequest.java | 105 +++++++++++
.../restapi/SegmentGeneratorHandler.java | 134 +++++++++++++
.../apache/skywalking/restapi/SegmentRequest.java | 100 ++++++++++
.../apache/skywalking/restapi/SpanGenerator.java | 85 +++++++++
.../apache/skywalking/restapi/TagGenerator.java} | 30 +--
.../skywalking/starter/DataGeneratorStartUp.java} | 19 +-
...ywalking.oap.server.library.module.ModuleDefine | 3 +-
...alking.oap.server.library.module.ModuleProvider | 2 +-
.../src/main/resources/application.yml | 177 ++++++++++++++++++
.../skywalking/generator/IntGeneratorTest.java} | 27 ++-
.../generator/SequenceGeneratorTest.java | 55 ++++++
oap-server/server-tools/pom.xml | 1 +
.../profile/exporter/test/ProfileTraceDAO.java | 17 ++
.../profile/core/mock/MockHTTPHandlerRegister.java | 5 +-
pom.xml | 2 +-
skywalking-ui | 2 +-
test/e2e-v2/cases/istio/log4j2.xml | 1 -
.../e2e-v2/cases/profiling/ebpf/docker-compose.yml | 13 +-
test/e2e-v2/cases/rover/process/istio/e2e.yaml | 105 +++++++++++
.../process/istio}/expected/process.yml | 25 ++-
.../process/istio}/expected/service-instance.yml | 0
.../process/istio}/expected/service.yml | 52 +++---
.../cases/{ => rover/process}/istio/kind.yaml | 0
test/e2e-v2/cases/rover/process/istio/rover.yaml | 92 +++++++++
test/e2e-v2/script/env | 2 +-
test/e2e-v2/script/prepare/setup-oap/log4j2.xml | 1 -
166 files changed, 3628 insertions(+), 886 deletions(-)
copy dist-material/release-docs/licenses/{LICENSE-proto-google-common-protos.txt => ui-licenses/LICENSE-d3-flame-graph} (89%)
copy dist-material/release-docs/licenses/{LICENSE-proto-google-common-protos.txt => ui-licenses/LICENSE-vis-timeline} (89%)
copy docker/{oap => data-generator}/Dockerfile (100%)
copy docker/{oap => data-generator}/docker-entrypoint.sh (82%)
copy docker/{oap => data-generator}/log4j2.xml (100%)
create mode 100644 docs/en/setup/backend/backend-data-generator.md
create mode 100644 docs/en/setup/backend/logs-template.json
create mode 100644 docs/en/setup/backend/segment-template.json
copy oap-server/{exporter/src/test/java/org/apache/skywalking/oap/server/exporter/provider/grpc/MockMetrics.java => server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/searchtag/TagAutocompleteData.java} (50%)
create mode 100644 oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/segment/TraceTagAutocompleteData.java
copy oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/{service/ServiceMetaDispatcher.java => segment/TraceTagAutocompleteDispatcher.java} (64%)
copy oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/{Log.java => TagAutocomplete.java} (77%)
copy oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/{Source.java => TraceTagAutocomplete.java} (84%)
create mode 100644 oap-server/server-tools/data-generator/pom.xml
copy dist-material/bin/oapService.sh => oap-server/server-tools/data-generator/src/main/assembly/bin/start.sh (86%)
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/BoolGenerator.java
copy oap-server/{server-library/library-elasticsearch-client/src/main/java/org/apache/skywalking/library/elasticsearch/requests/search/Scroll.java => server-tools/data-generator/src/main/java/org/apache/skywalking/generator/FixedStringGenerator.java} (53%)
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/Generator.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/IntGenerator.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/ListGenerator.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/SequenceGenerator.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/StringGenerator.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator/UUIDGenerator.java
copy oap-server/{server-library/library-elasticsearch-client/src/main/java/org/apache/skywalking/library/elasticsearch/requests/search/Scroll.java => server-tools/data-generator/src/main/java/org/apache/skywalking/module/DataGeneratorModule.java} (63%)
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/module/DataGeneratorModuleProvider.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/LogGeneratorHandler.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/LogRequest.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentRequest.java
create mode 100644 oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SpanGenerator.java
copy oap-server/{server-library/library-elasticsearch-client/src/main/java/org/apache/skywalking/library/elasticsearch/requests/search/Scroll.java => server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/TagGenerator.java} (58%)
copy oap-server/{server-library/library-elasticsearch-client/src/main/java/org/apache/skywalking/library/elasticsearch/requests/search/Scroll.java => server-tools/data-generator/src/main/java/org/apache/skywalking/starter/DataGeneratorStartUp.java} (62%)
copy lombok.config => oap-server/server-tools/data-generator/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine (93%)
copy oap-server/{server-core => server-tools/data-generator}/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider (93%)
mode change 100755 => 100644
create mode 100755 oap-server/server-tools/data-generator/src/main/resources/application.yml
copy oap-server/{server-library/library-elasticsearch-client/src/main/java/org/apache/skywalking/library/elasticsearch/requests/search/Scroll.java => server-tools/data-generator/src/test/java/org/apache/skywalking/generator/IntGeneratorTest.java} (62%)
create mode 100644 oap-server/server-tools/data-generator/src/test/java/org/apache/skywalking/generator/SequenceGeneratorTest.java
create mode 100644 test/e2e-v2/cases/rover/process/istio/e2e.yaml
copy test/e2e-v2/cases/{profiling/ebpf => rover/process/istio}/expected/process.yml (75%)
copy test/e2e-v2/cases/{istio/metrics => rover/process/istio}/expected/service-instance.yml (100%)
copy test/e2e-v2/cases/{istio/metrics => rover/process/istio}/expected/service.yml (66%)
copy test/e2e-v2/cases/{ => rover/process}/istio/kind.yaml (100%)
create mode 100644 test/e2e-v2/cases/rover/process/istio/rover.yaml
[skywalking] 01/24: start from new code base and copy all previous stream impl
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 896223ab37b634dca2676ebafc13c795cc836dfa
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Sun Nov 28 23:08:31 2021 +0800
start from new code base and copy all previous stream impl
---
oap-server/server-storage-plugin/pom.xml | 1 +
.../{ => storage-banyandb-plugin}/pom.xml | 40 +++--
.../storage/plugin/banyandb/BanyanDBSchema.java | 66 ++++++++
.../plugin/banyandb/BanyanDBStorageClient.java | 55 ++++++
.../plugin/banyandb/BanyanDBStorageConfig.java | 47 ++++++
.../plugin/banyandb/BanyanDBStorageProvider.java | 89 ++++++++++
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 19 +++
.../plugin/banyandb/stream/BanyanDBBatchDAO.java | 75 +++++++++
.../stream/BanyanDBBrowserLogQueryDAO.java | 17 ++
.../banyandb/stream/BanyanDBEventQueryDAO.java | 23 +++
.../banyandb/stream/BanyanDBLogQueryDAO.java | 20 +++
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 55 ++++++
.../stream/BanyanDBNetworkAddressAliasDAO.java | 17 ++
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 18 ++
.../stream/BanyanDBProfileTaskQueryDAO.java | 23 +++
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 40 +++++
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 79 +++++++++
.../stream/BanyanDBSegmentRecordBuilder.java | 58 +++++++
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 85 ++++++++++
.../stream/BanyanDBStreamInsertRequest.java | 31 ++++
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 186 +++++++++++++++++++++
.../stream/BanyanDBUITemplateManagementDAO.java | 35 ++++
...alking.oap.server.library.module.ModuleProvider | 19 +++
23 files changed, 1086 insertions(+), 12 deletions(-)
diff --git a/oap-server/server-storage-plugin/pom.xml b/oap-server/server-storage-plugin/pom.xml
index fc7f896467..82e3353f4f 100644
--- a/oap-server/server-storage-plugin/pom.xml
+++ b/oap-server/server-storage-plugin/pom.xml
@@ -34,5 +34,6 @@
<module>storage-influxdb-plugin</module>
<module>storage-tidb-plugin</module>
<module>storage-iotdb-plugin</module>
+ <module>storage-banyandb-plugin</module>
</modules>
</project>
diff --git a/oap-server/server-storage-plugin/pom.xml b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
similarity index 53%
copy from oap-server/server-storage-plugin/pom.xml
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
index fc7f896467..76848c1b4b 100644
--- a/oap-server/server-storage-plugin/pom.xml
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
@@ -19,20 +19,36 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
- <artifactId>oap-server</artifactId>
+ <artifactId>server-storage-plugin</artifactId>
<groupId>org.apache.skywalking</groupId>
- <version>9.1.0-SNAPSHOT</version>
+ <version>8.9.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
- <artifactId>server-storage-plugin</artifactId>
- <packaging>pom</packaging>
- <modules>
- <module>storage-jdbc-hikaricp-plugin</module>
- <module>storage-elasticsearch-plugin</module>
- <module>storage-zipkin-elasticsearch-plugin</module>
- <module>storage-influxdb-plugin</module>
- <module>storage-tidb-plugin</module>
- <module>storage-iotdb-plugin</module>
- </modules>
+ <artifactId>storage-banyandb-plugin</artifactId>
+ <packaging>jar</packaging>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>server-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-client</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>library-datacarrier-queue</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>banyandb-java-client</artifactId>
+ <version>0.1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
</project>
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java
new file mode 100644
index 0000000000..3ebf87776b
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import lombok.Getter;
+
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+
+public class BanyanDBSchema {
+ public static final String NAME = "sw";
+ public static final String GROUP = "default";
+ public static final List<String> FIELD_NAMES;
+
+ public static final Set<String> INDEX_FIELDS = ImmutableSet.of("http.method", "status_code", "db.type",
+ "db.instance", "mq.queue", "mq.topic", "mq.broker");
+
+ static {
+ Set<String> fields = new LinkedHashSet<>();
+ fields.add("trace_id");
+ fields.add("state");
+ fields.add("service_id");
+ fields.add("service_instance_id");
+ fields.add("endpoint_id");
+ fields.add("duration");
+ fields.add("start_time");
+ fields.add("http.method");
+ fields.add("status_code");
+ fields.add("db.type");
+ fields.add("db.instance");
+ fields.add("mq.queue");
+ fields.add("mq.topic");
+ fields.add("mq.broker");
+ FIELD_NAMES = ImmutableList.copyOf(fields);
+ }
+
+ public enum TraceState {
+ ALL(0), SUCCESS(1), ERROR(2);
+
+ @Getter
+ private final int state;
+
+ TraceState(int state) {
+ this.state = state;
+ }
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
new file mode 100644
index 0000000000..eed4386e57
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -0,0 +1,55 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
+import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.library.client.Client;
+import org.apache.skywalking.oap.server.library.client.healthcheck.DelegatedHealthChecker;
+import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckable;
+import org.apache.skywalking.oap.server.library.util.HealthChecker;
+
+import java.io.IOException;
+
+/**
+ * BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
+ * which implement {@link Client} and {@link HealthCheckable}.
+ */
+public class BanyanDBStorageClient implements Client, HealthCheckable {
+ private final BanyanDBClient client;
+ private final DelegatedHealthChecker healthChecker = new DelegatedHealthChecker();
+
+ public BanyanDBStorageClient(String host, int port, String group) {
+ this.client = new BanyanDBClient(host, port, group);
+ }
+
+ @Override
+ public void connect() throws Exception {
+ this.client.connect();
+ }
+
+ @Override
+ public void shutdown() throws IOException {
+ this.client.close();
+ }
+
+ public StreamQueryResponse query(StreamQuery streamQuery) {
+ try {
+ StreamQueryResponse response = this.client.queryStreams(streamQuery);
+ this.healthChecker.health();
+ return response;
+ } catch (Throwable t) {
+ healthChecker.unHealth(t);
+ throw t;
+ }
+ }
+
+ public StreamBulkWriteProcessor createBulkProcessor(int maxBulkSize, int flushInterval, int concurrency) {
+ return this.client.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
+ }
+
+ @Override
+ public void registerChecker(HealthChecker healthChecker) {
+ this.healthChecker.register(healthChecker);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
new file mode 100644
index 0000000000..c37aed1162
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import lombok.Getter;
+import lombok.Setter;
+import org.apache.skywalking.oap.server.library.module.ModuleConfig;
+
+@Getter
+@Setter
+public class BanyanDBStorageConfig extends ModuleConfig {
+ private String host = "127.0.0.1";
+ private int port = 17912;
+ /**
+ * Group of the schema in BanyanDB
+ */
+ private String group = "default";
+
+ /**
+ * The maximum size of write entities in a single batch write call.
+ */
+ private int maxBulkSize = 5000;
+ /**
+ * Period of flush interval. In the timeunit of seconds.
+ */
+ private int flushInterval = 15;
+ /**
+ * Concurrent consumer threads for batch writing.
+ */
+ private int concurrentWriteThreads = 2;
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
new file mode 100644
index 0000000000..c4aa1016ca
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.config.ConfigService;
+import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
+import org.apache.skywalking.oap.server.core.storage.StorageModule;
+import org.apache.skywalking.oap.server.library.module.*;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+public class BanyanDBStorageProvider extends ModuleProvider {
+ private BanyanDBStorageConfig config;
+ private BanyanDBStorageClient client;
+
+ public BanyanDBStorageProvider() {
+ this.config = new BanyanDBStorageConfig();
+ }
+
+ @Override
+ public String name() {
+ return "banyandb";
+ }
+
+ @Override
+ public Class<? extends ModuleDefine> module() {
+ return StorageModule.class;
+ }
+
+ @Override
+ public ModuleConfig createConfigBeanIfAbsent() {
+ return config;
+ }
+
+ @Override
+ public void prepare() throws ServiceNotProvidedException, ModuleStartException {
+ this.registerServiceImplementation(StorageBuilderFactory.class, new StorageBuilderFactory.Default());
+
+ this.client = new BanyanDBStorageClient(config.getHost(), config.getPort(), config.getGroup());
+ }
+
+ @Override
+ public void start() throws ServiceNotProvidedException, ModuleStartException {
+ final ConfigService configService = getManager().find(CoreModule.NAME)
+ .provider()
+ .getService(ConfigService.class);
+
+ MetricsCreator metricCreator = getManager().find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+ HealthCheckMetrics healthChecker = metricCreator.createHealthCheckerGauge(
+ "storage_banyandb", MetricsTag.EMPTY_KEY, MetricsTag.EMPTY_VALUE);
+ this.client.registerChecker(healthChecker);
+ try {
+ this.client.connect();
+ } catch (Exception e) {
+ throw new ModuleStartException(e.getMessage(), e);
+ }
+ }
+
+ @Override
+ public void notifyAfterCompleted() throws ServiceNotProvidedException, ModuleStartException {
+
+ }
+
+ @Override
+ public String[] requiredModules() {
+ return new String[]{CoreModule.NAME};
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
new file mode 100644
index 0000000000..c7bc9cc13d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -0,0 +1,19 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.type.Alarms;
+import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+
+import java.io.IOException;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.alarm.AlarmRecord} is a stream,
+ * which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
+ */
+public class BanyanDBAlarmQueryDAO implements IAlarmQueryDAO {
+ @Override
+ public Alarms getAlarm(Integer scopeId, String keyword, int limit, int from, long startTB, long endTB, List<Tag> tags) throws IOException {
+ return new Alarms();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
new file mode 100644
index 0000000000..9404996887
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
+import org.apache.skywalking.oap.server.core.storage.IBatchDAO;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.PrepareRequest;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+
+public class BanyanDBBatchDAO extends AbstractDAO<BanyanDBStorageClient> implements IBatchDAO {
+ private StreamBulkWriteProcessor bulkProcessor;
+
+ private final int maxBulkSize;
+ private final int flushInterval;
+ private final int concurrency;
+
+ public BanyanDBBatchDAO(BanyanDBStorageClient client, int maxBulkSize, int flushInterval, int concurrency) {
+ super(client);
+ this.maxBulkSize = maxBulkSize;
+ this.flushInterval = flushInterval;
+ this.concurrency = concurrency;
+ }
+
+ @Override
+ public void insert(InsertRequest insertRequest) {
+ if (bulkProcessor == null) {
+ this.bulkProcessor = getClient().createBulkProcessor(maxBulkSize, flushInterval, concurrency);
+ }
+
+ if (insertRequest instanceof BanyanDBStreamInsertRequest) {
+ this.bulkProcessor.add(((BanyanDBStreamInsertRequest) insertRequest).getStreamWrite());
+ }
+ }
+
+ @Override
+ public CompletableFuture<Void> flush(List<PrepareRequest> prepareRequests) {
+ if (bulkProcessor == null) {
+ this.bulkProcessor = getClient().createBulkProcessor(maxBulkSize, flushInterval, concurrency);
+ }
+
+ if (CollectionUtils.isNotEmpty(prepareRequests)) {
+ return CompletableFuture.allOf(prepareRequests.stream().map(prepareRequest -> {
+ if (prepareRequest instanceof InsertRequest) {
+ // TODO: return CompletableFuture<Void>
+ this.bulkProcessor.add(((BanyanDBStreamInsertRequest) prepareRequest).getStreamWrite());
+ }
+ return CompletableFuture.completedFuture(null);
+ }).toArray(CompletableFuture[]::new));
+ }
+
+ return CompletableFuture.completedFuture(null);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
new file mode 100644
index 0000000000..7607668eba
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -0,0 +1,17 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.browser.source.BrowserErrorCategory;
+import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLogs;
+import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
+
+import java.io.IOException;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
+ */
+public class BanyanDBBrowserLogQueryDAO implements IBrowserLogQueryDAO {
+ @Override
+ public BrowserErrorLogs queryBrowserErrorLogs(String serviceId, String serviceVersionId, String pagePathId, BrowserErrorCategory category, long startSecondTB, long endSecondTB, int limit, int from) throws IOException {
+ return new BrowserErrorLogs();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
new file mode 100644
index 0000000000..5ab67d95c6
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
@@ -0,0 +1,23 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.query.type.event.EventQueryCondition;
+import org.apache.skywalking.oap.server.core.query.type.event.Events;
+import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
+
+import java.util.List;
+
+/**
+ * ???
+ * {@link org.apache.skywalking.oap.server.core.source.Event} is a stream
+ */
+public class BanyanDBEventQueryDAO implements IEventQueryDAO {
+ @Override
+ public Events queryEvents(EventQueryCondition condition) throws Exception {
+ return new Events();
+ }
+
+ @Override
+ public Events queryEvents(List<EventQueryCondition> conditionList) throws Exception {
+ return new Events();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
new file mode 100644
index 0000000000..baba93b886
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -0,0 +1,20 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.enumeration.Order;
+import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
+
+import java.io.IOException;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
+ */
+public class BanyanDBLogQueryDAO implements ILogQueryDAO {
+ @Override
+ public Logs queryLogs(String serviceId, String serviceInstanceId, String endpointId, TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit, long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent, List<String> excludingKeywordsOfContent) throws IOException {
+ return new Logs();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
new file mode 100644
index 0000000000..678b981f16
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -0,0 +1,55 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.analysis.NodeType;
+import org.apache.skywalking.oap.server.core.query.type.Database;
+import org.apache.skywalking.oap.server.core.query.type.Endpoint;
+import org.apache.skywalking.oap.server.core.query.type.Service;
+import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic},
+ * {@link org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic}
+ * {@link org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic}
+ * are all streams.
+ */
+public class BanyanDBMetadataQueryDAO implements IMetadataQueryDAO {
+ @Override
+ public List<Service> getAllServices(String group) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Service> getAllBrowserServices() throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Database> getAllDatabases() throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Service> searchServices(NodeType nodeType, String keyword) throws IOException {
+ return null;
+ }
+
+ @Override
+ public Service searchService(NodeType nodeType, String serviceCode) throws IOException {
+ return null;
+ }
+
+ @Override
+ public List<Endpoint> searchEndpoint(String keyword, String serviceId, int limit) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<ServiceInstance> getServiceInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
new file mode 100644
index 0000000000..10675a49a1
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -0,0 +1,17 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
+
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link NetworkAddressAlias} is a stream
+ */
+public class BanyanDBNetworkAddressAliasDAO implements INetworkAddressAliasDAO {
+ @Override
+ public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
new file mode 100644
index 0000000000..2b6290e2d2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -0,0 +1,18 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
+ */
+public class BanyanDBProfileTaskLogQueryDAO implements IProfileTaskLogQueryDAO {
+ @Override
+ public List<ProfileTaskLog> getTaskLogList() throws IOException {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
new file mode 100644
index 0000000000..c99d267cbe
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -0,0 +1,23 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
+ */
+public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+ @Override
+ public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ProfileTask getById(String id) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
new file mode 100644
index 0000000000..8a27e885fe
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -0,0 +1,40 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link ProfileThreadSnapshotRecord} is a stream
+ */
+public class BanyanDBProfileThreadSnapshotQueryDAO implements IProfileThreadSnapshotQueryDAO {
+ @Override
+ public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public int queryMinSequence(String segmentId, long start, long end) throws IOException {
+ return 0;
+ }
+
+ @Override
+ public int queryMaxSequence(String segmentId, long start, long end) throws IOException {
+ return 0;
+ }
+
+ @Override
+ public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
+ return null;
+ }
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
new file mode 100644
index 0000000000..f59bc25f9e
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
+import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+@RequiredArgsConstructor
+public class BanyanDBRecordDAO implements IRecordDAO {
+ private final StorageHashMapBuilder<Record> storageBuilder;
+
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
+ if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
+ SegmentRecord segmentRecord = (SegmentRecord) record;
+ StreamWrite streamWrite = StreamWrite.builder()
+ .name(BanyanDBSchema.NAME)
+ .binary(segmentRecord.getDataBinary())
+ .timestamp(segmentRecord.getStartTime())
+ .elementId(segmentRecord.getSegmentId())
+ .tags(buildFieldObjects(this.storageBuilder.entity2Storage(segmentRecord)))
+ .build();
+ return new BanyanDBStreamInsertRequest(streamWrite);
+ }
+ // TODO: support other stream types
+ return new InsertRequest() {
+ };
+ }
+
+ /**
+ * Convert storageEntity in Map to an ordered list of {@link SerializableTag}
+ *
+ * @param segmentRecordMap which comes from {@link SegmentRecord}
+ * @return an ordered list of {@link SerializableTag} which is accepted by BanyanDB Client
+ */
+ static List<SerializableTag<Banyandb.TagValue>> buildFieldObjects(Map<String, Object> segmentRecordMap) {
+ List<SerializableTag<Banyandb.TagValue>> tagList = new ArrayList<>(BanyanDBSchema.FIELD_NAMES.size());
+ for (String fieldName : BanyanDBSchema.FIELD_NAMES) {
+ Object val = segmentRecordMap.get(fieldName);
+ if (val == null) {
+ tagList.add(Tag.nullField());
+ } else {
+ tagList.add((SerializableTag<Banyandb.TagValue>) val);
+ }
+ }
+ return tagList;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java
new file mode 100644
index 0000000000..b8c4744380
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class BanyanDBSegmentRecordBuilder implements StorageHashMapBuilder<Record> {
+ @Override
+ public SegmentRecord storage2Entity(Map<String, Object> dbMap) {
+ return new SegmentRecord();
+ }
+
+ /**
+ * Map SegmentRecord to Skywalking-BanyanDB compatible Map with indexed tags and
+ * without binaryData, entityId
+ */
+ @Override
+ public Map<String, Object> entity2Storage(Record record) {
+ final SegmentRecord segmentRecord = (SegmentRecord) record;
+ Map<String, Object> map = new HashMap<>();
+ map.put(SegmentRecord.TRACE_ID, TagAndValue.stringField(segmentRecord.getTraceId()));
+ map.put(SegmentRecord.SERVICE_ID, TagAndValue.stringField(segmentRecord.getServiceId()));
+ map.put(SegmentRecord.SERVICE_INSTANCE_ID, TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
+ map.put(SegmentRecord.ENDPOINT_ID, TagAndValue.stringField(segmentRecord.getEndpointId()));
+ map.put(SegmentRecord.START_TIME, TagAndValue.longField(segmentRecord.getStartTime()));
+ map.put("duration", TagAndValue.longField(segmentRecord.getLatency()));
+ map.put("state", TagAndValue.longField(segmentRecord.getIsError()));
+ if (segmentRecord.getTagsRawData() != null) {
+ for (final Tag tag : segmentRecord.getTagsRawData()) {
+ map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
+ }
+ }
+ return map;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
new file mode 100644
index 0000000000..f5ee5fb255
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.storage.*;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+@Slf4j
+public class BanyanDBStorageDAO implements StorageDAO {
+ @Override
+ public IMetricsDAO newMetricsDao(StorageBuilder storageBuilder) {
+ return new IMetricsDAO() {
+ @Override
+ public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
+ return new InsertRequest() {
+ };
+ }
+
+ @Override
+ public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
+ return new UpdateRequest() {
+ };
+ }
+ };
+ }
+
+ @Override
+ public IRecordDAO newRecordDao(StorageBuilder storageBuilder) {
+ try {
+ if (SegmentRecord.class.equals(storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType())) {
+ return new BanyanDBRecordDAO(new BanyanDBSegmentRecordBuilder());
+ } else {
+ return (model, record) -> new InsertRequest() {
+ };
+ }
+ } catch (NoSuchMethodException noSuchMethodException) {
+ log.error("cannot find method storage2Entity", noSuchMethodException);
+ throw new RuntimeException("cannot find method storage2Entity");
+ }
+ }
+
+ @Override
+ public INoneStreamDAO newNoneStreamDao(StorageBuilder storageBuilder) {
+ return (model, noneStream) -> {
+ };
+ }
+
+ @Override
+ public IManagementDAO newManagementDao(StorageBuilder storageBuilder) {
+ return (model, storageData) -> {
+ };
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
new file mode 100644
index 0000000000..aef8ec94e7
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+
+@RequiredArgsConstructor
+public class BanyanDBStreamInsertRequest implements InsertRequest {
+
+ @Getter
+ private final StreamWrite streamWrite;
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
new file mode 100644
index 0000000000..9fc403afc6
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.*;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.type.*;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.joda.time.DateTimeZone;
+import org.joda.time.format.DateTimeFormat;
+import org.joda.time.format.DateTimeFormatter;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.stream.Collectors;
+
+public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements ITraceQueryDAO {
+ private static final DateTimeFormatter YYYYMMDDHHMMSS = DateTimeFormat.forPattern("yyyyMMddHHmmss");
+
+ private static final List<String> BASIC_QUERY_PROJ = ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time");
+ private static final List<String> TRACE_ID_QUERY_PROJ = ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time");
+
+ public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
+ @Override
+ public TraceBrief queryBasicTraces(long startSecondTB, long endSecondTB, long minDuration, long maxDuration, String serviceId, String serviceInstanceId, String endpointId, String traceId, int limit, int from, TraceState traceState, QueryOrder queryOrder, List<Tag> tags) throws IOException {
+ StreamQuery query;
+ if (startSecondTB != 0 && endSecondTB != 0) {
+ query = new StreamQuery(BanyanDBSchema.NAME, new TimestampRange(parseMillisFromStartSecondTB(startSecondTB), parseMillisFromEndSecondTB(endSecondTB)), BASIC_QUERY_PROJ);
+ } else {
+ query = new StreamQuery(BanyanDBSchema.NAME, BASIC_QUERY_PROJ);
+ }
+ if (minDuration != 0) {
+ // duration >= minDuration
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", "duration", minDuration));
+ }
+ if (maxDuration != 0) {
+ // duration <= maxDuration
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", "duration", maxDuration));
+ }
+
+ if (!Strings.isNullOrEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_id", serviceId));
+ }
+
+ if (!Strings.isNullOrEmpty(serviceInstanceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_instance_id", serviceInstanceId));
+ }
+
+ if (!Strings.isNullOrEmpty(endpointId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "endpoint_id", endpointId));
+ }
+
+ switch (traceState) {
+ case ERROR:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ERROR.getState()));
+ break;
+ case SUCCESS:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.SUCCESS.getState()));
+ break;
+ default:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ALL.getState()));
+ break;
+ }
+
+ switch (queryOrder) {
+ case BY_START_TIME:
+ query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
+ break;
+ case BY_DURATION:
+ query.setOrderBy(new StreamQuery.OrderBy("duration", StreamQuery.OrderBy.Type.DESC));
+ break;
+ }
+
+ if (CollectionUtils.isNotEmpty(tags)) {
+ for (final Tag tag : tags) {
+ if (BanyanDBSchema.INDEX_FIELDS.contains(tag.getKey())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ }
+ }
+ }
+
+ query.setLimit(limit);
+ query.setOffset(from);
+
+ // build request
+ StreamQueryResponse response = this.getClient().query(query);
+ TraceBrief brief = new TraceBrief();
+ brief.setTotal(response.size());
+ brief.getTraces().addAll(response.getElements().stream().map(elem -> {
+ BasicTrace trace = new BasicTrace();
+ trace.setSegmentId(elem.getId());
+ final List<TagAndValue<?>> searchable = elem.getTagFamilies().get(0);
+ trace.getTraceIds().add((String) searchable.get(0).getValue());
+ trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
+ trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ (String) searchable.get(2).getValue()
+ ).getEndpointName());
+ trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
+ trace.setStart(String.valueOf(searchable.get(4).getValue()));
+ return trace;
+ }).collect(Collectors.toList()));
+ return brief;
+ }
+
+ @Override
+ public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
+ StreamQuery query = new StreamQuery(BanyanDBSchema.NAME, TRACE_ID_QUERY_PROJ);
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
+ query.setDataBinary(true);
+ StreamQueryResponse response = this.getClient().query(query);
+ return response.getElements().stream().map(elem -> {
+ SegmentRecord record = new SegmentRecord();
+ final List<TagAndValue<?>> searchable = elem.getTagFamilies().get(0);
+ record.setSegmentId(elem.getId());
+ record.setTraceId((String) searchable.get(0).getValue());
+ record.setIsError(((Number) searchable.get(1).getValue()).intValue());
+ record.setServiceId((String) searchable.get(2).getValue());
+ record.setServiceInstanceId((String) searchable.get(3).getValue());
+ record.setEndpointId((String) searchable.get(4).getValue());
+ record.setLatency(((Number) searchable.get(5).getValue()).intValue());
+ record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ final List<TagAndValue<?>> data = elem.getTagFamilies().get(1);
+ // TODO: support binary data in the client SDK
+ record.setDataBinary((byte[]) data.get(0).getValue());
+ return record;
+ }).collect(Collectors.toList());
+ }
+
+ @Override
+ public List<Span> doFlexibleTraceQuery(String traceId) throws IOException {
+ return Collections.emptyList();
+ }
+
+ static long parseMillisFromStartSecondTB(long startSecondTB) {
+ return YYYYMMDDHHMMSS.withZone(DateTimeZone.UTC).parseMillis(String.valueOf(startSecondTB));
+ }
+
+ static long parseMillisFromEndSecondTB(long endSecondTB) {
+ long t = endSecondTB;
+ long second = t % 100;
+ if (second > 59) {
+ second = 0;
+ }
+ t = t / 100;
+ long minute = t % 100;
+ if (minute > 59) {
+ minute = 0;
+ }
+ t = t / 100;
+ long hour = t % 100;
+ if (hour > 23) {
+ hour = 0;
+ }
+ t = t / 100;
+ return YYYYMMDDHHMMSS.withZone(DateTimeZone.UTC)
+ .parseMillis(String.valueOf(((t * 100 + hour) * 100 + minute) * 100 + second));
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
new file mode 100644
index 0000000000..ad8fed926e
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -0,0 +1,35 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
+import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
+import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
+import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
+ */
+public class BanyanDBUITemplateManagementDAO implements UITemplateManagementDAO {
+ @Override
+ public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
+ return TemplateChangeStatus.builder().status(false).message("Can't add a new template").build();
+ }
+
+ @Override
+ public TemplateChangeStatus changeTemplate(DashboardSetting setting) throws IOException {
+ return TemplateChangeStatus.builder().status(false).message("Can't add/update the template").build();
+ }
+
+ @Override
+ public TemplateChangeStatus disableTemplate(String name) throws IOException {
+ return TemplateChangeStatus.builder().status(false).message("Can't add/update the template").build();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
new file mode 100644
index 0000000000..6b9674593d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider
@@ -0,0 +1,19 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+
+org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageProvider
[skywalking] 04/24: add alarm, networkAddressAlias and logs
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit cb705ed0cc8d014839f2f13078442307b6e2b9df
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Wed Dec 1 23:29:21 2021 +0800
add alarm, networkAddressAlias and logs
---
.../banyandb/converter/ProfileTaskMapper.java | 28 ---------
.../ProfileThreadSnapshotRecordMapper.java | 27 --------
.../banyandb/deserializer/AlarmMessageMapper.java | 51 +++++++++++++++
.../BasicTraceMapper.java | 2 +-
.../deserializer/BrowserErrorLogMapper.java | 59 ++++++++++++++++++
.../DashboardConfigurationMapper.java | 2 +-
.../plugin/banyandb/deserializer/EventMapper.java | 24 ++++++++
.../plugin/banyandb/deserializer/LogMapper.java | 61 ++++++++++++++++++
.../deserializer/NetworkAddressAliasMapper.java | 40 ++++++++++++
.../deserializer/ProfileTaskLogMapper.java | 39 ++++++++++++
.../banyandb/deserializer/ProfileTaskMapper.java | 42 +++++++++++++
.../ProfileThreadSnapshotRecordMapper.java | 36 +++++++++++
.../RowEntityMapper.java | 2 +-
.../SegmentRecordMapper.java | 2 +-
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 47 +++++++++++++-
.../stream/BanyanDBBrowserLogQueryDAO.java | 50 ++++++++++++++-
.../banyandb/stream/BanyanDBLogQueryDAO.java | 72 +++++++++++++++++++++-
.../stream/BanyanDBNetworkAddressAliasDAO.java | 23 ++++++-
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 31 +++++++++-
.../stream/BanyanDBProfileTaskQueryDAO.java | 32 +++++++---
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 8 +--
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 6 +-
.../stream/BanyanDBUITemplateManagementDAO.java | 4 +-
23 files changed, 597 insertions(+), 91 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java
deleted file mode 100644
index ea1ca09cb0..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java
+++ /dev/null
@@ -1,28 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-
-import java.util.Collections;
-import java.util.List;
-
-public class ProfileTaskMapper implements RowEntityMapper<ProfileTask> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME, ProfileTaskRecord.START_TIME,
- ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD, ProfileTaskRecord.DUMP_PERIOD,
- ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT);
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.emptyList();
- }
-
- @Override
- public ProfileTask map(RowEntity row) {
- return null;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java
deleted file mode 100644
index 9a25688132..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java
+++ /dev/null
@@ -1,27 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-
-import java.util.Collections;
-import java.util.List;
-
-public class ProfileThreadSnapshotRecordMapper implements RowEntityMapper<ProfileThreadSnapshotRecord> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
- ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
- ProfileThreadSnapshotRecord.STACK_BINARY);
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.emptyList();
- }
-
- @Override
- public ProfileThreadSnapshotRecord map(RowEntity row) {
- return null;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
new file mode 100644
index 0000000000..d906dbb78f
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
@@ -0,0 +1,51 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.query.enumeration.Scope;
+import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
+import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+
+import java.util.List;
+
+@RequiredArgsConstructor
+public class AlarmMessageMapper implements RowEntityMapper<AlarmMessage> {
+ private final IAlarmQueryDAO alarmQueryDAO;
+
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(AlarmRecord.SCOPE, // 0
+ AlarmRecord.START_TIME); // 1
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return ImmutableList.of(AlarmRecord.ID0, // 0
+ AlarmRecord.ID1, // 1
+ AlarmRecord.ALARM_MESSAGE, // 2
+ AlarmRecord.TAGS_RAW_DATA); // 3
+ }
+
+ @Override
+ public AlarmMessage map(RowEntity row) {
+ AlarmMessage alarmMessage = new AlarmMessage();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ int scopeID = ((Number) searchable.get(0).getValue()).intValue();
+ alarmMessage.setScopeId(scopeID);
+ alarmMessage.setScope(Scope.Finder.valueOf(scopeID));
+ alarmMessage.setStartTime(((Number) searchable.get(1).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ alarmMessage.setId((String) data.get(0).getValue());
+ alarmMessage.setId1((String) data.get(1).getValue());
+ alarmMessage.setMessage((String) data.get(2).getValue());
+ Object o = data.get(3).getValue();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ this.alarmQueryDAO.parserDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
+ }
+ return alarmMessage;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
similarity index 98%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
index b3512158f2..bbcf7091d0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
@@ -1,4 +1,4 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
new file mode 100644
index 0000000000..0807955376
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
@@ -0,0 +1,59 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
+import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
+import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
+
+import java.util.List;
+
+public class BrowserErrorLogMapper implements RowEntityMapper<BrowserErrorLog> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
+ BrowserErrorLogRecord.SERVICE_VERSION_ID,
+ BrowserErrorLogRecord.PAGE_PATH_ID,
+ BrowserErrorLogRecord.ERROR_CATEGORY,
+ BrowserErrorLogRecord.TIMESTAMP
+ );
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return ImmutableList.of(BrowserErrorLogRecord.DATA_BINARY);
+ }
+
+ @Override
+ public BrowserErrorLog map(RowEntity row) {
+ // FIXME: use protobuf directly
+ BrowserErrorLog log = new BrowserErrorLog();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ log.setService((String) searchable.get(0).getValue());
+ log.setServiceVersion((String) searchable.get(1).getValue());
+ log.setPagePath((String) searchable.get(2).getValue());
+ log.setCategory(ErrorCategory.valueOf((String) searchable.get(3).getValue()));
+ log.setTime(((Number) searchable.get(4).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ Object o = data.get(0).getValue();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ try {
+ org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog browserErrorLog = org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog
+ .parseFrom((ByteString) o);
+ log.setGrade(browserErrorLog.getGrade());
+ log.setCol(browserErrorLog.getCol());
+ log.setLine(browserErrorLog.getLine());
+ log.setMessage(browserErrorLog.getMessage());
+ log.setErrorUrl(browserErrorLog.getErrorUrl());
+ log.setStack(browserErrorLog.getStack());
+ log.setFirstReportedError(browserErrorLog.getFirstReportedError());
+ } catch (InvalidProtocolBufferException ex) {
+ throw new RuntimeException("fail to parse proto buffer", ex);
+ }
+ }
+ return log;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
similarity index 99%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
index 3f7df9cb72..6b9e0962e6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
@@ -1,4 +1,4 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
new file mode 100644
index 0000000000..1c94903cbd
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
@@ -0,0 +1,24 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.oap.server.core.query.type.event.Event;
+
+import java.util.List;
+
+public class EventMapper implements RowEntityMapper<Event> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of();
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return null;
+ }
+
+ @Override
+ public Event map(RowEntity row) {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
new file mode 100644
index 0000000000..f1c5aaf442
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
@@ -0,0 +1,61 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.apm.network.common.v3.KeyStringValuePair;
+import org.apache.skywalking.apm.network.logging.v3.LogTags;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
+import org.apache.skywalking.oap.server.core.query.type.KeyValue;
+import org.apache.skywalking.oap.server.core.query.type.Log;
+
+import java.util.List;
+
+public class LogMapper implements RowEntityMapper<Log> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(
+ AbstractLogRecord.SERVICE_ID, // 0
+ AbstractLogRecord.SERVICE_INSTANCE_ID, // 1
+ AbstractLogRecord.ENDPOINT_ID, // 2
+ AbstractLogRecord.TRACE_ID, // 3
+ AbstractLogRecord.TRACE_SEGMENT_ID,
+ AbstractLogRecord.SPAN_ID,
+ AbstractLogRecord.TIMESTAMP); // 6
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return ImmutableList.of(AbstractLogRecord.CONTENT_TYPE,
+ AbstractLogRecord.CONTENT,
+ AbstractLogRecord.TAGS_RAW_DATA); // 2
+ }
+
+ @Override
+ public Log map(RowEntity row) {
+ Log log = new Log();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ log.setServiceId((String) searchable.get(0).getValue());
+ log.setServiceInstanceId((String) searchable.get(1).getValue());
+ log.setEndpointId((String) searchable.get(2).getValue());
+ log.setTraceId((String) searchable.get(3).getValue());
+ log.setTimestamp(((Number) searchable.get(6).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ if (data.get(2).getValue() == null || ((ByteString) data.get(2).getValue()).isEmpty()) {
+ log.setContent("");
+ } else {
+ try {
+ // Don't read the tags as they have been in the data binary already.
+ LogTags logTags = LogTags.parseFrom((ByteString) data.get(2).getValue());
+ for (final KeyStringValuePair pair : logTags.getDataList()) {
+ log.getTags().add(new KeyValue(pair.getKey(), pair.getValue()));
+ }
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ return log;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
new file mode 100644
index 0000000000..aeb823c752
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
@@ -0,0 +1,40 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+
+import java.util.List;
+
+public class NetworkAddressAliasMapper implements RowEntityMapper<NetworkAddressAlias> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ // TODO: make these static fields public
+ return ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id");
+ }
+
+ @Override
+ public NetworkAddressAlias map(RowEntity row) {
+ NetworkAddressAlias model = new NetworkAddressAlias();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // searchable - last_update_time_bucket
+ model.setLastUpdateTimeBucket(((Number) searchable.get(0).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // data - time_bucket
+ model.setTimeBucket(((Number) data.get(0).getValue()).longValue());
+ // data - address
+ model.setAddress((String) data.get(1).getValue());
+ // data - represent_service_id
+ model.setRepresentServiceId((String) data.get(2).getValue());
+ // data - represent_service_instance_id
+ model.setRepresentServiceInstanceId((String) data.get(3).getValue());
+ return model;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
new file mode 100644
index 0000000000..bc157253f0
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
@@ -0,0 +1,39 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationType;
+
+import java.util.List;
+
+public class ProfileTaskLogMapper implements RowEntityMapper<ProfileTaskLog> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return ImmutableList.of(ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.INSTANCE_ID,
+ ProfileTaskLogRecord.OPERATION_TYPE);
+ }
+
+ @Override
+ public ProfileTaskLog map(RowEntity row) {
+ ProfileTaskLog profileTaskLog = new ProfileTaskLog();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // searchable - operation_time
+ profileTaskLog.setOperationTime(((Number) searchable.get(0).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // searchable - task_id
+ profileTaskLog.setTaskId((String) data.get(0).getValue());
+ // searchable - instance_id
+ profileTaskLog.setInstanceId((String) data.get(1).getValue());
+ // searchable - operation_type
+ profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(2).getValue()).intValue()));
+ return profileTaskLog;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
new file mode 100644
index 0000000000..8a49404d23
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
@@ -0,0 +1,42 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileTaskMapper implements RowEntityMapper<ProfileTask> {
+ public static final String ID = "profile_task_query_id";
+
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
+ ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ProfileTask map(RowEntity row) {
+ ProfileTask profileTask = new ProfileTask();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ profileTask.setId((String) searchable.get(0).getValue());
+ profileTask.setServiceId((String) searchable.get(1).getValue());
+ profileTask.setEndpointName((String) searchable.get(2).getValue());
+ profileTask.setStartTime(((Number) searchable.get(3).getValue()).longValue());
+ profileTask.setDuration(((Number) searchable.get(4).getValue()).intValue());
+ profileTask.setMinDurationThreshold(((Number) searchable.get(5).getValue()).intValue());
+ profileTask.setDumpPeriod(((Number) searchable.get(6).getValue()).intValue());
+ profileTask.setCreateTime(((Number) searchable.get(7).getValue()).intValue());
+ profileTask.setMaxSamplingCount(((Number) searchable.get(8).getValue()).intValue());
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
new file mode 100644
index 0000000000..99f23117f2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
@@ -0,0 +1,36 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileThreadSnapshotRecordMapper implements RowEntityMapper<ProfileThreadSnapshotRecord> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY);
+ }
+
+ @Override
+ public ProfileThreadSnapshotRecord map(RowEntity row) {
+ ProfileThreadSnapshotRecord record = new ProfileThreadSnapshotRecord();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ record.setTaskId((String) searchable.get(0).getValue());
+ record.setSegmentId((String) searchable.get(1).getValue());
+ record.setDumpTime(((Number) searchable.get(2).getValue()).longValue());
+ record.setSequence(((Number) searchable.get(3).getValue()).intValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ record.setStackBinary(((ByteString) data.get(0).getValue()).toByteArray());
+ return record;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
similarity index 95%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
index b2facb3373..cc1d48d94f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
@@ -1,4 +1,4 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
similarity index 99%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
index 6ea579ff55..70318136f6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
@@ -1,4 +1,4 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
import com.google.protobuf.ByteString;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index c7bc9cc13d..7e122611bf 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -1,19 +1,62 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
import org.apache.skywalking.oap.server.core.query.type.Alarms;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.AlarmMessageMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.alarm.AlarmRecord} is a stream,
* which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
*/
-public class BanyanDBAlarmQueryDAO implements IAlarmQueryDAO {
+public class BanyanDBAlarmQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IAlarmQueryDAO {
+ private final RowEntityMapper<AlarmMessage> mapper;
+
+ public BanyanDBAlarmQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ mapper = new AlarmMessageMapper(this);
+ }
+
@Override
public Alarms getAlarm(Integer scopeId, String keyword, int limit, int from, long startTB, long endTB, List<Tag> tags) throws IOException {
- return new Alarms();
+ final StreamQuery query = new StreamQuery(AlarmRecord.INDEX_NAME, mapper.searchableProjection());
+
+ if (Objects.nonNull(scopeId)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AlarmRecord.SCOPE, (long) scopeId));
+ }
+ if (startTB != 0 && endTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(startTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(endTB)));
+ }
+
+ // TODO: support keyword search
+
+ // TODO: support tag search
+
+ query.setLimit(limit);
+ query.setOffset(from);
+
+ StreamQueryResponse resp = getClient().query(query);
+
+ List<AlarmMessage> messages = resp.getElements().stream().map(mapper::map).collect(Collectors.toList());
+
+ Alarms alarms = new Alarms();
+ alarms.setTotal(messages.size());
+ alarms.getMsgs().addAll(messages);
+ return alarms;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index 7607668eba..25f2c27f4b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -1,17 +1,63 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
import org.apache.skywalking.oap.server.core.browser.source.BrowserErrorCategory;
+import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLogs;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BrowserErrorLogMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
+import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
*/
-public class BanyanDBBrowserLogQueryDAO implements IBrowserLogQueryDAO {
+public class BanyanDBBrowserLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IBrowserLogQueryDAO {
+ private static final RowEntityMapper<BrowserErrorLog> MAPPER = new BrowserErrorLogMapper();
+
+ public BanyanDBBrowserLogQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public BrowserErrorLogs queryBrowserErrorLogs(String serviceId, String serviceVersionId, String pagePathId, BrowserErrorCategory category, long startSecondTB, long endSecondTB, int limit, int from) throws IOException {
- return new BrowserErrorLogs();
+ final StreamQuery query = new StreamQuery(BrowserErrorLogRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.setDataProjections(MAPPER.dataProjection());
+
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_ID, serviceId));
+
+ if (startSecondTB != 0 && endSecondTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startSecondTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endSecondTB)));
+ }
+ if (StringUtil.isNotEmpty(serviceVersionId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
+ }
+ if (StringUtil.isNotEmpty(pagePathId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.PAGE_PATH_ID, pagePathId));
+ }
+ if (Objects.nonNull(category)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", BrowserErrorLogRecord.ERROR_CATEGORY, (long) category.getValue()));
+ }
+
+ query.setOffset(from);
+ query.setLimit(limit);
+
+ final StreamQueryResponse resp = getClient().query(query);
+
+ final BrowserErrorLogs logs = new BrowserErrorLogs();
+ logs.getLogs().addAll(resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList()));
+ logs.setTotal(logs.getLogs().size());
+ return logs;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index baba93b886..661f65b930 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -1,20 +1,86 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Order;
import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.Log;
import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.LogMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
*/
-public class BanyanDBLogQueryDAO implements ILogQueryDAO {
+public class BanyanDBLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements ILogQueryDAO {
+ private static final RowEntityMapper<Log> MAPPER = new LogMapper();
+
+ public BanyanDBLogQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
- public Logs queryLogs(String serviceId, String serviceInstanceId, String endpointId, TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit, long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent, List<String> excludingKeywordsOfContent) throws IOException {
- return new Logs();
+ public Logs queryLogs(String serviceId, String serviceInstanceId, String endpointId,
+ TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit,
+ long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent,
+ List<String> excludingKeywordsOfContent) throws IOException {
+ final StreamQuery query = new StreamQuery(LogRecord.INDEX_NAME, MAPPER.searchableProjection());
+ if (StringUtil.isNotEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_ID, serviceId));
+ }
+
+ if (startTB != 0 && endTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endTB)));
+ }
+
+ if (StringUtil.isNotEmpty(serviceInstanceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
+ }
+ if (StringUtil.isNotEmpty(endpointId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.ENDPOINT_ID, endpointId));
+ }
+ if (Objects.nonNull(relatedTrace)) {
+ if (StringUtil.isNotEmpty(relatedTrace.getTraceId())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_ID, relatedTrace.getTraceId()));
+ }
+ if (StringUtil.isNotEmpty(relatedTrace.getSegmentId())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_SEGMENT_ID, relatedTrace.getSegmentId()));
+ }
+ if (Objects.nonNull(relatedTrace.getSpanId())) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AbstractLogRecord.SPAN_ID, (long) relatedTrace.getSpanId()));
+ }
+ }
+
+ // TODO: if we allow to index tags?
+// if (CollectionUtils.isNotEmpty(tags)) {
+// for (final Tag tag : tags) {
+// query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+// }
+// }
+
+ StreamQueryResponse resp = getClient().query(query);
+
+ List<Log> entities = resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+
+ Logs logs = new Logs();
+ logs.getLogs().addAll(entities);
+ logs.setTotal(entities.size());
+
+ return logs;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
index 10675a49a1..337c288e96 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -1,17 +1,34 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.NetworkAddressAliasMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
-import java.util.Collections;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link NetworkAddressAlias} is a stream
*/
-public class BanyanDBNetworkAddressAliasDAO implements INetworkAddressAliasDAO {
+public class BanyanDBNetworkAddressAliasDAO extends AbstractDAO<BanyanDBStorageClient> implements INetworkAddressAliasDAO {
+ private static final RowEntityMapper<NetworkAddressAlias> MAPPER = new NetworkAddressAliasMapper();
+
+ public BanyanDBNetworkAddressAliasDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
- return Collections.emptyList();
+ StreamQuery query = new StreamQuery(NetworkAddressAlias.INDEX_NAME, MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
+
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 2b6290e2d2..6c19e2f701 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -1,18 +1,43 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskLogMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
-import java.util.Collections;
+import java.util.Comparator;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
*/
-public class BanyanDBProfileTaskLogQueryDAO implements IProfileTaskLogQueryDAO {
+public class BanyanDBProfileTaskLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileTaskLogQueryDAO {
+ private static final RowEntityMapper<ProfileTaskLog> MAPPER = new ProfileTaskLogMapper();
+
+ private final int queryMaxSize;
+
+ public BanyanDBProfileTaskLogQueryDAO(BanyanDBStorageClient client, int queryMaxSize) {
+ super(client);
+ this.queryMaxSize = queryMaxSize;
+ }
+
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
- return Collections.emptyList();
+ final StreamQuery query = new StreamQuery(ProfileTaskLogRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.setDataProjections(MAPPER.dataProjection());
+ query.setLimit(this.queryMaxSize);
+
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream()
+ .map(MAPPER::map)
+ .sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
+ .collect(Collectors.toList());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 34af74eb93..67f28ce62f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -10,8 +10,8 @@ import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.ProfileTaskMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
@@ -34,34 +34,46 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractDAO<BanyanDBStorageClie
query.setDataProjections(MAPPER.dataProjection());
if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskRecord.SERVICE_ID, serviceId));
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
+ ProfileTaskRecord.SERVICE_ID, serviceId));
}
if (StringUtil.isNotEmpty(endpointName)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskRecord.ENDPOINT_NAME, endpointName));
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
+ ProfileTaskRecord.ENDPOINT_NAME, endpointName));
}
if (Objects.nonNull(startTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable",
+ ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
}
if (Objects.nonNull(endTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable",
+ ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
}
- // TODO: why delete?
-
if (Objects.nonNull(limit)) {
query.setLimit(limit);
}
+ query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
+
StreamQueryResponse resp = getClient().query(query);
return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
@Override
public ProfileTask getById(String id) throws IOException {
- // TODO: support id query
- throw new UnsupportedOperationException("element id get is not supported");
+ if (StringUtil.isEmpty(id)) {
+ return null;
+ }
+
+ final StreamQuery query = new StreamQuery(ProfileTaskRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskMapper.ID, id));
+ query.setLimit(1);
+
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(MAPPER::map).findAny().orElse(null);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index 456636f287..cb5067e890 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -9,10 +9,10 @@ import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.BasicTraceMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.ProfileThreadSnapshotRecordMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.SegmentRecordMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileThreadSnapshotRecordMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
import java.io.IOException;
import java.util.ArrayList;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index a37cfbb54c..838e8a1eb3 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -31,9 +31,9 @@ import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.BasicTraceMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.SegmentRecordMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
import org.joda.time.DateTimeZone;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index a4875e65f1..33a69a969c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -13,8 +13,8 @@ import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.DashboardConfigurationMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.DashboardConfigurationMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
[skywalking] 02/24: abstract row entity mapper
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit c0bbcf1f68bd98430cdc510a67bd239fa0c4f044
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Mon Nov 29 19:32:46 2021 +0800
abstract row entity mapper
---
.../plugin/banyandb/BanyanDBStorageClient.java | 4 +++
.../banyandb/converter/BasicTraceMapper.java | 25 +++++++++++++++
.../converter/DashboardConfigurationMapper.java | 20 ++++++++++++
.../plugin/banyandb/converter/RowEntityMapper.java | 7 ++++
.../banyandb/converter/SegmentRecordMapper.java | 27 ++++++++++++++++
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 37 +++++-----------------
.../stream/BanyanDBUITemplateManagementDAO.java | 35 +++++++++++++++++---
7 files changed, 121 insertions(+), 34 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index eed4386e57..6ae741f971 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -44,6 +44,10 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
+ public void write() {
+ this.client
+ }
+
public StreamBulkWriteProcessor createBulkProcessor(int maxBulkSize, int flushInterval, int concurrency) {
return this.client.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
new file mode 100644
index 0000000000..bb43a8080a
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
@@ -0,0 +1,25 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+
+import java.util.List;
+
+public class BasicTraceMapper implements RowEntityMapper<BasicTrace> {
+ @Override
+ public BasicTrace map(RowEntity row) {
+ BasicTrace trace = new BasicTrace();
+ trace.setSegmentId(row.getId());
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ trace.getTraceIds().add((String) searchable.get(0).getValue());
+ trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
+ trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ (String) searchable.get(2).getValue()
+ ).getEndpointName());
+ trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
+ trace.setStart(String.valueOf(searchable.get(4).getValue()));
+ return trace;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
new file mode 100644
index 0000000000..f4c4c19beb
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
@@ -0,0 +1,20 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+
+import java.util.List;
+
+public class DashboardConfigurationMapper implements RowEntityMapper<DashboardConfiguration> {
+ @Override
+ public DashboardConfiguration map(RowEntity row) {
+ DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ dashboardConfiguration.setName((String) searchable.get(0).getValue());
+ dashboardConfiguration.setDisabled(BooleanUtils.valueToBoolean(((Number) searchable.get(1).getValue()).intValue()));
+ // TODO: convert back from data?
+ return dashboardConfiguration;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
new file mode 100644
index 0000000000..eb9b5fadba
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
@@ -0,0 +1,7 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+
+public interface RowEntityMapper<T> {
+ T map(RowEntity row);
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
new file mode 100644
index 0000000000..2b20227931
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
@@ -0,0 +1,27 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+
+import java.util.List;
+
+public class SegmentRecordMapper implements RowEntityMapper<SegmentRecord> {
+ @Override
+ public SegmentRecord map(RowEntity row) {
+ SegmentRecord record = new SegmentRecord();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ record.setSegmentId(row.getId());
+ record.setTraceId((String) searchable.get(0).getValue());
+ record.setIsError(((Number) searchable.get(1).getValue()).intValue());
+ record.setServiceId((String) searchable.get(2).getValue());
+ record.setServiceInstanceId((String) searchable.get(3).getValue());
+ record.setEndpointId((String) searchable.get(4).getValue());
+ record.setLatency(((Number) searchable.get(5).getValue()).intValue());
+ record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // TODO: support binary data in the client SDK
+ record.setDataBinary((byte[]) data.get(0).getValue());
+ return record;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 9fc403afc6..0a3f0e53b9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -30,6 +30,9 @@ import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.BasicTraceMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.SegmentRecordMapper;
import org.joda.time.DateTimeZone;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
@@ -40,6 +43,9 @@ import java.util.List;
import java.util.stream.Collectors;
public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements ITraceQueryDAO {
+ private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
+ private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
+
private static final DateTimeFormatter YYYYMMDDHHMMSS = DateTimeFormat.forPattern("yyyyMMddHHmmss");
private static final List<String> BASIC_QUERY_PROJ = ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time");
@@ -114,19 +120,7 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
StreamQueryResponse response = this.getClient().query(query);
TraceBrief brief = new TraceBrief();
brief.setTotal(response.size());
- brief.getTraces().addAll(response.getElements().stream().map(elem -> {
- BasicTrace trace = new BasicTrace();
- trace.setSegmentId(elem.getId());
- final List<TagAndValue<?>> searchable = elem.getTagFamilies().get(0);
- trace.getTraceIds().add((String) searchable.get(0).getValue());
- trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
- trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- (String) searchable.get(2).getValue()
- ).getEndpointName());
- trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
- trace.setStart(String.valueOf(searchable.get(4).getValue()));
- return trace;
- }).collect(Collectors.toList()));
+ brief.getTraces().addAll(response.getElements().stream().map(BASIC_TRACE_MAPPER::map).collect(Collectors.toList()));
return brief;
}
@@ -136,22 +130,7 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
query.setDataBinary(true);
StreamQueryResponse response = this.getClient().query(query);
- return response.getElements().stream().map(elem -> {
- SegmentRecord record = new SegmentRecord();
- final List<TagAndValue<?>> searchable = elem.getTagFamilies().get(0);
- record.setSegmentId(elem.getId());
- record.setTraceId((String) searchable.get(0).getValue());
- record.setIsError(((Number) searchable.get(1).getValue()).intValue());
- record.setServiceId((String) searchable.get(2).getValue());
- record.setServiceInstanceId((String) searchable.get(3).getValue());
- record.setEndpointId((String) searchable.get(4).getValue());
- record.setLatency(((Number) searchable.get(5).getValue()).intValue());
- record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- final List<TagAndValue<?>> data = elem.getTagFamilies().get(1);
- // TODO: support binary data in the client SDK
- record.setDataBinary((byte[]) data.get(0).getValue());
- return record;
- }).collect(Collectors.toList());
+ return response.getElements().stream().map(SEGMENT_RECORD_MAPPER::map).collect(Collectors.toList());
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index ad8fed926e..102540c7b5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -1,31 +1,56 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.DashboardConfigurationMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
import java.io.IOException;
-import java.util.Collections;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
*/
-public class BanyanDBUITemplateManagementDAO implements UITemplateManagementDAO {
+public class BanyanDBUITemplateManagementDAO extends AbstractDAO<BanyanDBStorageClient> implements UITemplateManagementDAO {
+ private static final RowEntityMapper<DashboardConfiguration> MAPPER = new DashboardConfigurationMapper();
+
+ public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- return Collections.emptyList();
+ StreamQuery query = new StreamQuery(UITemplate.INDEX_NAME, ImmutableList.of(
+ UITemplate.NAME,
+ UITemplate.DISABLED
+ ));
+ query.setLimit(10000);
+ if (!includingDisabled) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", UITemplate.DISABLED, (long) BooleanUtils.FALSE));
+ }
+ StreamQueryResponse resp = this.getClient().query(query);
+ return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
@Override
public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
- return TemplateChangeStatus.builder().status(false).message("Can't add a new template").build();
+ // TODO: support single write
}
@Override
public TemplateChangeStatus changeTemplate(DashboardSetting setting) throws IOException {
- return TemplateChangeStatus.builder().status(false).message("Can't add/update the template").build();
+ return TemplateChangeStatus.builder().status(false).message("Can't update the template").build();
}
@Override
[skywalking] 03/24: complete UITemplate, ProfileTaskRecord and ProfileThreasSnapshot query
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit b89c8ced7aa01ba5aa57646aa1a4fd8b4b6640e5
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Tue Nov 30 19:47:54 2021 +0800
complete UITemplate, ProfileTaskRecord and ProfileThreasSnapshot query
---
.../plugin/banyandb/BanyanDBStorageClient.java | 5 +-
.../banyandb/converter/BasicTraceMapper.java | 12 +++
.../converter/DashboardConfigurationMapper.java | 23 ++++-
.../banyandb/converter/ProfileTaskMapper.java | 28 ++++++
.../ProfileThreadSnapshotRecordMapper.java | 27 ++++++
.../plugin/banyandb/converter/RowEntityMapper.java | 6 ++
.../banyandb/converter/SegmentRecordMapper.java | 16 +++-
.../stream/BanyanDBProfileTaskQueryDAO.java | 52 +++++++++-
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 105 +++++++++++++++++++--
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 6 +-
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 19 ++--
.../stream/BanyanDBUITemplateManagementDAO.java | 33 +++++--
12 files changed, 297 insertions(+), 35 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 6ae741f971..3ad4daf918 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -4,6 +4,7 @@ import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.client.healthcheck.DelegatedHealthChecker;
import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckable;
@@ -44,8 +45,8 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
- public void write() {
- this.client
+ public void write(StreamWrite streamWrite) {
+ this.client.write(streamWrite);
}
public StreamBulkWriteProcessor createBulkProcessor(int maxBulkSize, int flushInterval, int concurrency) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
index bb43a8080a..b3512158f2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/BasicTraceMapper.java
@@ -1,10 +1,12 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+import java.util.Collections;
import java.util.List;
public class BasicTraceMapper implements RowEntityMapper<BasicTrace> {
@@ -22,4 +24,14 @@ public class BasicTraceMapper implements RowEntityMapper<BasicTrace> {
trace.setStart(String.valueOf(searchable.get(4).getValue()));
return trace;
}
+
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time");
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.emptyList();
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
index f4c4c19beb..3f7df9cb72 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/DashboardConfigurationMapper.java
@@ -1,7 +1,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
+import org.apache.skywalking.oap.server.core.query.enumeration.TemplateType;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
@@ -12,9 +15,27 @@ public class DashboardConfigurationMapper implements RowEntityMapper<DashboardCo
public DashboardConfiguration map(RowEntity row) {
DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // name
dashboardConfiguration.setName((String) searchable.get(0).getValue());
+ // disabled
dashboardConfiguration.setDisabled(BooleanUtils.valueToBoolean(((Number) searchable.get(1).getValue()).intValue()));
- // TODO: convert back from data?
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // activated
+ dashboardConfiguration.setActivated(BooleanUtils.valueToBoolean(((Number) data.get(0).getValue()).intValue()));
+ // configuration
+ dashboardConfiguration.setConfiguration((String) data.get(1).getValue());
+ // type
+ dashboardConfiguration.setType(TemplateType.forName((String) data.get(2).getValue()));
return dashboardConfiguration;
}
+
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE);
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java
new file mode 100644
index 0000000000..ea1ca09cb0
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileTaskMapper.java
@@ -0,0 +1,28 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileTaskMapper implements RowEntityMapper<ProfileTask> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME, ProfileTaskRecord.START_TIME,
+ ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD, ProfileTaskRecord.DUMP_PERIOD,
+ ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ProfileTask map(RowEntity row) {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java
new file mode 100644
index 0000000000..9a25688132
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/ProfileThreadSnapshotRecordMapper.java
@@ -0,0 +1,27 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileThreadSnapshotRecordMapper implements RowEntityMapper<ProfileThreadSnapshotRecord> {
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
+ ProfileThreadSnapshotRecord.STACK_BINARY);
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ProfileThreadSnapshotRecord map(RowEntity row) {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
index eb9b5fadba..b2facb3373 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/RowEntityMapper.java
@@ -2,6 +2,12 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import java.util.List;
+
public interface RowEntityMapper<T> {
+ List<String> searchableProjection();
+
+ List<String> dataProjection();
+
T map(RowEntity row);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
index 2b20227931..6ea579ff55 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/converter/SegmentRecordMapper.java
@@ -1,9 +1,12 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.converter;
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import java.util.Collections;
import java.util.List;
public class SegmentRecordMapper implements RowEntityMapper<SegmentRecord> {
@@ -20,8 +23,17 @@ public class SegmentRecordMapper implements RowEntityMapper<SegmentRecord> {
record.setLatency(((Number) searchable.get(5).getValue()).intValue());
record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // TODO: support binary data in the client SDK
- record.setDataBinary((byte[]) data.get(0).getValue());
+ record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
return record;
}
+
+ @Override
+ public List<String> searchableProjection() {
+ return ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time");
+ }
+
+ @Override
+ public List<String> dataProjection() {
+ return Collections.singletonList("data_binary");
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index c99d267cbe..34af74eb93 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -1,23 +1,67 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.ProfileTaskMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
import java.io.IOException;
-import java.util.Collections;
import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
*/
-public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+public class BanyanDBProfileTaskQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileTaskQueryDAO {
+ private static final RowEntityMapper<ProfileTask> MAPPER = new ProfileTaskMapper();
+
+ public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
- return Collections.emptyList();
+ final StreamQuery query = new StreamQuery(ProfileTaskRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.setDataProjections(MAPPER.dataProjection());
+
+ if (StringUtil.isNotEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskRecord.SERVICE_ID, serviceId));
+ }
+
+ if (StringUtil.isNotEmpty(endpointName)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskRecord.ENDPOINT_NAME, endpointName));
+ }
+
+ if (Objects.nonNull(startTimeBucket)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
+ }
+
+ if (Objects.nonNull(endTimeBucket)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
+ }
+
+ // TODO: why delete?
+
+ if (Objects.nonNull(limit)) {
+ query.setLimit(limit);
+ }
+
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
@Override
public ProfileTask getById(String id) throws IOException {
- return null;
+ // TODO: support id query
+ throw new UnsupportedOperationException("element id get is not supported");
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index 8a27e885fe..456636f287 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -1,40 +1,133 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.BasicTraceMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.ProfileThreadSnapshotRecordMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.converter.SegmentRecordMapper;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.Collections;
+import java.util.Comparator;
+import java.util.LinkedList;
import java.util.List;
+import java.util.function.Function;
+import java.util.stream.Collectors;
/**
* {@link ProfileThreadSnapshotRecord} is a stream
*/
-public class BanyanDBProfileThreadSnapshotQueryDAO implements IProfileThreadSnapshotQueryDAO {
+public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileThreadSnapshotQueryDAO {
+ private static final RowEntityMapper<ProfileThreadSnapshotRecord> MAPPER = new ProfileThreadSnapshotRecordMapper();
+ private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
+ private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
+
+ public BanyanDBProfileThreadSnapshotQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
- return Collections.emptyList();
+ final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.TASK_ID, taskId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEQUENCE, 0L));
+ StreamQueryResponse resp = getClient().query(query);
+
+ final List<String> segmentIDs = new ArrayList<>(resp.size());
+ resp.getElements().forEach(elem -> segmentIDs.add(MAPPER.map(elem).getSegmentId()));
+ if (segmentIDs.isEmpty()) {
+ return Collections.emptyList();
+ }
+
+ // TODO: support `IN` or `OR` logic operation in BanyanDB
+ List<BasicTrace> basicTraces = new LinkedList<>();
+ for (String segmentID : segmentIDs) {
+ final StreamQuery traceQuery = new StreamQuery(SegmentRecord.INDEX_NAME, BASIC_TRACE_MAPPER.searchableProjection());
+ traceQuery.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.SEGMENT_ID, segmentID));
+ StreamQueryResponse traceResponse = getClient().query(traceQuery);
+ basicTraces.addAll(traceResponse.getElements().stream().map(BASIC_TRACE_MAPPER::map).collect(Collectors.toList()));
+ }
+
+ // TODO: Sort in DB with DESC
+ basicTraces = basicTraces.stream()
+ // comparing start_time
+ .sorted(Comparator.comparing((Function<BasicTrace, Long>) basicTrace -> Long.parseLong(basicTrace.getStart()))
+ // and sort in reverse order
+ .reversed())
+ .collect(Collectors.toList());
+ return basicTraces;
}
@Override
public int queryMinSequence(String segmentId, long start, long end) throws IOException {
- return 0;
+ return querySequenceWithAgg(AggType.MIN, segmentId, start, end);
}
@Override
public int queryMaxSequence(String segmentId, long start, long end) throws IOException {
- return 0;
+ return querySequenceWithAgg(AggType.MAX, segmentId, start, end);
}
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
- return Collections.emptyList();
+ final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) maxSequence))
+ .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) minSequence));
+ query.setDataProjections(MAPPER.dataProjection());
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
- return null;
+ final StreamQuery query = new StreamQuery(SegmentRecord.INDEX_NAME, SEGMENT_RECORD_MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.INDEX_NAME, segmentId));
+ query.setDataProjections(SEGMENT_RECORD_MAPPER.dataProjection());
+ StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(SEGMENT_RECORD_MAPPER::map).findFirst().orElse(null);
+ }
+
+ private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) {
+ final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, end))
+ .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, start));
+ query.setDataProjections(MAPPER.dataProjection());
+
+ StreamQueryResponse resp = getClient().query(query);
+ List<ProfileThreadSnapshotRecord> records = resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+
+ switch (aggType) {
+ case MIN:
+ int minValue = Integer.MAX_VALUE;
+ for (final ProfileThreadSnapshotRecord record : records) {
+ int sequence = record.getSequence();
+ minValue = Math.min(minValue, sequence);
+ }
+ return minValue;
+ case MAX:
+ int maxValue = Integer.MIN_VALUE;
+ for (ProfileThreadSnapshotRecord record : records) {
+ int sequence = record.getSequence();
+ maxValue = Math.max(maxValue, sequence);
+ }
+ return maxValue;
+ default:
+ throw new IllegalArgumentException("should not reach this line");
+ }
+ }
+
+ enum AggType {
+ MIN, MAX
}
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index f59bc25f9e..54870d98c6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -45,11 +45,11 @@ public class BanyanDBRecordDAO implements IRecordDAO {
if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
SegmentRecord segmentRecord = (SegmentRecord) record;
StreamWrite streamWrite = StreamWrite.builder()
- .name(BanyanDBSchema.NAME)
- .binary(segmentRecord.getDataBinary())
+ .name(SegmentRecord.INDEX_NAME)
+ .dataTag(Tag.binaryField(segmentRecord.getDataBinary()))
.timestamp(segmentRecord.getStartTime())
.elementId(segmentRecord.getSegmentId())
- .tags(buildFieldObjects(this.storageBuilder.entity2Storage(segmentRecord)))
+ .searchableTags(buildFieldObjects(this.storageBuilder.entity2Storage(segmentRecord)))
.build();
return new BanyanDBStreamInsertRequest(streamWrite);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 0a3f0e53b9..a37cfbb54c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -19,9 +19,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.*;
-import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.query.type.*;
@@ -48,9 +49,6 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
private static final DateTimeFormatter YYYYMMDDHHMMSS = DateTimeFormat.forPattern("yyyyMMddHHmmss");
- private static final List<String> BASIC_QUERY_PROJ = ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time");
- private static final List<String> TRACE_ID_QUERY_PROJ = ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time");
-
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -59,9 +57,10 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
public TraceBrief queryBasicTraces(long startSecondTB, long endSecondTB, long minDuration, long maxDuration, String serviceId, String serviceInstanceId, String endpointId, String traceId, int limit, int from, TraceState traceState, QueryOrder queryOrder, List<Tag> tags) throws IOException {
StreamQuery query;
if (startSecondTB != 0 && endSecondTB != 0) {
- query = new StreamQuery(BanyanDBSchema.NAME, new TimestampRange(parseMillisFromStartSecondTB(startSecondTB), parseMillisFromEndSecondTB(endSecondTB)), BASIC_QUERY_PROJ);
+ query = new StreamQuery(BanyanDBSchema.NAME, new TimestampRange(parseMillisFromStartSecondTB(startSecondTB),
+ parseMillisFromEndSecondTB(endSecondTB)), BASIC_TRACE_MAPPER.searchableProjection());
} else {
- query = new StreamQuery(BanyanDBSchema.NAME, BASIC_QUERY_PROJ);
+ query = new StreamQuery(BanyanDBSchema.NAME, BASIC_TRACE_MAPPER.searchableProjection());
}
if (minDuration != 0) {
// duration >= minDuration
@@ -126,9 +125,9 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
- StreamQuery query = new StreamQuery(BanyanDBSchema.NAME, TRACE_ID_QUERY_PROJ);
+ StreamQuery query = new StreamQuery(BanyanDBSchema.NAME, SEGMENT_RECORD_MAPPER.searchableProjection());
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
- query.setDataBinary(true);
+ query.setDataProjections(SEGMENT_RECORD_MAPPER.dataProjection());
StreamQueryResponse response = this.getClient().query(query);
return response.getElements().stream().map(SEGMENT_RECORD_MAPPER::map).collect(Collectors.toList());
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index 102540c7b5..a4875e65f1 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -1,9 +1,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.Tag;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
@@ -25,27 +26,45 @@ import java.util.stream.Collectors;
public class BanyanDBUITemplateManagementDAO extends AbstractDAO<BanyanDBStorageClient> implements UITemplateManagementDAO {
private static final RowEntityMapper<DashboardConfiguration> MAPPER = new DashboardConfigurationMapper();
+ private static final long UI_TEMPLATE_TIMESTAMP = 1L;
+
public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- StreamQuery query = new StreamQuery(UITemplate.INDEX_NAME, ImmutableList.of(
- UITemplate.NAME,
- UITemplate.DISABLED
- ));
+ StreamQuery query = new StreamQuery(UITemplate.INDEX_NAME, MAPPER.dataProjection());
query.setLimit(10000);
if (!includingDisabled) {
query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", UITemplate.DISABLED, (long) BooleanUtils.FALSE));
}
+ query.setDataProjections(MAPPER.dataProjection());
StreamQueryResponse resp = this.getClient().query(query);
return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
}
@Override
public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
- // TODO: support single write
+ final UITemplate uiTemplate = setting.toEntity();
+
+ StreamWrite request = StreamWrite.builder()
+ .name(UITemplate.INDEX_NAME)
+ // searchable - name
+ .searchableTag(Tag.stringField(uiTemplate.getName()))
+ // searchable - disabled
+ .searchableTag(Tag.longField(uiTemplate.getDisabled()))
+ // data - type
+ .dataTag(Tag.stringField(uiTemplate.getType()))
+ // data - configuration
+ .dataTag(Tag.stringField(uiTemplate.getConfiguration()))
+ // data - activated
+ .dataTag(Tag.longField(uiTemplate.getActivated()))
+ .timestamp(UI_TEMPLATE_TIMESTAMP)
+ .elementId(uiTemplate.id())
+ .build();
+ getClient().write(request);
+ return TemplateChangeStatus.builder().status(true).build();
}
@Override
@@ -55,6 +74,6 @@ public class BanyanDBUITemplateManagementDAO extends AbstractDAO<BanyanDBStorage
@Override
public TemplateChangeStatus disableTemplate(String name) throws IOException {
- return TemplateChangeStatus.builder().status(false).message("Can't add/update the template").build();
+ return TemplateChangeStatus.builder().status(false).message("Can't disable the template").build();
}
}
[skywalking] 15/24: support metadata-v2
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit fa24b07b79e5c64db282891a7b613fcfd9a2880b
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Wed Jan 19 23:06:51 2022 +0800
support metadata-v2
---
...aller.java => BanyanDBAggregationQueryDAO.java} | 29 ++--
.../plugin/banyandb/BanyanDBIndexInstaller.java | 3 +
.../plugin/banyandb/BanyanDBMetricsQueryDAO.java | 51 +++++++
.../banyandb/BanyanDBStorageBuilderFactory.java | 26 +++-
.../plugin/banyandb/BanyanDBStorageClient.java | 8 +-
.../plugin/banyandb/BanyanDBStorageProvider.java | 18 ++-
...aller.java => BanyanDBTopNRecordsQueryDAO.java} | 29 ++--
.../storage/plugin/banyandb/StreamMetaInfo.java | 103 +++-----------
.../plugin/banyandb/schema/AlarmRecordBuilder.java | 10 +-
.../schema/BanyanDBStorageDataBuilder.java | 14 +-
.../schema/BrowserErrorLogRecordBuilder.java | 8 +-
... EndpointRelationServerSideMetricsBuilder.java} | 22 +--
.../plugin/banyandb/schema/EventBuilder.java | 8 +-
.../plugin/banyandb/schema/LogRecordBuilder.java | 8 +-
.../storage/plugin/banyandb/schema/Metadata.java | 52 +++++--
.../schema/NetworkAddressAliasBuilder.java | 8 +-
.../schema/ProfileTaskLogRecordBuilder.java | 8 +-
.../banyandb/schema/ProfileTaskRecordBuilder.java | 6 +-
.../schema/ProfileThreadSnapshotRecordBuilder.java | 8 +-
.../banyandb/schema/SegmentRecordBuilder.java | 8 +-
...eInstanceRelationClientSideMetricsBuilder.java} | 24 ++--
...eInstanceRelationServerSideMetricsBuilder.java} | 24 ++--
...> ServiceRelationClientSideMetricsBuilder.java} | 22 +--
...> ServiceRelationServerSideMetricsBuilder.java} | 22 +--
.../plugin/banyandb/schema/UITemplateBuilder.java | 8 +-
.../BanyanDBHistoryDeleteDAO.java} | 25 +---
.../banyandb/stream/BanyanDBManagementDAO.java | 10 +-
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 103 +++++---------
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 57 ++++++++
.../banyandb/stream/BanyanDBTopologyQueryDAO.java | 155 +++++++++++++++++++++
.../stream/BanyanDBUITemplateManagementDAO.java | 3 +-
.../metadata/index_rules/segment/db.instance.json | 5 +-
.../segment/{db.instance.json => db.type.json} | 7 +-
.../segment/{db.instance.json => duration.json} | 9 +-
.../segment/{db.instance.json => endpoint_id.json} | 7 +-
.../segment/{db.instance.json => http.method.json} | 7 +-
.../segment/{db.instance.json => mq.broker.json} | 7 +-
.../segment/{db.instance.json => mq.queue.json} | 7 +-
.../segment/{db.instance.json => mq.topic.json} | 7 +-
.../segment/{db.instance.json => status_code.json} | 7 +-
.../metadata/index_rules/segment/trace_id.json | 12 ++
.../db.instance.json => ui_template/disabled.json} | 7 +-
.../db.instance.json => ui_template/name.json} | 7 +-
.../src/main/resources/metadata/ui_template.json | 50 +++++++
44 files changed, 640 insertions(+), 379 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBAggregationQueryDAO.java
similarity index 50%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBAggregationQueryDAO.java
index ffc5aaf586..771ec36bb4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBAggregationQueryDAO.java
@@ -18,27 +18,18 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-import org.apache.skywalking.oap.server.core.storage.StorageException;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
-import org.apache.skywalking.oap.server.library.client.Client;
-import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.TopNCondition;
+import org.apache.skywalking.oap.server.core.query.type.KeyValue;
+import org.apache.skywalking.oap.server.core.query.type.SelectedRecord;
+import org.apache.skywalking.oap.server.core.storage.query.IAggregationQueryDAO;
-public class BanyanDBIndexInstaller extends ModelInstaller {
- public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
- super(client, moduleManager);
- }
-
- @Override
- protected boolean isExists(Model model) throws StorageException {
- return false;
- }
+import java.io.IOException;
+import java.util.List;
+public class BanyanDBAggregationQueryDAO implements IAggregationQueryDAO {
@Override
- protected void createTable(Model model) throws StorageException {
- StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
- if (metaInfo != null) {
- ((BanyanDBStorageClient) client).createStream(metaInfo);
- }
+ public List<SelectedRecord> sortMetrics(TopNCondition condition, String valueColumnName, Duration duration, List<KeyValue> additionalConditions) throws IOException {
+ return null;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index ffc5aaf586..c9566b1549 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -18,12 +18,14 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.server.core.storage.StorageException;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.module.ModuleManager;
+@Slf4j
public class BanyanDBIndexInstaller extends ModelInstaller {
public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
super(client, moduleManager);
@@ -38,6 +40,7 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
protected void createTable(Model model) throws StorageException {
StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
if (metaInfo != null) {
+ log.info("install index {}", model.getName());
((BanyanDBStorageClient) client).createStream(metaInfo);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsQueryDAO.java
new file mode 100644
index 0000000000..96a12760f9
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsQueryDAO.java
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.MetricsCondition;
+import org.apache.skywalking.oap.server.core.query.type.HeatMap;
+import org.apache.skywalking.oap.server.core.query.type.MetricsValues;
+import org.apache.skywalking.oap.server.core.storage.query.IMetricsQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBMetricsQueryDAO implements IMetricsQueryDAO {
+ @Override
+ public long readMetricsValue(MetricsCondition condition, String valueColumnName, Duration duration) throws IOException {
+ return 0;
+ }
+
+ @Override
+ public MetricsValues readMetricsValues(MetricsCondition condition, String valueColumnName, Duration duration) throws IOException {
+ return null;
+ }
+
+ @Override
+ public List<MetricsValues> readLabeledMetricsValues(MetricsCondition condition, String valueColumnName, List<String> labels, Duration duration) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public HeatMap readHeatMap(MetricsCondition condition, String valueColumnName, Duration duration) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
index b87d3d1dc5..0cb10763eb 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
@@ -18,11 +18,17 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
@@ -37,6 +43,7 @@ import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BrowserErrorLogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EndpointRelationServerSideMetricsBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EventBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.Metadata;
@@ -45,9 +52,16 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTa
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskRecordBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileThreadSnapshotRecordBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceInstanceRelationClientSideMetricsBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceInstanceRelationServerSideMetricsBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceRelationClientSideMetricsBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceRelationServerSideMetricsBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
+@Slf4j
public class BanyanDBStorageBuilderFactory implements StorageBuilderFactory {
+ private static final StorageBuilderFactory FALLBACK = new StorageBuilderFactory.Default();
+
@Override
public BuilderTemplateDefinition builderTemplate() {
return new BuilderTemplateDefinition(StorageHashMapBuilder.class.getName(), "metrics-builder");
@@ -81,8 +95,18 @@ public class BanyanDBStorageBuilderFactory implements StorageBuilderFactory {
return Metadata.EndpointTrafficBuilder.class;
} else if (NetworkAddressAlias.class.equals(dataType)) {
return NetworkAddressAliasBuilder.class;
+ } else if (EndpointRelationServerSideMetrics.class.equals(dataType)) {
+ return EndpointRelationServerSideMetricsBuilder.class;
+ } else if (ServiceRelationServerSideMetrics.class.equals(dataType)) {
+ return ServiceRelationServerSideMetricsBuilder.class;
+ } else if (ServiceRelationClientSideMetrics.class.equals(dataType)) {
+ return ServiceRelationClientSideMetricsBuilder.class;
+ } else if (ServiceInstanceRelationServerSideMetrics.class.equals(dataType)) {
+ return ServiceInstanceRelationServerSideMetricsBuilder.class;
+ } else if (ServiceInstanceRelationClientSideMetrics.class.equals(dataType)) {
+ return ServiceInstanceRelationClientSideMetricsBuilder.class;
}
- throw new UnsupportedOperationException("unsupported storage type");
+ return FALLBACK.builderOf(dataType, defaultBuilder);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 2ee45b12f0..7d61e4e085 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -31,9 +31,6 @@ import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckab
import org.apache.skywalking.oap.server.library.util.HealthChecker;
import java.io.IOException;
-import java.time.Instant;
-import java.time.ZoneOffset;
-import java.time.ZonedDateTime;
/**
* BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
@@ -71,10 +68,7 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
public void createStream(StreamMetaInfo streamMetaInfo) {
Stream stm = this.client.define(streamMetaInfo.getStream());
if (stm != null) {
- // TODO: should be fixed in SDK
- this.client.defineIndexRules(stm, ZonedDateTime.from(Instant.now()),
- ZonedDateTime.of(2099, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC),
- streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
+ this.client.defineIndexRules(stm, streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 64f8e96cbf..6d4340e48d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -21,6 +21,7 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.IBatchDAO;
+import org.apache.skywalking.oap.server.core.storage.IHistoryDeleteDAO;
import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
import org.apache.skywalking.oap.server.core.storage.StorageDAO;
import org.apache.skywalking.oap.server.core.storage.StorageModule;
@@ -30,11 +31,15 @@ import org.apache.skywalking.oap.server.core.storage.model.ModelCreator;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IAggregationQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IMetricsQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.ITopNRecordsQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.module.ModuleConfig;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
@@ -45,6 +50,7 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBA
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBatchDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEventQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBHistoryDeleteDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBMetadataQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBNetworkAddressAliasDAO;
@@ -52,6 +58,7 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBP
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBStorageDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBTopologyQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBTraceQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
@@ -88,25 +95,28 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.client = new BanyanDBStorageClient(config.getHost(), config.getPort(), config.getGroup());
+ // Stream
this.registerServiceImplementation(IBatchDAO.class, new BanyanDBBatchDAO(client, config.getMaxBulkSize(), config.getFlushInterval(), config.getConcurrentWriteThreads()));
this.registerServiceImplementation(StorageDAO.class, new BanyanDBStorageDAO(client));
-
this.registerServiceImplementation(INetworkAddressAliasDAO.class, new BanyanDBNetworkAddressAliasDAO(client));
-
this.registerServiceImplementation(ITraceQueryDAO.class, new BanyanDBTraceQueryDAO(client));
this.registerServiceImplementation(IBrowserLogQueryDAO.class, new BanyanDBBrowserLogQueryDAO(client));
this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO(client));
this.registerServiceImplementation(IAlarmQueryDAO.class, new BanyanDBAlarmQueryDAO(client));
this.registerServiceImplementation(ILogQueryDAO.class, new BanyanDBLogQueryDAO(client));
-
this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO(client));
this.registerServiceImplementation(IProfileTaskLogQueryDAO.class, new BanyanDBProfileTaskLogQueryDAO(client, this.config.getFetchTaskLogMaxSize()));
this.registerServiceImplementation(
IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO(client));
-
this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO(client));
+ this.registerServiceImplementation(ITopologyQueryDAO.class, new BanyanDBTopologyQueryDAO(client));
+ // TODO: metrics
+ this.registerServiceImplementation(IHistoryDeleteDAO.class, new BanyanDBHistoryDeleteDAO());
+ this.registerServiceImplementation(IMetricsQueryDAO.class, new BanyanDBMetricsQueryDAO());
+ this.registerServiceImplementation(IAggregationQueryDAO.class, new BanyanDBAggregationQueryDAO());
+ this.registerServiceImplementation(ITopNRecordsQueryDAO.class, new BanyanDBTopNRecordsQueryDAO());
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTopNRecordsQueryDAO.java
similarity index 50%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTopNRecordsQueryDAO.java
index ffc5aaf586..97f549ddfb 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBTopNRecordsQueryDAO.java
@@ -18,27 +18,18 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-import org.apache.skywalking.oap.server.core.storage.StorageException;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
-import org.apache.skywalking.oap.server.library.client.Client;
-import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.TopNCondition;
+import org.apache.skywalking.oap.server.core.query.type.SelectedRecord;
+import org.apache.skywalking.oap.server.core.storage.query.ITopNRecordsQueryDAO;
-public class BanyanDBIndexInstaller extends ModelInstaller {
- public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
- super(client, moduleManager);
- }
-
- @Override
- protected boolean isExists(Model model) throws StorageException {
- return false;
- }
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+public class BanyanDBTopNRecordsQueryDAO implements ITopNRecordsQueryDAO {
@Override
- protected void createTable(Model model) throws StorageException {
- StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
- if (metaInfo != null) {
- ((BanyanDBStorageClient) client).createStream(metaInfo);
- }
+ public List<SelectedRecord> readSampledRecords(TopNCondition condition, String valueColumnName, Duration duration) throws IOException {
+ return Collections.emptyList();
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
index 1fbf2eb636..9dda0ead72 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
@@ -24,7 +24,7 @@ import com.google.protobuf.util.JsonFormat;
import lombok.Builder;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase;
import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
@@ -46,6 +46,8 @@ public class StreamMetaInfo {
public static final String TAG_FAMILY_SEARCHABLE = "searchable";
public static final String TAG_FAMILY_DATA = "data";
+ public static final String ID = "id";
+
private static final Map<String, StreamMetaInfo> STREAMS = new HashMap<>();
private final Model model;
@@ -62,35 +64,33 @@ public class StreamMetaInfo {
private final List<IndexRule> indexRules;
public static StreamMetaInfo addModel(Model model) {
- BanyandbMetadata.Stream pbStream = parseStreamFromJSON(model.getName());
+ BanyandbDatabase.Stream pbStream = parseStreamFromJSON(model.getName());
if (pbStream == null) {
- log.warn("fail to find the stream schema {}", model.getName());
+ log.warn("fail to find stream schema {}", model.getName());
return null;
}
- BanyandbMetadata.Duration duration = pbStream.getOpts().getTtl();
- Duration ttl = fromProtobuf(duration);
+ BanyandbDatabase.Duration duration = pbStream.getOpts().getTtl();
+ Duration ttl = Duration.fromProtobuf(duration);
final Stream stream = new Stream(pbStream.getMetadata().getName(), pbStream.getOpts().getShardNum(), ttl);
List<IndexRule> indexRules = new ArrayList<>();
stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
- for (BanyandbMetadata.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
- final TagFamilySpec tagFamilySpec = new TagFamilySpec(pbTagFamilySpec.getName());
- final boolean needIndexParse = pbTagFamilySpec.getName().equals(TAG_FAMILY_SEARCHABLE);
- for (final BanyandbMetadata.TagSpec pbTagSpec : pbTagFamilySpec.getTagsList()) {
- tagFamilySpec.addTagSpec(parseTagSpec(pbTagSpec));
-
- // if the tag family equals to "searchable", build index rules
- if (needIndexParse) {
- BanyandbMetadata.IndexRule pbIndexRule = parseIndexRulesFromJSON(model.getName(), pbTagSpec.getName());
+
+
+ for (BanyandbDatabase.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
+ final TagFamilySpec tagFamilySpec = TagFamilySpec.fromProtobuf(pbTagFamilySpec);
+ stream.addTagFamilySpec(tagFamilySpec);
+
+ // if the tag family equals to "searchable", build index rules
+ if (tagFamilySpec.getTagFamilyName().equals(TAG_FAMILY_SEARCHABLE)) {
+ for (final TagFamilySpec.TagSpec tagSpec : tagFamilySpec.getTagSpecs()) {
+ BanyandbDatabase.IndexRule pbIndexRule = parseIndexRulesFromJSON(model.getName(), tagSpec.getTagName());
if (pbIndexRule == null) {
- log.warn("fail to find the index rule for {}", pbTagSpec.getName());
+ log.warn("fail to find the index rule for {}", tagSpec.getTagName());
continue;
}
- IndexRule.IndexType indexType = fromProtobuf(pbIndexRule.getType());
- IndexRule.IndexLocation indexLocation = fromProtobuf(pbIndexRule.getLocation());
- IndexRule indexRule = new IndexRule(pbIndexRule.getMetadata().getName(), indexType, indexLocation);
- indexRule.setTags(new ArrayList<>(pbIndexRule.getTagsList()));
+ IndexRule indexRule = IndexRule.fromProtobuf(pbIndexRule);
indexRules.add(indexRule);
}
}
@@ -99,31 +99,14 @@ public class StreamMetaInfo {
return StreamMetaInfo.builder().model(model).stream(stream).indexRules(indexRules).build();
}
- private static TagFamilySpec.TagSpec parseTagSpec(BanyandbMetadata.TagSpec pbTagSpec) {
- switch (pbTagSpec.getType()) {
- case TAG_TYPE_INT:
- return TagFamilySpec.TagSpec.newIntTag(pbTagSpec.getName());
- case TAG_TYPE_INT_ARRAY:
- return TagFamilySpec.TagSpec.newIntArrayTag(pbTagSpec.getName());
- case TAG_TYPE_STRING:
- return TagFamilySpec.TagSpec.newStringTag(pbTagSpec.getName());
- case TAG_TYPE_STRING_ARRAY:
- return TagFamilySpec.TagSpec.newStringArrayTag(pbTagSpec.getName());
- case TAG_TYPE_DATA_BINARY:
- return TagFamilySpec.TagSpec.newBinaryTag(pbTagSpec.getName());
- default:
- throw new IllegalArgumentException("unrecognized tag type");
- }
- }
-
- private static BanyandbMetadata.Stream parseStreamFromJSON(String name) {
+ private static BanyandbDatabase.Stream parseStreamFromJSON(String name) {
try {
InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream("metadata/" + name + ".json");
if (is == null) {
return null;
}
String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
- BanyandbMetadata.Stream.Builder b = BanyandbMetadata.Stream.newBuilder();
+ BanyandbDatabase.Stream.Builder b = BanyandbDatabase.Stream.newBuilder();
JsonFormat.parser().merge(result, b);
return b.build();
} catch (IOException ioEx) {
@@ -132,7 +115,7 @@ public class StreamMetaInfo {
}
}
- private static BanyandbMetadata.IndexRule parseIndexRulesFromJSON(String streamName, String name) {
+ private static BanyandbDatabase.IndexRule parseIndexRulesFromJSON(String streamName, String name) {
try {
InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream(String.join("/",
new String[]{"metadata", "index_rules", streamName, name + ".json"}));
@@ -140,7 +123,7 @@ public class StreamMetaInfo {
return null;
}
String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
- BanyandbMetadata.IndexRule.Builder b = BanyandbMetadata.IndexRule.newBuilder();
+ BanyandbDatabase.IndexRule.Builder b = BanyandbDatabase.IndexRule.newBuilder();
JsonFormat.parser().merge(result, b);
return b.build();
} catch (IOException ioEx) {
@@ -148,44 +131,4 @@ public class StreamMetaInfo {
return null;
}
}
-
- // TODO: change modifier to public in SDK
- static Duration fromProtobuf(BanyandbMetadata.Duration duration) {
- switch (duration.getUnit()) {
- case DURATION_UNIT_DAY:
- return Duration.ofDays(duration.getVal());
- case DURATION_UNIT_HOUR:
- return Duration.ofHours(duration.getVal());
- case DURATION_UNIT_MONTH:
- return Duration.ofMonths(duration.getVal());
- case DURATION_UNIT_WEEK:
- return Duration.ofWeeks(duration.getVal());
- default:
- throw new IllegalArgumentException("unrecognized DurationUnit");
- }
- }
-
- // TODO: change modifier to public in SDK
- private static IndexRule.IndexType fromProtobuf(BanyandbMetadata.IndexRule.Type type) {
- switch (type) {
- case TYPE_TREE:
- return IndexRule.IndexType.TREE;
- case TYPE_INVERTED:
- return IndexRule.IndexType.INVERTED;
- default:
- throw new IllegalArgumentException("unrecognized index type");
- }
- }
-
- // TODO: change modifier to public in SDK
- private static IndexRule.IndexLocation fromProtobuf(BanyandbMetadata.IndexRule.Location loc) {
- switch (loc) {
- case LOCATION_GLOBAL:
- return IndexRule.IndexLocation.GLOBAL;
- case LOCATION_SERIES:
- return IndexRule.IndexLocation.SERIES;
- default:
- throw new IllegalArgumentException("unrecognized index location");
- }
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
index a8a2863306..260fc17664 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
@@ -33,8 +33,8 @@ public class AlarmRecordBuilder extends BanyanDBStorageDataBuilder<AlarmRecord>
);
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(AlarmRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(AlarmRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
searchable.add(TagAndValue.longField(entity.getScope()));
searchable.add(TagAndValue.longField(entity.getStartTime()));
searchable.addAll(filterSearchableTags(entity.getTags(), INDEXED_TAGS));
@@ -42,8 +42,8 @@ public class AlarmRecordBuilder extends BanyanDBStorageDataBuilder<AlarmRecord>
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(AlarmRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>(6);
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(AlarmRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(6);
data.add(TagAndValue.stringField(entity.getName()));
data.add(TagAndValue.stringField(entity.getId0()));
data.add(TagAndValue.stringField(entity.getId1()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
index 03ee0328f3..814f3149f7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
@@ -55,17 +55,17 @@ public abstract class BanyanDBStorageDataBuilder<T extends StorageData> implemen
return null;
}
- protected List<SerializableTag<Banyandb.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
if (rawTags == null) {
return Collections.emptyList();
}
- Map<String, SerializableTag<Banyandb.TagValue>> map = new HashMap<>();
+ Map<String, SerializableTag<BanyandbModel.TagValue>> map = new HashMap<>();
for (final Tag tag : rawTags) {
map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
}
- final List<SerializableTag<Banyandb.TagValue>> tags = new ArrayList<>();
+ final List<SerializableTag<BanyandbModel.TagValue>> tags = new ArrayList<>();
for (String indexedTag : indexTags) {
- SerializableTag<Banyandb.TagValue> tag = map.get(indexedTag);
+ SerializableTag<BanyandbModel.TagValue> tag = map.get(indexedTag);
if (tag == null) {
tags.add(TagAndValue.nullField());
} else {
@@ -80,9 +80,9 @@ public abstract class BanyanDBStorageDataBuilder<T extends StorageData> implemen
return entity.id();
}
- abstract protected List<SerializableTag<Banyandb.TagValue>> searchableTags(T entity);
+ abstract protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(T entity);
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(T entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(T entity) {
return Collections.emptyList();
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
index 98b9bd0e6b..034565249b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
@@ -29,8 +29,8 @@ import java.util.List;
public class BrowserErrorLogRecordBuilder extends BanyanDBStorageDataBuilder<BrowserErrorLogRecord> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(BrowserErrorLogRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>();
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(BrowserErrorLogRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
searchable.add(TagAndValue.stringField(entity.getUniqueId()));
searchable.add(TagAndValue.stringField(entity.getServiceId()));
searchable.add(TagAndValue.stringField(entity.getServiceVersionId()));
@@ -40,7 +40,7 @@ public class BrowserErrorLogRecordBuilder extends BanyanDBStorageDataBuilder<Bro
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(BrowserErrorLogRecord entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(BrowserErrorLogRecord entity) {
return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
similarity index 57%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
index 83bf0abfab..ec25c79e10 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
@@ -18,27 +18,27 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
+public class EndpointRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<EndpointRelationServerSideMetrics> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointRelationServerSideMetrics entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getSourceEndpoint()));
+ searchable.add(TagAndValue.stringField(entity.getDestEndpoint()));
+ searchable.add(TagAndValue.stringField(entity.getEntityId()));
+ return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getAddress()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(EndpointRelationServerSideMetrics entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
index bc604776c9..512fe2aa5c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.source.Event;
@@ -29,8 +29,8 @@ import java.util.List;
public class EventBuilder extends BanyanDBStorageDataBuilder<Event> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(Event entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(8);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(Event entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(8);
searchable.add(TagAndValue.stringField(entity.getUuid()));
searchable.add(TagAndValue.stringField(entity.getService()));
searchable.add(TagAndValue.stringField(entity.getServiceInstance()));
@@ -43,7 +43,7 @@ public class EventBuilder extends BanyanDBStorageDataBuilder<Event> {
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(Event entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(Event entity) {
return ImmutableList.of(
TagAndValue.stringField(entity.getMessage()),
TagAndValue.stringField(entity.getParameters())
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
index 02809d68aa..8fddd927a4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
@@ -33,8 +33,8 @@ public class LogRecordBuilder extends BanyanDBStorageDataBuilder<LogRecord> {
);
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(LogRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>();
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(LogRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
searchable.add(TagAndValue.stringField(entity.getUniqueId()));
searchable.add(TagAndValue.stringField(entity.getServiceId()));
searchable.add(TagAndValue.stringField(entity.getServiceInstanceId()));
@@ -48,7 +48,7 @@ public class LogRecordBuilder extends BanyanDBStorageDataBuilder<LogRecord> {
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(LogRecord entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(LogRecord entity) {
return ImmutableList.of(
TagAndValue.stringField(entity.getContent()),
TagAndValue.longField(entity.getContentType()),
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
index 66f3218965..545cd507d6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
@@ -18,9 +18,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.Layer;
import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
@@ -29,39 +30,68 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
+import static org.apache.skywalking.oap.server.core.Const.DOUBLE_COLONS_SPLIT;
+
public class Metadata {
public static class ServiceTrafficBuilder extends BanyanDBStorageDataBuilder<ServiceTraffic> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ServiceTraffic entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(3);
- searchable.add(TagAndValue.stringField(entity.getName()));
- searchable.add(TagAndValue.longField(entity.getNodeType().value()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceTraffic entity) {
+ final String serviceName = entity.getName();
+ entity.setShortName(serviceName);
+ if (entity.isNormal()) {
+ int groupIdx = serviceName.indexOf(DOUBLE_COLONS_SPLIT);
+ if (groupIdx > 0) {
+ entity.setGroup(serviceName.substring(0, groupIdx));
+ entity.setShortName(serviceName.substring(groupIdx + 2));
+ }
+ }
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(4);
+ // 0 - serviceName
+ searchable.add(TagAndValue.stringField(serviceName));
+ // 1 - serviceID
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ // 2 - layer
+ Layer layer = entity.getLayer();
+ searchable.add(TagAndValue.longField(layer != null ? layer.value() : Layer.UNDEFINED.value()));
+ // 3 - group
searchable.add(TagAndValue.stringField(entity.getGroup()));
return searchable;
}
+
+ @Override
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceTraffic entity) {
+ // shortName
+ return Collections.singletonList(TagAndValue.stringField(entity.getShortName()));
+ }
}
public static class EndpointTrafficBuilder extends BanyanDBStorageDataBuilder<EndpointTraffic> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(EndpointTraffic entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointTraffic entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
+ // 0 - name
searchable.add(TagAndValue.stringField(entity.getName()));
+ // 1 - serviceID
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
return searchable;
}
}
public static class InstanceTrafficBuilder extends BanyanDBStorageDataBuilder<InstanceTraffic> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(InstanceTraffic entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(InstanceTraffic entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
+ // serviceID
searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ // lastPingTimestamp
searchable.add(TagAndValue.longField(entity.getLastPingTimestamp()));
+ // ID: we have to duplicate "ID" here for query
+ searchable.add(TagAndValue.stringField(entity.id()));
return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(InstanceTraffic entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(InstanceTraffic entity) {
return Collections.singletonList(TagAndValue.binaryField(
entity.serialize().build().toByteArray()
));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
index 83bf0abfab..ee85a50b03 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
@@ -29,13 +29,13 @@ import java.util.List;
public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(NetworkAddressAlias entity) {
return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(NetworkAddressAlias entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
data.add(TagAndValue.stringField(entity.getAddress()));
data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
index 888cac6b10..a0cab9c112 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
@@ -29,13 +29,13 @@ import java.util.List;
public class ProfileTaskLogRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskLogRecord> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
return Collections.singletonList(TagAndValue.longField(entity.getOperationTime()));
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(ProfileTaskLogRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ProfileTaskLogRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
data.add(TagAndValue.stringField(entity.getTaskId()));
data.add(TagAndValue.stringField(entity.getInstanceId()));
data.add(TagAndValue.longField(entity.getOperationType()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
index b107cadc31..bc91290976 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
@@ -28,8 +28,8 @@ import java.util.List;
public class ProfileTaskRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskRecord> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(9);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(9);
searchable.add(TagAndValue.stringField(entity.id()));
searchable.add(TagAndValue.stringField(entity.getServiceId()));
searchable.add(TagAndValue.stringField(entity.getEndpointName()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
index 10032246ec..28c0ba5f38 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
@@ -29,8 +29,8 @@ import java.util.List;
public class ProfileThreadSnapshotRecordBuilder extends BanyanDBStorageDataBuilder<ProfileThreadSnapshotRecord> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileThreadSnapshotRecord entity) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(4);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileThreadSnapshotRecord entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(4);
searchable.add(TagAndValue.stringField(entity.getTaskId()));
searchable.add(TagAndValue.stringField(entity.getSegmentId()));
searchable.add(TagAndValue.longField(entity.getDumpTime()));
@@ -39,7 +39,7 @@ public class ProfileThreadSnapshotRecordBuilder extends BanyanDBStorageDataBuild
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(ProfileThreadSnapshotRecord entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ProfileThreadSnapshotRecord entity) {
return Collections.singletonList(TagAndValue.binaryField(entity.getStackBinary()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
index a87dc9aa86..0aa54cd3c1 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
@@ -40,8 +40,8 @@ public class SegmentRecordBuilder extends BanyanDBStorageDataBuilder<SegmentReco
);
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(SegmentRecord segmentRecord) {
- List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(10);
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(SegmentRecord segmentRecord) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(10);
searchable.add(TagAndValue.stringField(segmentRecord.getTraceId()));
searchable.add(TagAndValue.stringField(segmentRecord.getServiceId()));
searchable.add(TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
@@ -54,7 +54,7 @@ public class SegmentRecordBuilder extends BanyanDBStorageDataBuilder<SegmentReco
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(SegmentRecord entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(SegmentRecord entity) {
return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
similarity index 52%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
index 83bf0abfab..e940f6d291 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
@@ -18,27 +18,29 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
+public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationClientSideMetrics> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationClientSideMetrics entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(entity.getEntityId()));
+ return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getAddress()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationClientSideMetrics entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
similarity index 52%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
index 83bf0abfab..43495bc148 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
@@ -18,27 +18,29 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
+public class ServiceInstanceRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationServerSideMetrics> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationServerSideMetrics entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(entity.getEntityId()));
+ return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getAddress()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationServerSideMetrics entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
similarity index 57%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
index 83bf0abfab..3d50dde043 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
@@ -18,27 +18,27 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
+public class ServiceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceRelationClientSideMetrics> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceRelationClientSideMetrics entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getEntityId()));
+ return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getAddress()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceRelationClientSideMetrics entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java
similarity index 57%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java
index 83bf0abfab..92a9a13fb9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java
@@ -18,27 +18,27 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
+public class ServiceRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceRelationServerSideMetrics> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceRelationServerSideMetrics entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getEntityId()));
+ return searchable;
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getAddress()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceRelationServerSideMetrics entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
index 41104ee588..6b53a63174 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
@@ -29,7 +29,7 @@ import java.util.List;
public class UITemplateBuilder extends BanyanDBStorageDataBuilder<UITemplate> {
@Override
- protected List<SerializableTag<Banyandb.TagValue>> searchableTags(UITemplate entity) {
+ protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(UITemplate entity) {
return ImmutableList.of(
TagAndValue.stringField(entity.getName()),
TagAndValue.longField(entity.getDisabled())
@@ -37,8 +37,8 @@ public class UITemplateBuilder extends BanyanDBStorageDataBuilder<UITemplate> {
}
@Override
- protected List<SerializableTag<Banyandb.TagValue>> dataTags(UITemplate entity) {
- List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(UITemplate entity) {
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
data.add(TagAndValue.stringField(entity.getType()));
data.add(TagAndValue.stringField(entity.getConfiguration()));
data.add(TagAndValue.longField(entity.getActivated()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBHistoryDeleteDAO.java
similarity index 53%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBHistoryDeleteDAO.java
index ffc5aaf586..0133d30e3d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBHistoryDeleteDAO.java
@@ -16,29 +16,16 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.oap.server.core.storage.StorageException;
+import org.apache.skywalking.oap.server.core.storage.IHistoryDeleteDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
-import org.apache.skywalking.oap.server.library.client.Client;
-import org.apache.skywalking.oap.server.library.module.ModuleManager;
-public class BanyanDBIndexInstaller extends ModelInstaller {
- public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
- super(client, moduleManager);
- }
-
- @Override
- protected boolean isExists(Model model) throws StorageException {
- return false;
- }
+import java.io.IOException;
+public class BanyanDBHistoryDeleteDAO implements IHistoryDeleteDAO {
@Override
- protected void createTable(Model model) throws StorageException {
- StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
- if (metaInfo != null) {
- ((BanyanDBStorageClient) client).createStream(metaInfo);
- }
+ public void deleteHistory(Model model, String timeBucketColumnName, int ttl) throws IOException {
+ // do thing since we currently do not support deletion
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
index 7e37288428..1990181804 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
@@ -30,6 +30,9 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageC
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
import java.io.IOException;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
import java.util.Collections;
/**
@@ -38,6 +41,9 @@ import java.util.Collections;
* @param <T> The only ManagementData we have now is {@link UITemplate}
*/
public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBanyanDBDAO implements IManagementDAO {
+ private final static long START_TIME_MILLI =
+ ZonedDateTime.of(2022, 1, 1, 0, 0, 0, 0,
+ ZoneOffset.UTC).toInstant().toEpochMilli();
private final BanyanDBStorageDataBuilder<T> storageBuilder;
public BanyanDBManagementDAO(BanyanDBStorageClient client, BanyanDBStorageDataBuilder<T> storageBuilder) {
@@ -50,7 +56,7 @@ public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBan
// ensure only insert once
StreamQueryResponse resp = query(UITemplate.INDEX_NAME,
Collections.singletonList(UITemplate.NAME),
- new TimestampRange(0L, 2L),
+ new TimestampRange(START_TIME_MILLI, Instant.now().toEpochMilli()),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
@@ -65,7 +71,7 @@ public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBan
StreamWrite.StreamWriteBuilder streamWrite = this.storageBuilder
.entity2Storage((T) storageData)
.name(model.getName())
- .timestamp(1L);
+ .timestamp(Instant.now().toEpochMilli());
getClient().write(streamWrite.build());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index fdc1ccb71f..b9fc53258c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -22,19 +22,17 @@ import com.google.common.collect.ImmutableList;
import com.google.gson.JsonElement;
import com.google.protobuf.ByteString;
import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.NodeType;
+import org.apache.skywalking.oap.server.core.analysis.Layer;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
import org.apache.skywalking.oap.server.core.query.enumeration.Language;
import org.apache.skywalking.oap.server.core.query.type.Attribute;
-import org.apache.skywalking.oap.server.core.query.type.Database;
import org.apache.skywalking.oap.server.core.query.type.Endpoint;
import org.apache.skywalking.oap.server.core.query.type.Service;
import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
@@ -42,6 +40,7 @@ import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
import java.io.IOException;
import java.util.Collections;
@@ -61,12 +60,15 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
}
@Override
- public List<Service> getAllServices(String group) throws IOException {
+ public List<Service> listServices(final String layer, final String group) throws IOException {
StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.LAYER, ServiceTraffic.GROUP),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(layer)) {
+ query.appendCondition(eq(ServiceTraffic.LAYER, Layer.valueOf(layer).value()));
+ }
if (StringUtil.isNotEmpty(group)) {
query.appendCondition(eq(ServiceTraffic.GROUP, group));
}
@@ -77,13 +79,13 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
}
@Override
- public List<Service> getAllBrowserServices() throws IOException {
+ public List<Service> getServices(String serviceId) throws IOException {
StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.LAYER, ServiceTraffic.GROUP),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(eq(ServiceTraffic.NODE_TYPE, NodeType.Browser.value()));
+ query.appendCondition(eq(ServiceTraffic.SERVICE_ID, serviceId));
}
});
@@ -91,53 +93,41 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
}
@Override
- public List<Database> getAllDatabases() throws IOException {
- StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME, ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE),
+ public List<ServiceInstance> listInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
+ StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
+ ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(eq(ServiceTraffic.NODE_TYPE, (long) NodeType.Database.value()));
- }
- });
+ query.setDataProjections(Collections.singletonList("data_binary"));
- return resp.getElements().stream().map(new DatabaseDeserializer()).collect(Collectors.toList());
- }
+ final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
+ final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
- @Override
- public List<Service> searchServices(NodeType nodeType, String keyword) throws IOException {
- StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(eq(ServiceTraffic.NODE_TYPE, nodeType.value()));
+ query.appendCondition(gte(InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
+ query.appendCondition(lte(InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
+ query.appendCondition(eq(InstanceTraffic.SERVICE_ID, serviceId));
}
});
- return resp.getElements().stream().map(new ServiceDeserializer())
- .filter(s -> s.getName().contains(keyword)) // TODO: support analyzer in database
- .collect(Collectors.toList());
+ return resp.getElements().stream().map(new ServiceInstanceDeserializer()).collect(Collectors.toList());
}
@Override
- public Service searchService(NodeType nodeType, String serviceCode) throws IOException {
- StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
- new QueryBuilder() {
+ public ServiceInstance getInstance(String instanceId) throws IOException {
+ StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
+ ImmutableList.of(InstanceTraffic.NAME, InstanceTraffic.SERVICE_ID), new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.NAME, serviceCode));
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
- // only get one
- query.setLimit(1);
+ query.appendCondition(eq(StreamMetaInfo.ID, instanceId));
}
});
- return resp.getElements().stream().map(new ServiceDeserializer()).findAny().orElse(null);
+ return resp.getElements().stream().map(new ServiceInstanceDeserializer()).findFirst().orElse(null);
}
@Override
- public List<Endpoint> searchEndpoint(String keyword, String serviceId, int limit) throws IOException {
+ public List<Endpoint> findEndpoint(String keyword, String serviceId, int limit) throws IOException {
StreamQueryResponse resp = query(EndpointTraffic.INDEX_NAME,
ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID), new QueryBuilder() {
@Override
@@ -150,38 +140,6 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
.limit(limit).collect(Collectors.toList());
}
- @Override
- public List<ServiceInstance> getServiceInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
- StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList("data_binary"));
-
- final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
- final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
-
- query.appendCondition(gte(InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
- query.appendCondition(lte(InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
- query.appendCondition(eq(InstanceTraffic.SERVICE_ID, serviceId));
- }
- });
-
- return resp.getElements().stream().map(new ServiceInstanceDeserializer()).collect(Collectors.toList());
- }
-
- public static class DatabaseDeserializer implements RowEntityDeserializer<Database> {
- @Override
- public Database apply(RowEntity row) {
- Database database = new Database();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- database.setId(row.getId());
- database.setName((String) searchable.get(0).getValue());
- return database;
- }
- }
-
public static class EndpointDeserializer implements RowEntityDeserializer<Endpoint> {
@Override
public Endpoint apply(RowEntity row) {
@@ -198,9 +156,13 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
public Service apply(RowEntity row) {
Service service = new Service();
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- service.setId(row.getId());
service.setName((String) searchable.get(0).getValue());
- service.setGroup((String) searchable.get(2).getValue());
+ service.setId((String) searchable.get(1).getValue());
+ String layerName = Layer.valueOf(((Number) searchable.get(2).getValue()).intValue()).name();
+ service.getLayers().add(layerName);
+ service.setGroup((String) searchable.get(3).getValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ service.setShortName((String) data.get(0).getValue());
return service;
}
}
@@ -218,6 +180,7 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
instanceTraffic.deserialize(remoteData);
serviceInstance.setName(instanceTraffic.getName());
serviceInstance.setId(instanceTraffic.getServiceId());
+ serviceInstance.setLayer(instanceTraffic.getLayer().name());
if (instanceTraffic.getProperties() != null) {
for (Map.Entry<String, JsonElement> property : instanceTraffic.getProperties().entrySet()) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
index 98f31ebd7e..3c3b4b4a6c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -23,16 +23,25 @@ import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.analysis.topn.TopN;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.StorageDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
@Slf4j
public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> implements StorageDAO {
public BanyanDBStorageDAO(BanyanDBStorageClient client) {
@@ -41,11 +50,59 @@ public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> imple
@Override
public IMetricsDAO newMetricsDao(StorageBuilder storageBuilder) {
+ // SKIP:
+ // 1. OAL runtime metrics builder
+ // 2. Analysis Function builder
+ if (storageBuilder.getClass().getName().startsWith("org.apache.skywalking.oap.server.core.")) {
+ log.warn("metrics builder {} is not supported yet", storageBuilder.getClass());
+ return new IMetricsDAO() {
+ @Override
+ public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
+ return new InsertRequest() {
+ };
+ }
+
+ @Override
+ public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
+ return new UpdateRequest() {
+ };
+ }
+ };
+ }
return new BanyanDBMetricsDAO<>((BanyanDBStorageDataBuilder<Metrics>) storageBuilder);
}
@Override
public IRecordDAO newRecordDao(StorageBuilder storageBuilder) {
+ try {
+ final Class<?> returnType = storageBuilder.getClass().getDeclaredMethod("storage2Entity", Map.class).getReturnType();
+ // FIXME: this is currently a hack to avoid TopN insertion since we will impl TopN later in BanyanDB side
+ if (TopN.class.isAssignableFrom(returnType)) {
+ return new IRecordDAO() {
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
+ return new InsertRequest() {
+ };
+ }
+ };
+ } else if (returnType.getName().equals("org.apache.skywalking.oap.server.storage.plugin.zipkin.ZipkinSpanRecord")) {
+ // SKIP ZipkinSpanRecord
+ return new IRecordDAO() {
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
+ return new InsertRequest() {
+ };
+ }
+ };
+ }
+ } catch (NoSuchMethodException ex) {
+ log.error("fail to get declared method");
+ }
return new BanyanDBRecordDAO<>((BanyanDBStorageDataBuilder<Record>) storageBuilder);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
new file mode 100644
index 0000000000..f574e9b24a
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.oap.server.core.UnexpectedException;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
+import org.apache.skywalking.oap.server.core.query.type.Call;
+import org.apache.skywalking.oap.server.core.source.DetectPoint;
+import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITopologyQueryDAO {
+ public BanyanDBTopologyQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
+ if (CollectionUtils.isEmpty(serviceIds)) {
+ throw new UnexpectedException("Service id is empty");
+ }
+
+ return loadServiceCalls(
+ ServiceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceRelationServerSideMetrics.SOURCE_SERVICE_ID,
+ ServiceRelationServerSideMetrics.DEST_SERVICE_ID, serviceIds, DetectPoint.SERVER
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
+ if (CollectionUtils.isEmpty(serviceIds)) {
+ throw new UnexpectedException("Service id is empty");
+ }
+
+ return loadServiceCalls(
+ ServiceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceRelationClientSideMetrics.SOURCE_SERVICE_ID,
+ ServiceRelationClientSideMetrics.DEST_SERVICE_ID, serviceIds, DetectPoint.CLIENT
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB) throws IOException {
+ return loadServiceCalls(
+ ServiceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceRelationServerSideMetrics.SOURCE_SERVICE_ID,
+ ServiceRelationServerSideMetrics.DEST_SERVICE_ID, new ArrayList<>(0), DetectPoint.SERVER
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB) throws IOException {
+ return loadServiceCalls(
+ ServiceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceRelationClientSideMetrics.SOURCE_SERVICE_ID,
+ ServiceRelationClientSideMetrics.DEST_SERVICE_ID, new ArrayList<>(0), DetectPoint.CLIENT
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadInstanceRelationDetectedAtServerSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
+ return loadServiceInstanceCalls(
+ ServiceInstanceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceInstanceRelationServerSideMetrics.SOURCE_SERVICE_ID,
+ ServiceInstanceRelationServerSideMetrics.DEST_SERVICE_ID, clientServiceId, serverServiceId,
+ DetectPoint.SERVER
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadInstanceRelationDetectedAtClientSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
+ return loadServiceInstanceCalls(
+ ServiceInstanceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
+ ServiceInstanceRelationClientSideMetrics.SOURCE_SERVICE_ID,
+ ServiceInstanceRelationClientSideMetrics.DEST_SERVICE_ID, clientServiceId, serverServiceId,
+ DetectPoint.CLIENT
+ );
+ }
+
+ @Override
+ public List<Call.CallDetail> loadEndpointRelation(long startTB, long endTB, String destEndpointId) throws IOException {
+ List<Call.CallDetail> calls = loadEndpointFromSide(
+ EndpointRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
+ EndpointRelationServerSideMetrics.SOURCE_ENDPOINT,
+ EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId, false
+ );
+ calls.addAll(
+ loadEndpointFromSide(EndpointRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
+ EndpointRelationServerSideMetrics.SOURCE_ENDPOINT,
+ EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId, true
+ ));
+ return calls;
+ }
+
+ private List<Call.CallDetail> loadServiceCalls(String tableName,
+ long startTB,
+ long endTB,
+ String sourceCName,
+ String destCName,
+ List<String> serviceIds,
+ DetectPoint detectPoint) throws IOException {
+ // TODO: we will impl this method after we support `OR`
+ return Collections.emptyList();
+ }
+
+ private List<Call.CallDetail> loadServiceInstanceCalls(String tableName,
+ long startTB,
+ long endTB,
+ String sourceCName,
+ String descCName,
+ String sourceServiceId,
+ String destServiceId,
+ DetectPoint detectPoint) throws IOException {
+ // TODO: we will impl this method after we support `OR`
+ return Collections.emptyList();
+ }
+
+ private List<Call.CallDetail> loadEndpointFromSide(String tableName,
+ long startTB,
+ long endTB,
+ String sourceCName,
+ String destCName,
+ String id,
+ boolean isSourceId) throws IOException {
+ // TODO: we will impl this method after we support `OR`
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index 1d9f977b0a..7888b0f56e 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -35,6 +35,7 @@ import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.io.IOException;
+import java.time.Instant;
import java.util.List;
import java.util.stream.Collectors;
@@ -79,7 +80,7 @@ public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO impleme
.dataTag(Tag.stringField(uiTemplate.getConfiguration()))
// data - activated
.dataTag(Tag.longField(uiTemplate.getActivated()))
- .timestamp(1L)
+ .timestamp(Instant.now().toEpochMilli())
.elementId(uiTemplate.id())
.build();
getClient().write(request);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
index 662cff2eac..d1b85c2be2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
@@ -1,7 +1,6 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_db.instance",
"group": "default"
},
"tags": [
@@ -10,4 +9,4 @@
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json
index 662cff2eac..479751ba13 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_db.type",
"group": "default"
},
"tags": [
- "db.instance"
+ "db.type"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
similarity index 61%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
index 662cff2eac..9ae7387089 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_duration",
"group": "default"
},
"tags": [
- "db.instance"
+ "duration"
],
- "type": "TYPE_INVERTED",
+ "type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json
index 662cff2eac..ca368266a8 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_endpoint_id",
"group": "default"
},
"tags": [
- "db.instance"
+ "endpoint_id"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json
index 662cff2eac..3b5794ec09 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_http.method",
"group": "default"
},
"tags": [
- "db.instance"
+ "http.method"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
index 662cff2eac..4dca9074a5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_mq.broker",
"group": "default"
},
"tags": [
- "db.instance"
+ "mq.broker"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json
index 662cff2eac..1b3126e056 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_mq.queue",
"group": "default"
},
"tags": [
- "db.instance"
+ "mq.queue"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json
index 662cff2eac..ddae44b1a1 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_mq.topic",
"group": "default"
},
"tags": [
- "db.instance"
+ "mq.topic"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json
index 662cff2eac..3e472a1669 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "segment_status_code",
"group": "default"
},
"tags": [
- "db.instance"
+ "status_code"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json
new file mode 100644
index 0000000000..3ac579999c
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "segment_trace_id",
+ "group": "default"
+ },
+ "tags": [
+ "trace_id"
+ ],
+ "type": "TYPE_TREE",
+ "location": "LOCATION_GLOBAL",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json
index 662cff2eac..9146c34d11 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "ui_template_disabled",
"group": "default"
},
"tags": [
- "db.instance"
+ "disabled"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
index 662cff2eac..067ca7f123 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
@@ -1,13 +1,12 @@
{
"metadata": {
- "id": 1,
- "name": "db.instance",
+ "name": "ui_template_name",
"group": "default"
},
"tags": [
- "db.instance"
+ "name"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
new file mode 100644
index 0000000000..e2a404ff5f
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
@@ -0,0 +1,50 @@
+{
+ "metadata": {
+ "name": "ui_template",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "type",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "configuration",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "activated",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "name",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "disabled",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 52000,
+ "unit": "DURATION_UNIT_WEEK"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
[skywalking] 07/24: register necessary modules
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 32c0b82a5956f7058f1569d636643082039ecc88
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Sun Dec 5 22:50:15 2021 +0800
register necessary modules
---
.../plugin/banyandb/BanyanDBStorageConfig.java | 6 ++-
.../plugin/banyandb/BanyanDBStorageProvider.java | 46 ++++++++++++++++++++++
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
index c37aed1162..b3b2cdad17 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageConfig.java
@@ -31,7 +31,6 @@ public class BanyanDBStorageConfig extends ModuleConfig {
* Group of the schema in BanyanDB
*/
private String group = "default";
-
/**
* The maximum size of write entities in a single batch write call.
*/
@@ -44,4 +43,9 @@ public class BanyanDBStorageConfig extends ModuleConfig {
* Concurrent consumer threads for batch writing.
*/
private int concurrentWriteThreads = 2;
+ /**
+ * Max size of {@link org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog} to be fetched
+ * in a single request.
+ */
+ private int fetchTaskLogMaxSize;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 09b72509a8..1df391d302 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -20,13 +20,39 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
+import org.apache.skywalking.oap.server.core.storage.IBatchDAO;
import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
+import org.apache.skywalking.oap.server.core.storage.StorageDAO;
import org.apache.skywalking.oap.server.core.storage.StorageModule;
+import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
+import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.module.ModuleConfig;
import org.apache.skywalking.oap.server.library.module.ModuleDefine;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
import org.apache.skywalking.oap.server.library.module.ModuleStartException;
import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBAlarmQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBatchDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBrowserLogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEventQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBLogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBMetadataQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBNetworkAddressAliasDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskLogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBStorageDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBTraceQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
import org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics;
import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
@@ -60,6 +86,26 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.registerServiceImplementation(StorageBuilderFactory.class, new StorageBuilderFactory.Default());
this.client = new BanyanDBStorageClient(config.getHost(), config.getPort(), config.getGroup());
+
+ this.registerServiceImplementation(IBatchDAO.class, new BanyanDBBatchDAO(client, config.getMaxBulkSize(), config.getFlushInterval(), config.getConcurrentWriteThreads()));
+ this.registerServiceImplementation(StorageDAO.class, new BanyanDBStorageDAO(client));
+
+ this.registerServiceImplementation(INetworkAddressAliasDAO.class, new BanyanDBNetworkAddressAliasDAO(client));
+
+ this.registerServiceImplementation(ITraceQueryDAO.class, new BanyanDBTraceQueryDAO(client));
+ this.registerServiceImplementation(IBrowserLogQueryDAO.class, new BanyanDBBrowserLogQueryDAO(client));
+ this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO(client));
+ this.registerServiceImplementation(IAlarmQueryDAO.class, new BanyanDBAlarmQueryDAO(client));
+ this.registerServiceImplementation(ILogQueryDAO.class, new BanyanDBLogQueryDAO(client));
+
+ this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO(client));
+ this.registerServiceImplementation(IProfileTaskLogQueryDAO.class, new BanyanDBProfileTaskLogQueryDAO(client, this.config.getFetchTaskLogMaxSize()));
+ this.registerServiceImplementation(
+ IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
+ this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO(client));
+
+ this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO(client));
+
}
@Override
[skywalking] 19/24: fix styles
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit d567b1f143dd060145c15b69eb536fa01bbbb441
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Tue Feb 15 21:57:07 2022 +0800
fix styles
---
.../server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java | 3 ---
1 file changed, 3 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
index a87a94434d..b719090919 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
@@ -21,7 +21,6 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
@@ -31,8 +30,6 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBS
import java.io.IOException;
import java.time.Instant;
-import java.time.ZoneOffset;
-import java.time.ZonedDateTime;
import java.util.Collections;
/**
[skywalking] 22/24: support UI Template
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 6afdd8b39880151b9c69bff43dd7be262dd5c0a1
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Fri Apr 22 11:18:03 2022 +0800
support UI Template
---
.../plugin/banyandb/BanyanDBIndexInstaller.java | 4 +-
.../plugin/banyandb/BanyanDBManagementDAO.java | 4 +-
.../plugin/banyandb/BanyanDBStorageClient.java | 60 +++++++++++-
.../plugin/banyandb/BanyanDBStorageProvider.java | 2 +-
.../measure/BanyanDBUITemplateManagementDAO.java | 108 +++++++++++++++++++--
.../banyandb/stream/AbstractBanyanDBDAO.java | 7 +-
6 files changed, 163 insertions(+), 22 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index d894c9990a..32f3480769 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -31,6 +31,8 @@ import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import java.io.IOException;
+
@Slf4j
public class BanyanDBIndexInstaller extends ModelInstaller {
private final ConfigService configService;
@@ -75,7 +77,7 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
} else if (!model.isTimeSeries()) { // UITemplate
log.info("skip property index {}", model.getName());
}
- } catch (BanyanDBException ex) {
+ } catch (IOException ex) {
throw new StorageException("fail to install schema", ex);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java
index 080bf4577a..44a5f91874 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java
@@ -1,6 +1,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
@@ -9,12 +10,13 @@ import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import java.io.IOException;
@RequiredArgsConstructor
+@Slf4j
public class BanyanDBManagementDAO implements IManagementDAO {
private final BanyanDBStorageClient client;
private final StorageBuilder<ManagementData> storageBuilder;
@Override
public void insert(Model model, ManagementData storageData) throws IOException {
-
+ log.info("insert Model {}", model);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index dd92c6fd08..2df53a2ed9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -18,12 +18,14 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+import io.grpc.Status;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
+import org.apache.skywalking.banyandb.v1.client.metadata.Property;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.client.healthcheck.DelegatedHealthChecker;
@@ -31,6 +33,8 @@ import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckab
import org.apache.skywalking.oap.server.library.util.HealthChecker;
import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
/**
* BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
@@ -54,19 +58,67 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
this.client.close();
}
- public StreamQueryResponse query(StreamQuery streamQuery) throws BanyanDBException {
+ public List<Property> listProperties(String group, String name) throws IOException {
+ try {
+ List<Property> properties = this.client.findProperties(group, name);
+ this.healthChecker.health();
+ return properties;
+ } catch (BanyanDBException ex) {
+ if (ex.getStatus().equals(Status.Code.NOT_FOUND)) {
+ this.healthChecker.health();
+ return Collections.emptyList();
+ }
+
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to list properties", ex);
+ }
+ }
+
+ public Property queryProperty(String group, String name, String id) throws IOException {
+ try {
+ Property p = this.client.findProperty(group, name, id);
+ this.healthChecker.health();
+ return p;
+ } catch (BanyanDBException ex) {
+ if (ex.getStatus().equals(Status.Code.NOT_FOUND)) {
+ this.healthChecker.health();
+ return null;
+ }
+
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to query property", ex);
+ }
+ }
+
+ public StreamQueryResponse query(StreamQuery streamQuery) throws IOException {
try {
StreamQueryResponse response = this.client.query(streamQuery);
this.healthChecker.health();
return response;
} catch (BanyanDBException ex) {
healthChecker.unHealth(ex);
- throw ex;
+ throw new IOException("fail to query stream", ex);
+ }
+ }
+
+ public void define(Property property) throws IOException {
+ try {
+ this.client.save(property);
+ this.healthChecker.health();
+ } catch (BanyanDBException ex) {
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to define property", ex);
}
}
- public void define(Stream stream) throws BanyanDBException {
- this.client.define(stream);
+ public void define(Stream stream) throws IOException {
+ try {
+ this.client.define(stream);
+ this.healthChecker.health();
+ } catch (BanyanDBException ex) {
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to define stream", ex);
+ }
}
public void write(StreamWrite streamWrite) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 88a81d3372..26c0e8d501 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -115,7 +115,7 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO());
this.registerServiceImplementation(IProfileTaskLogQueryDAO.class, new BanyanDBProfileTaskLogQueryDAO(client, this.config.getFetchTaskLogMaxSize()));
this.registerServiceImplementation(IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
- this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO());
+ this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO(client));
this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO());
this.registerServiceImplementation(ITopologyQueryDAO.class, new BanyanDBTopologyQueryDAO());
this.registerServiceImplementation(IEBPFProfilingTaskDAO.class, new BanyanDBEBPFProfilingTaskDAO());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
index efbd9ce106..dba7844413 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
@@ -18,38 +18,128 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.metadata.Property;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.AbstractBanyanDBDAO;
import java.io.IOException;
-import java.util.Collections;
import java.util.List;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO implements UITemplateManagementDAO {
+ private static final String GROUP = "sw";
+
+ public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
-public class BanyanDBUITemplateManagementDAO implements UITemplateManagementDAO {
@Override
public DashboardConfiguration getTemplate(String id) throws IOException {
- return null;
+ Property p = getClient().queryProperty(GROUP, UITemplate.INDEX_NAME, id);
+ if (p == null) {
+ return null;
+ }
+ return fromEntity(parse(p));
}
@Override
public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- return Collections.emptyList();
+ List<Property> propertyList = getClient().listProperties(GROUP, UITemplate.INDEX_NAME);
+ return propertyList.stream().map(p -> fromEntity(parse(p)))
+ .filter(conf -> includingDisabled || !conf.isDisabled())
+ .collect(Collectors.toList());
}
@Override
- public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
- return null;
+ public TemplateChangeStatus addTemplate(DashboardSetting setting) {
+ Property newTemplate = convert(setting.toEntity());
+ try {
+ this.getClient().define(newTemplate);
+ return TemplateChangeStatus.builder()
+ .status(true)
+ .id(newTemplate.id())
+ .build();
+ } catch (IOException ioEx) {
+ log.error("fail to add new template", ioEx);
+ return TemplateChangeStatus.builder().status(false).id(setting.getId()).message("Can't add a new template")
+ .build();
+ }
}
@Override
- public TemplateChangeStatus changeTemplate(DashboardSetting setting) throws IOException {
- return null;
+ public TemplateChangeStatus changeTemplate(DashboardSetting setting) {
+ Property newTemplate = convert(setting.toEntity());
+ try {
+ this.getClient().define(newTemplate);
+ return TemplateChangeStatus.builder()
+ .status(true)
+ .id(newTemplate.id())
+ .build();
+ } catch (IOException ioEx) {
+ log.error("fail to modify the template", ioEx);
+ return TemplateChangeStatus.builder().status(false).id(setting.getId()).message("Can't change an existed template")
+ .build();
+ }
}
@Override
public TemplateChangeStatus disableTemplate(String id) throws IOException {
- return null;
+ Property oldProperty = this.getClient().queryProperty(GROUP, UITemplate.INDEX_NAME, id);
+ if (oldProperty == null) {
+ return TemplateChangeStatus.builder().status(false).id(id).message("Can't find the template")
+ .build();
+ }
+ UITemplate uiTemplate = parse(oldProperty);
+ uiTemplate.setDisabled(BooleanUtils.FALSE);
+ try {
+ this.getClient().define(convert(uiTemplate));
+ return TemplateChangeStatus.builder()
+ .status(true)
+ .id(uiTemplate.id())
+ .build();
+ } catch (IOException ioEx) {
+ log.error("fail to disable the template", ioEx);
+ return TemplateChangeStatus.builder().status(false).id(uiTemplate.id()).message("Can't disable the template")
+ .build();
+ }
+ }
+
+ public DashboardConfiguration fromEntity(UITemplate uiTemplate) {
+ DashboardConfiguration conf = new DashboardConfiguration();
+ conf.fromEntity(uiTemplate);
+ return conf;
+ }
+
+ public UITemplate parse(Property property) {
+ UITemplate uiTemplate = new UITemplate();
+ uiTemplate.setTemplateId(property.id());
+
+ for (TagAndValue<?> tagAndValue : property.tags()) {
+ if (tagAndValue.getTagName().equals(UITemplate.CONFIGURATION)) {
+ uiTemplate.setConfiguration((String) tagAndValue.getValue());
+ } else if (tagAndValue.getTagName().equals(UITemplate.DISABLED)) {
+ uiTemplate.setDisabled(((Number) tagAndValue.getValue()).intValue());
+ } else if (tagAndValue.getTagName().equals(UITemplate.UPDATE_TIME)) {
+ uiTemplate.setUpdateTime(((Number) tagAndValue.getValue()).longValue());
+ }
+ }
+ return uiTemplate;
+ }
+
+ public Property convert(UITemplate uiTemplate) {
+ return Property.create(GROUP, UITemplate.INDEX_NAME, uiTemplate.id())
+ .addTag(TagAndValue.newStringTag(UITemplate.CONFIGURATION, uiTemplate.getConfiguration()))
+ .addTag(TagAndValue.newLongTag(UITemplate.DISABLED, uiTemplate.getDisabled()))
+ .addTag(TagAndValue.newLongTag(UITemplate.UPDATE_TIME, uiTemplate.getUpdateTime()))
+ .build();
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index c40ac35924..53e8cc4f80 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -22,7 +22,6 @@ import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
-import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
@@ -55,11 +54,7 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
builder.apply(query);
- try {
- return getClient().query(query);
- } catch (BanyanDBException ex) {
- throw new IOException(ex);
- }
+ return getClient().query(query);
}
protected abstract static class QueryBuilder {
[skywalking] 11/24: bump version to 9.0.0-SNAPSHOT
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 773307e9a8dc62534b37c9ef0518894660aa0b78
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 9 18:24:23 2021 +0800
bump version to 9.0.0-SNAPSHOT
---
oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
index d5ebbfc969..e2c9ae7817 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
@@ -21,7 +21,7 @@
<parent>
<artifactId>server-storage-plugin</artifactId>
<groupId>org.apache.skywalking</groupId>
- <version>8.10.0-SNAPSHOT</version>
+ <version>9.0.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
[skywalking] 20/24: remove all hardcoded builder
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit bd50d4afd0da8a33261498114d1fefa755f42ef7
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Mon Mar 21 23:39:23 2022 +0800
remove all hardcoded builder
---
.../storage/plugin/banyandb/BanyanDBConverter.java | 139 +++++++++++
.../plugin/banyandb/BanyanDBIndexInstaller.java | 28 ++-
.../plugin/banyandb/BanyanDBManagementDAO.java | 20 ++
.../plugin/banyandb/BanyanDBMetricsDAO.java | 36 +++
.../{stream => }/BanyanDBNoneStreamDAO.java | 32 +--
.../banyandb/BanyanDBStorageBuilderFactory.java | 112 ---------
.../plugin/banyandb/BanyanDBStorageClient.java | 24 +-
.../plugin/banyandb/BanyanDBStorageProvider.java | 31 +--
.../storage/plugin/banyandb/MetadataRegistry.java | 212 ++++++++++++++++
.../storage/plugin/banyandb/StreamMetaInfo.java | 140 -----------
.../storage/plugin/banyandb/StreamMetadata.java | 71 ++++++
.../BanyanDBEventQueryDAO.java} | 27 +-
.../banyandb/measure/BanyanDBMetadataQueryDAO.java | 66 +++++
.../BanyanDBNetworkAddressAliasDAO.java} | 22 +-
.../BanyanDBProfileTaskQueryDAO.java} | 28 ++-
.../banyandb/measure/BanyanDBTopologyQueryDAO.java | 63 +++++
.../measure/BanyanDBUITemplateManagementDAO.java | 55 ++++
.../plugin/banyandb/schema/AlarmRecordBuilder.java | 55 ----
.../schema/BanyanDBStorageDataBuilder.java | 88 -------
.../schema/BrowserErrorLogRecordBuilder.java | 46 ----
.../EndpointRelationServerSideMetricsBuilder.java | 48 ----
.../plugin/banyandb/schema/EventBuilder.java | 52 ----
.../plugin/banyandb/schema/LogRecordBuilder.java | 58 -----
.../storage/plugin/banyandb/schema/Metadata.java | 100 --------
.../schema/NetworkAddressAliasBuilder.java | 49 ----
.../schema/ProfileTaskLogRecordBuilder.java | 45 ----
.../banyandb/schema/ProfileTaskRecordBuilder.java | 53 ----
.../schema/ProfileThreadSnapshotRecordBuilder.java | 45 ----
.../banyandb/schema/SegmentRecordBuilder.java | 68 -----
...ceInstanceRelationClientSideMetricsBuilder.java | 47 ----
...ceInstanceRelationServerSideMetricsBuilder.java | 47 ----
.../ServiceRelationClientSideMetricsBuilder.java | 44 ----
.../ServiceRelationServerSideMetricsBuilder.java | 44 ----
.../plugin/banyandb/schema/UITemplateBuilder.java | 47 ----
.../banyandb/stream/AbstractBanyanDBDAO.java | 25 +-
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 68 ++---
.../stream/BanyanDBBrowserLogQueryDAO.java | 77 +++---
.../banyandb/stream/BanyanDBEventQueryDAO.java | 148 -----------
.../banyandb/stream/BanyanDBLogQueryDAO.java | 83 ++++---
.../banyandb/stream/BanyanDBManagementDAO.java | 70 ------
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 214 ----------------
.../plugin/banyandb/stream/BanyanDBMetricsDAO.java | 57 -----
.../stream/BanyanDBNetworkAddressAliasDAO.java | 76 ------
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 49 ++--
.../stream/BanyanDBProfileTaskQueryDAO.java | 120 ---------
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 146 ++++++-----
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 27 +-
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 69 +-----
.../stream/BanyanDBStreamInsertRequest.java | 2 +-
.../banyandb/stream/BanyanDBTopologyQueryDAO.java | 276 ---------------------
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 105 ++++----
.../stream/BanyanDBUITemplateManagementDAO.java | 119 ---------
52 files changed, 1073 insertions(+), 2700 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
new file mode 100644
index 0000000000..565245ad55
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.storage.type.Convert2Entity;
+import org.apache.skywalking.oap.server.core.storage.type.Convert2Storage;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+import java.util.List;
+import java.util.function.Function;
+
+public class BanyanDBConverter {
+ @RequiredArgsConstructor
+ public static class StreamToEntity implements Convert2Entity {
+ private final StreamMetadata metadata;
+ private final RowEntity rowEntity;
+
+ @Override
+ public Object get(String fieldName) {
+ final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
+ if (metadata == null) {
+ return null;
+ }
+ return rowEntity.getValue(metadata.getTagFamilyName(), metadata.getTagSpec().getTagName());
+ }
+
+ @Override
+ public <T, R> R getWith(String fieldName, Function<T, R> typeDecoder) {
+ return (R) this.get(fieldName);
+ }
+ }
+
+ @RequiredArgsConstructor
+ public static class StreamToStorage implements Convert2Storage<StreamWrite> {
+ private final StreamMetadata metadata;
+ private final StreamWrite streamWrite;
+
+ @Override
+ public void accept(String fieldName, Object fieldValue) {
+ // skip "time_bucket"
+ if (Record.TIME_BUCKET.equals(fieldName)) {
+ return;
+ }
+ final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
+ if (metadata == null) {
+ return;
+ }
+ switch (metadata.getTagFamilyName()) {
+ case StreamMetadata.TAG_FAMILY_DATA:
+ this.streamWrite.dataTag(metadata.getTagIndex(), buildTag(fieldValue));
+ break;
+ case StreamMetadata.TAG_FAMILY_SEARCHABLE:
+ this.streamWrite.searchableTag(metadata.getTagIndex(), buildTag(fieldValue));
+ break;
+ default:
+ throw new IllegalStateException("tag family is not supported");
+ }
+ }
+
+ private SerializableTag<BanyandbModel.TagValue> buildTag(Object value) {
+ if (Integer.class.equals(value.getClass()) || Long.class.equals(value.getClass())) {
+ return TagAndValue.longField((long) value);
+ } else if (String.class.equals(value.getClass())) {
+ return TagAndValue.stringField((String) value);
+ }
+ throw new IllegalStateException(value.getClass() + " is not supported");
+ }
+
+ @Override
+ public void accept(String fieldName, byte[] fieldValue) {
+ final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
+ if (metadata == null) {
+ return;
+ }
+ if (StreamMetadata.TAG_FAMILY_SEARCHABLE.equals(metadata.getTagFamilyName())) {
+ this.streamWrite.searchableTag(metadata.getTagIndex(), TagAndValue.binaryField((fieldValue)));
+ } else {
+ throw new IllegalStateException("binary tag should not be store in the `data` family");
+ }
+ }
+
+ @Override
+ public void accept(String fieldName, List<String> fieldValue) {
+ for (final String tagKeyAndValue : fieldValue) {
+ if (StringUtil.isEmpty(tagKeyAndValue)) {
+ continue;
+ }
+ int pos = tagKeyAndValue.indexOf("=");
+ if (pos == -1) {
+ continue;
+ }
+ String key = tagKeyAndValue.substring(0, pos);
+ String value = tagKeyAndValue.substring(pos + 1);
+ this.accept(key, value);
+ }
+ }
+
+ @Override
+ public Object get(String fieldName) {
+ final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
+ if (metadata == null) {
+ return null;
+ }
+ // TODO: get an unmodifiable view of tag
+ return null;
+ }
+
+ @Override
+ public StreamWrite obtain() {
+ if (metadata.isUseIdAsEntity()) {
+ this.accept(StreamMetadata.ID, this.streamWrite.getElementID());
+ }
+ return this.streamWrite;
+ }
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index c9566b1549..f4483b0a14 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -19,6 +19,11 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
+import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
+import org.apache.skywalking.banyandb.v1.client.metadata.Group;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.StorageException;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
@@ -27,21 +32,36 @@ import org.apache.skywalking.oap.server.library.module.ModuleManager;
@Slf4j
public class BanyanDBIndexInstaller extends ModelInstaller {
+ private final ConfigService configService;
+
public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
super(client, moduleManager);
+ this.configService = moduleManager.find(CoreModule.NAME)
+ .provider()
+ .getService(ConfigService.class);
}
@Override
protected boolean isExists(Model model) throws StorageException {
+ // TODO: get from BanyanDB and make a diff?
return false;
}
@Override
protected void createTable(Model model) throws StorageException {
- StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
- if (metaInfo != null) {
- log.info("install index {}", model.getName());
- ((BanyanDBStorageClient) client).createStream(metaInfo);
+ if (model.isTimeSeries() && model.isRecord()) { // stream
+ StreamMetadata metaInfo = MetadataRegistry.INSTANCE.registerModel(model, this.configService);
+ if (metaInfo != null) {
+ log.info("install index {}", model.getName());
+ ((BanyanDBStorageClient) client).define(
+ new Group(metaInfo.getGroup(), Catalog.STREAM, 2, 10, Duration.ofDays(7))
+ );
+ ((BanyanDBStorageClient) client).define(metaInfo);
+ }
+ } else if (model.isTimeSeries() && !model.isRecord()) { // measure
+ log.info("skip measure index {}", model.getName());
+ } else if (!model.isTimeSeries()) { // UITemplate
+ log.info("skip property index {}", model.getName());
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java
new file mode 100644
index 0000000000..080bf4577a
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBManagementDAO.java
@@ -0,0 +1,20 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
+import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+
+import java.io.IOException;
+
+@RequiredArgsConstructor
+public class BanyanDBManagementDAO implements IManagementDAO {
+ private final BanyanDBStorageClient client;
+ private final StorageBuilder<ManagementData> storageBuilder;
+
+ @Override
+ public void insert(Model model, ManagementData storageData) throws IOException {
+
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
new file mode 100644
index 0000000000..af54dd350f
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
@@ -0,0 +1,36 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+@RequiredArgsConstructor
+public class BanyanDBMetricsDAO implements IMetricsDAO {
+ private final StorageBuilder<Metrics> storageBuilder;
+
+ @Override
+ public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
+ return new InsertRequest() {
+ };
+ }
+
+ @Override
+ public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
+ return new UpdateRequest() {
+ };
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBNoneStreamDAO.java
similarity index 51%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBNoneStreamDAO.java
index ae5c766897..bd4a5b0926 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBNoneStreamDAO.java
@@ -16,36 +16,26 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import java.io.IOException;
-/**
- * DAO for NoneStream, specifically ProfileTaskRecord
- *
- * @param <T> For NoneStream, we only have {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord}
- */
-@RequiredArgsConstructor
-public class BanyanDBNoneStreamDAO<T extends NoneStream> implements INoneStreamDAO {
- private final BanyanDBStorageClient client;
- private final BanyanDBStorageDataBuilder<T> storageBuilder;
+public class BanyanDBNoneStreamDAO extends AbstractDAO<BanyanDBStorageClient> implements INoneStreamDAO {
+ private final StorageBuilder<NoneStream> storageBuilder;
+
+ public BanyanDBNoneStreamDAO(BanyanDBStorageClient client, StorageBuilder<NoneStream> storageBuilder) {
+ super(client);
+ this.storageBuilder = storageBuilder;
+ }
@Override
public void insert(Model model, NoneStream noneStream) throws IOException {
- final long timestamp = TimeBucket.getTimestamp(noneStream.getTimeBucket(), model.getDownsampling());
- StreamWrite.StreamWriteBuilder builder =
- this.storageBuilder.entity2Storage((T) noneStream)
- .name(model.getName())
- .timestamp(timestamp);
- this.client.write(builder.build());
+
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
deleted file mode 100644
index 0cb10763eb..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
+++ /dev/null
@@ -1,112 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-
-import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-import org.apache.skywalking.oap.server.core.source.Event;
-import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
-import org.apache.skywalking.oap.server.core.storage.StorageData;
-import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
-import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BrowserErrorLogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EndpointRelationServerSideMetricsBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EventBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.Metadata;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.NetworkAddressAliasBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskLogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileThreadSnapshotRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceInstanceRelationClientSideMetricsBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceInstanceRelationServerSideMetricsBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceRelationClientSideMetricsBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ServiceRelationServerSideMetricsBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
-
-@Slf4j
-public class BanyanDBStorageBuilderFactory implements StorageBuilderFactory {
- private static final StorageBuilderFactory FALLBACK = new StorageBuilderFactory.Default();
-
- @Override
- public BuilderTemplateDefinition builderTemplate() {
- return new BuilderTemplateDefinition(StorageHashMapBuilder.class.getName(), "metrics-builder");
- }
-
- @Override
- public Class<? extends StorageBuilder> builderOf(Class<? extends StorageData> dataType, Class<? extends StorageBuilder> defaultBuilder) {
- if (SegmentRecord.class.equals(dataType)) {
- return SegmentRecordBuilder.class;
- } else if (AlarmRecord.class.equals(dataType)) {
- return AlarmRecordBuilder.class;
- } else if (BrowserErrorLogRecord.class.equals(dataType)) {
- return BrowserErrorLogRecordBuilder.class;
- } else if (LogRecord.class.equals(dataType)) {
- return LogRecordBuilder.class;
- } else if (ProfileTaskLogRecord.class.equals(dataType)) {
- return ProfileTaskLogRecordBuilder.class;
- } else if (ProfileThreadSnapshotRecord.class.equals(dataType)) {
- return ProfileThreadSnapshotRecordBuilder.class;
- } else if (ProfileTaskRecord.class.equals(dataType)) {
- return ProfileTaskRecordBuilder.class;
- } else if (UITemplate.class.equals(dataType)) {
- return UITemplateBuilder.class;
- } else if (Event.class.equals(dataType)) {
- return EventBuilder.class;
- } else if (ServiceTraffic.class.equals(dataType)) {
- return Metadata.ServiceTrafficBuilder.class;
- } else if (InstanceTraffic.class.equals(dataType)) {
- return Metadata.InstanceTrafficBuilder.class;
- } else if (EndpointTraffic.class.equals(dataType)) {
- return Metadata.EndpointTrafficBuilder.class;
- } else if (NetworkAddressAlias.class.equals(dataType)) {
- return NetworkAddressAliasBuilder.class;
- } else if (EndpointRelationServerSideMetrics.class.equals(dataType)) {
- return EndpointRelationServerSideMetricsBuilder.class;
- } else if (ServiceRelationServerSideMetrics.class.equals(dataType)) {
- return ServiceRelationServerSideMetricsBuilder.class;
- } else if (ServiceRelationClientSideMetrics.class.equals(dataType)) {
- return ServiceRelationClientSideMetricsBuilder.class;
- } else if (ServiceInstanceRelationServerSideMetrics.class.equals(dataType)) {
- return ServiceInstanceRelationServerSideMetricsBuilder.class;
- } else if (ServiceInstanceRelationClientSideMetrics.class.equals(dataType)) {
- return ServiceInstanceRelationClientSideMetricsBuilder.class;
- }
-
- return FALLBACK.builderOf(dataType, defaultBuilder);
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 6b4780cad7..b1ca206110 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -19,7 +19,6 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
-import org.apache.skywalking.banyandb.v1.client.GroupedBanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
@@ -33,6 +32,8 @@ import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckab
import org.apache.skywalking.oap.server.library.util.HealthChecker;
import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
/**
* BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
@@ -40,15 +41,16 @@ import java.io.IOException;
*/
public class BanyanDBStorageClient implements Client, HealthCheckable {
private final BanyanDBClient client;
- private GroupedBanyanDBClient streamClient;
+ private final Map<String, Group> groupMap;
private final DelegatedHealthChecker healthChecker = new DelegatedHealthChecker();
public BanyanDBStorageClient(String host, int port) {
this.client = new BanyanDBClient(host, port);
+ this.groupMap = new ConcurrentHashMap<>();
}
- public void defineStreamGroup(Group group) {
- this.streamClient = this.client.attachGroup(group);
+ public Group define(Group group) {
+ return groupMap.computeIfAbsent(group.getName(), s -> client.define(group));
}
@Override
@@ -63,7 +65,7 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
public StreamQueryResponse query(StreamQuery streamQuery) {
try {
- StreamQueryResponse response = this.streamClient.queryStreams(streamQuery);
+ StreamQueryResponse response = this.client.queryStreams(streamQuery);
this.healthChecker.health();
return response;
} catch (Throwable t) {
@@ -72,19 +74,19 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
- public void createStream(StreamMetaInfo streamMetaInfo) {
- Stream stm = this.streamClient.define(streamMetaInfo.getStream());
- if (stm != null) {
- this.streamClient.defineIndexRules(stm, streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
+ public void define(StreamMetadata streamMetadata) {
+ Stream stream = this.client.define(streamMetadata.getStream());
+ if (stream != null) {
+ this.client.defineIndexRules(stream, streamMetadata.getIndexRules().toArray(new IndexRule[]{}));
}
}
public void write(StreamWrite streamWrite) {
- this.streamClient.write(streamWrite);
+ this.client.write(streamWrite);
}
public StreamBulkWriteProcessor createBulkProcessor(int maxBulkSize, int flushInterval, int concurrency) {
- return this.streamClient.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
+ return this.client.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index b938ab1159..922bf2296a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -18,8 +18,6 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
-import org.apache.skywalking.banyandb.v1.client.metadata.Group;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.IBatchDAO;
@@ -48,21 +46,21 @@ import org.apache.skywalking.oap.server.library.module.ModuleDefine;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
import org.apache.skywalking.oap.server.library.module.ModuleStartException;
import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBEventQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBMetadataQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBNetworkAddressAliasDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBTopologyQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBAlarmQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBatchDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBrowserLogQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEventQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBHistoryDeleteDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBLogQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBMetadataQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBNetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskLogQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBStorageDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBTopologyQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBTraceQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
import org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics;
import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
@@ -93,26 +91,26 @@ public class BanyanDBStorageProvider extends ModuleProvider {
@Override
public void prepare() throws ServiceNotProvidedException, ModuleStartException {
- this.registerServiceImplementation(StorageBuilderFactory.class, new BanyanDBStorageBuilderFactory());
+ this.registerServiceImplementation(StorageBuilderFactory.class, new StorageBuilderFactory.Default());
this.client = new BanyanDBStorageClient(config.getHost(), config.getPort());
// Stream
this.registerServiceImplementation(IBatchDAO.class, new BanyanDBBatchDAO(client, config.getMaxBulkSize(), config.getFlushInterval(), config.getConcurrentWriteThreads()));
this.registerServiceImplementation(StorageDAO.class, new BanyanDBStorageDAO(client));
- this.registerServiceImplementation(INetworkAddressAliasDAO.class, new BanyanDBNetworkAddressAliasDAO(client));
+ this.registerServiceImplementation(INetworkAddressAliasDAO.class, new BanyanDBNetworkAddressAliasDAO());
this.registerServiceImplementation(ITraceQueryDAO.class, new BanyanDBTraceQueryDAO(client));
this.registerServiceImplementation(IBrowserLogQueryDAO.class, new BanyanDBBrowserLogQueryDAO(client));
- this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO(client));
+ this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO());
this.registerServiceImplementation(IAlarmQueryDAO.class, new BanyanDBAlarmQueryDAO(client));
this.registerServiceImplementation(ILogQueryDAO.class, new BanyanDBLogQueryDAO(client));
- this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO(client));
+ this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO());
this.registerServiceImplementation(IProfileTaskLogQueryDAO.class, new BanyanDBProfileTaskLogQueryDAO(client, this.config.getFetchTaskLogMaxSize()));
this.registerServiceImplementation(
IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
- this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO(client));
- this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO(client));
- this.registerServiceImplementation(ITopologyQueryDAO.class, new BanyanDBTopologyQueryDAO(client));
+ this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO());
+ this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO());
+ this.registerServiceImplementation(ITopologyQueryDAO.class, new BanyanDBTopologyQueryDAO());
// TODO: metrics
this.registerServiceImplementation(IHistoryDeleteDAO.class, new BanyanDBHistoryDeleteDAO());
@@ -135,9 +133,6 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.client.registerChecker(healthChecker);
try {
this.client.connect();
- // create stream group
- final Group streamGroup = new Group("default-stream", Catalog.STREAM, 2);
- this.client.defineStreamGroup(streamGroup);
BanyanDBIndexInstaller installer = new BanyanDBIndexInstaller(client, getManager());
getManager().find(CoreModule.NAME).provider().getService(ModelCreator.class).addModelListener(installer);
} catch (Exception e) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
new file mode 100644
index 0000000000..83af6da867
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import com.google.common.collect.ImmutableSet;
+import org.apache.skywalking.banyandb.common.v1.BanyandbCommon;
+import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase;
+import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
+import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.analysis.metrics.DataTable;
+import org.apache.skywalking.oap.server.core.config.ConfigService;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.model.ModelColumn;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public enum MetadataRegistry {
+ INSTANCE;
+
+ private final Map<String, StreamMetadata> streams = new HashMap<>();
+
+ public StreamMetadata registerModel(Model model, ConfigService configService) {
+ BanyandbDatabase.Stream pbStream = parseStreamFromModel(model, configService);
+
+ final boolean useIdAsEntity = pbStream.getEntity().getTagNamesCount() == 1 &&
+ StreamMetadata.ID.equals(pbStream.getEntity().getTagNames(0));
+
+ final Stream stream = new Stream(pbStream.getMetadata().getGroup(), pbStream.getMetadata().getName());
+
+ List<IndexRule> indexRules = new ArrayList<>();
+ Set<String> entityNameSet = ImmutableSet.copyOf(pbStream.getEntity().getTagNamesList());
+ stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
+
+ Map<String, StreamMetadata.TagMetadata> tagDefinition = new HashMap<>();
+
+ for (BanyandbDatabase.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
+ final TagFamilySpec tagFamilySpec = TagFamilySpec.fromProtobuf(pbTagFamilySpec);
+ stream.addTagFamilySpec(tagFamilySpec);
+
+ int tagIndex = 0;
+ for (final TagFamilySpec.TagSpec tagSpec : tagFamilySpec.getTagSpecs()) {
+ // register tag
+ tagDefinition.put(tagSpec.getTagName(), new StreamMetadata.TagMetadata(tagFamilySpec.getTagFamilyName(), tagSpec, tagIndex++));
+
+ // if the tag family equals to "searchable", build index rules
+ if (tagFamilySpec.getTagFamilyName().equals(StreamMetadata.TAG_FAMILY_SEARCHABLE)) {
+ // check if this spec exists in the entity names
+ if (entityNameSet.contains(tagSpec.getTagName())) {
+ continue;
+ }
+ BanyandbDatabase.IndexRule pbIndexRule = parseIndexRuleFromTagSpec(pbStream.getMetadata(), tagSpec);
+ IndexRule indexRule = IndexRule.fromProtobuf(pbIndexRule);
+ indexRules.add(indexRule);
+ }
+ }
+ }
+
+ StreamMetadata streamMetadata = StreamMetadata.builder().model(model).stream(stream)
+ .tagDefinition(tagDefinition)
+ .indexRules(indexRules)
+ .group(pbStream.getMetadata().getGroup())
+ .useIdAsEntity(useIdAsEntity)
+ .build();
+ streams.put(model.getName(), streamMetadata);
+ return streamMetadata;
+ }
+
+ public StreamMetadata findStreamMetadata(final String name) {
+ return this.streams.get(name);
+ }
+
+ private BanyandbDatabase.Stream parseStreamFromModel(Model model, ConfigService configService) {
+ List<ModelColumn> shardingColumns = new ArrayList<>();
+
+ List<BanyandbDatabase.TagSpec> searchableTagsSpecs = new ArrayList<>();
+ List<BanyandbDatabase.TagSpec> dataTagsSpecs = new ArrayList<>();
+ for (final ModelColumn modelColumn : model.getColumns()) {
+ if (modelColumn.getShardingKeyIdx() > -1) {
+ shardingColumns.add(modelColumn);
+ }
+ if (modelColumn.isIndexOnly()) {
+ // skip
+ } else if (modelColumn.isStorageOnly()) {
+ dataTagsSpecs.add(parseTagSpecFromModelColumn(modelColumn));
+ } else {
+ searchableTagsSpecs.add(parseTagSpecFromModelColumn(modelColumn));
+ }
+ }
+
+ Set<String> entities = shardingColumns.stream()
+ .sorted(Comparator.comparingInt(ModelColumn::getShardingKeyIdx))
+ .map(modelColumn -> modelColumn.getColumnName().getStorageName())
+ .collect(Collectors.toSet());
+
+ if (entities.isEmpty()) {
+ // if sharding keys are not defined, we have to use ID
+ entities = Collections.singleton(StreamMetadata.ID);
+ // append ID
+ searchableTagsSpecs.add(BanyandbDatabase.TagSpec.newBuilder()
+ .setName(StreamMetadata.ID)
+ .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build());
+ }
+
+ // add all user-defined indexed tags to the end of the "searchable" family
+ if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
+ searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableTracesTags()));
+ } else if (LogRecord.INDEX_NAME.equals(model.getName())) {
+ searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableLogsTags()));
+ } else if (AlarmRecord.INDEX_NAME.equals(model.getName())) {
+ searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableAlarmTags()));
+ }
+
+ String group = "default-stream";
+ if (model.isSuperDataset()) {
+ // for superDataset, we should use separate group
+ group = model.getName() + "-stream";
+ }
+
+ return BanyandbDatabase.Stream.newBuilder()
+ .addTagFamilies(BanyandbDatabase.TagFamilySpec.newBuilder()
+ .setName(StreamMetadata.TAG_FAMILY_DATA)
+ .addAllTags(dataTagsSpecs)
+ .build())
+ .addTagFamilies(BanyandbDatabase.TagFamilySpec.newBuilder()
+ .setName(StreamMetadata.TAG_FAMILY_SEARCHABLE)
+ .addAllTags(searchableTagsSpecs)
+ .build())
+ .setEntity(BanyandbDatabase.Entity.newBuilder()
+ .addAllTagNames(entities)
+ .build())
+ .setMetadata(BanyandbCommon.Metadata.newBuilder()
+ .setGroup(group)
+ .setName(model.getName())
+ .build())
+ .build();
+ }
+
+ private BanyandbDatabase.TagSpec parseTagSpecFromModelColumn(ModelColumn modelColumn) {
+ final Class<?> clazz = modelColumn.getType();
+ if (String.class.equals(clazz)) {
+ return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
+ .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build();
+ } else if (int.class.equals(clazz) || long.class.equals(clazz)) {
+ return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
+ .setType(BanyandbDatabase.TagType.TAG_TYPE_INT).build();
+ } else if (byte[].class.equals(clazz) || DataTable.class.equals(clazz)) {
+ return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
+ .setType(BanyandbDatabase.TagType.TAG_TYPE_DATA_BINARY).build();
+ } else {
+ throw new IllegalStateException("type " + modelColumn.getType().toString() + " is not supported");
+ }
+ }
+
+ private List<BanyandbDatabase.TagSpec> parseTagSpecsFromConfiguration(String tags) {
+ if (StringUtil.isEmpty(tags)) {
+ return Collections.emptyList();
+ }
+ String[] tagsArray = tags.split(",");
+ if (tagsArray.length == 0) {
+ return Collections.emptyList();
+ }
+ return Arrays.stream(tagsArray)
+ .map(tagName -> BanyandbDatabase.TagSpec.newBuilder().setName(tagName)
+ .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build())
+ .collect(Collectors.toList());
+ }
+
+ private BanyandbDatabase.IndexRule parseIndexRuleFromTagSpec(BanyandbCommon.Metadata metadata, TagFamilySpec.TagSpec tagSpec) {
+ // In SkyWalking, only "trace_id" should be stored as a global index
+ BanyandbDatabase.IndexRule.Location loc = "trace_id".equals(tagSpec.getTagName()) ?
+ BanyandbDatabase.IndexRule.Location.LOCATION_GLOBAL :
+ BanyandbDatabase.IndexRule.Location.LOCATION_SERIES;
+
+ return BanyandbDatabase.IndexRule.newBuilder()
+ .setMetadata(BanyandbCommon.Metadata.newBuilder()
+ .setName(tagSpec.getTagName()).setGroup(metadata.getGroup()))
+ .setLocation(loc)
+ .addTags(tagSpec.getTagName())
+ // TODO: support TYPE_TREE
+ .setType(BanyandbDatabase.IndexRule.Type.TYPE_INVERTED)
+ .build();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
deleted file mode 100644
index 0a2af94339..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
+++ /dev/null
@@ -1,140 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-
-import com.google.common.base.Charsets;
-import com.google.common.collect.ImmutableSet;
-import com.google.common.io.CharStreams;
-import com.google.protobuf.util.JsonFormat;
-import lombok.Builder;
-import lombok.Getter;
-import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase;
-import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
-import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
-import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-@Getter
-@Builder
-@Slf4j
-public class StreamMetaInfo {
- public static final String TAG_FAMILY_SEARCHABLE = "searchable";
- public static final String TAG_FAMILY_DATA = "data";
-
- public static final String ID = "id";
-
- private static final Map<String, StreamMetaInfo> STREAMS = new HashMap<>();
-
- private final Model model;
-
- /**
- * stream is the metadata to be used for schema creation,
- * 1. Read json from resources/metadata/{model.name}.json and deserialize to protobuf,
- * 2. Iterate over tag families,
- * 3. Iterate over tags in each tag family
- * 4.
- */
- private final Stream stream;
-
- private final List<IndexRule> indexRules;
-
- public static StreamMetaInfo addModel(Model model) {
- BanyandbDatabase.Stream pbStream = parseStreamFromJSON(model.getName());
- if (pbStream == null) {
- log.warn("fail to find stream schema {}", model.getName());
- return null;
- }
- final Stream stream = new Stream(pbStream.getMetadata().getName());
-
- List<IndexRule> indexRules = new ArrayList<>();
-
- Set<String> entityNameSet = ImmutableSet.copyOf(pbStream.getEntity().getTagNamesList());
-
- stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
-
- for (BanyandbDatabase.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
- final TagFamilySpec tagFamilySpec = TagFamilySpec.fromProtobuf(pbTagFamilySpec);
- stream.addTagFamilySpec(tagFamilySpec);
-
- // if the tag family equals to "searchable", build index rules
- if (tagFamilySpec.getTagFamilyName().equals(TAG_FAMILY_SEARCHABLE)) {
- for (final TagFamilySpec.TagSpec tagSpec : tagFamilySpec.getTagSpecs()) {
- // check if this spec exists in the entity names
- if (entityNameSet.contains(tagSpec.getTagName())) {
- continue;
- }
- BanyandbDatabase.IndexRule pbIndexRule = parseIndexRulesFromJSON(model.getName(), tagSpec.getTagName());
- if (pbIndexRule == null) {
- log.warn("fail to find the index rule for {}", tagSpec.getTagName());
- continue;
- }
- IndexRule indexRule = IndexRule.fromProtobuf(pbIndexRule);
- indexRules.add(indexRule);
- }
- }
- }
-
- return StreamMetaInfo.builder().model(model).stream(stream).indexRules(indexRules).build();
- }
-
- private static BanyandbDatabase.Stream parseStreamFromJSON(String name) {
- try {
- InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream("metadata/" + name + ".json");
- if (is == null) {
- log.warn("fail to find definition for {}", name);
- return null;
- }
- String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
- BanyandbDatabase.Stream.Builder b = BanyandbDatabase.Stream.newBuilder();
- JsonFormat.parser().merge(result, b);
- return b.build();
- } catch (IOException ioEx) {
- log.error("fail to read json", ioEx);
- return null;
- }
- }
-
- private static BanyandbDatabase.IndexRule parseIndexRulesFromJSON(String streamName, String name) {
- try {
- InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream(String.join("/",
- new String[]{"metadata", "index_rules", name + ".json"}));
- if (is == null) {
- log.warn("fail to find index rules for {}", streamName);
- return null;
- }
- String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
- BanyandbDatabase.IndexRule.Builder b = BanyandbDatabase.IndexRule.newBuilder();
- JsonFormat.parser().merge(result, b);
- return b.build();
- } catch (IOException ioEx) {
- log.error("fail to read json", ioEx);
- return null;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java
new file mode 100644
index 0000000000..814f5f62e2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import lombok.Builder;
+import lombok.Data;
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
+import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.util.List;
+import java.util.Map;
+
+@Getter
+@Builder
+@Slf4j
+public class StreamMetadata {
+ public static final String TAG_FAMILY_SEARCHABLE = "searchable";
+ public static final String TAG_FAMILY_DATA = "data";
+
+ public static final String ID = "id";
+
+ private final Model model;
+
+ private final Map<String, TagMetadata> tagDefinition;
+
+ /**
+ * Group of the stream
+ */
+ private final String group;
+ /**
+ * Spec of the stream
+ */
+ private final Stream stream;
+ /**
+ * Index rules attached to the stream
+ */
+ private final List<IndexRule> indexRules;
+
+ private final int dataFamilySize;
+ private final int searchableFamilySize;
+
+ private final boolean useIdAsEntity;
+
+ @Getter
+ @Data
+ public static class TagMetadata {
+ private final String tagFamilyName;
+ private final TagFamilySpec.TagSpec tagSpec;
+ private final int tagIndex;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBEventQueryDAO.java
similarity index 59%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBEventQueryDAO.java
index f8287973bc..f06de0cff5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBEventQueryDAO.java
@@ -16,15 +16,22 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.query.type.event.EventQueryCondition;
+import org.apache.skywalking.oap.server.core.query.type.event.Events;
+import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
-@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+import java.util.List;
+
+public class BanyanDBEventQueryDAO implements IEventQueryDAO {
+ @Override
+ public Events queryEvents(EventQueryCondition condition) throws Exception {
+ return new Events();
+ }
+
+ @Override
+ public Events queryEvents(List<EventQueryCondition> conditionList) throws Exception {
+ return new Events();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
new file mode 100644
index 0000000000..5833c98c56
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+
+import org.apache.skywalking.oap.server.core.query.type.Endpoint;
+import org.apache.skywalking.oap.server.core.query.type.Process;
+import org.apache.skywalking.oap.server.core.query.type.Service;
+import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBMetadataQueryDAO implements IMetadataQueryDAO {
+ @Override
+ public List<Service> listServices(String layer, String group) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Service> getServices(String serviceId) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<ServiceInstance> listInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ServiceInstance getInstance(String instanceId) throws IOException {
+ return null;
+ }
+
+ @Override
+ public List<Endpoint> findEndpoint(String keyword, String serviceId, int limit) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Process> listProcesses(String serviceId, String instanceId) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public Process getProcess(String processId) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBNetworkAddressAliasDAO.java
similarity index 65%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBNetworkAddressAliasDAO.java
index f8287973bc..685d7b8083 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBNetworkAddressAliasDAO.java
@@ -16,15 +16,17 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
-@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBNetworkAddressAliasDAO implements INetworkAddressAliasDAO {
+ @Override
+ public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
similarity index 57%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
index f8287973bc..0c6e1ebfd1 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
@@ -16,15 +16,23 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
-@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+ @Override
+ public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public ProfileTask getById(String id) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBTopologyQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBTopologyQueryDAO.java
new file mode 100644
index 0000000000..9ce2282f56
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBTopologyQueryDAO.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+
+import org.apache.skywalking.oap.server.core.query.type.Call;
+import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBTopologyQueryDAO implements ITopologyQueryDAO {
+ @Override
+ public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadInstanceRelationDetectedAtServerSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadInstanceRelationDetectedAtClientSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<Call.CallDetail> loadEndpointRelation(long startTB, long endTB, String destEndpointId) throws IOException {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
new file mode 100644
index 0000000000..efbd9ce106
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+
+import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
+import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
+import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
+import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class BanyanDBUITemplateManagementDAO implements UITemplateManagementDAO {
+ @Override
+ public DashboardConfiguration getTemplate(String id) throws IOException {
+ return null;
+ }
+
+ @Override
+ public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
+ return null;
+ }
+
+ @Override
+ public TemplateChangeStatus changeTemplate(DashboardSetting setting) throws IOException {
+ return null;
+ }
+
+ @Override
+ public TemplateChangeStatus disableTemplate(String id) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
deleted file mode 100644
index 260fc17664..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class AlarmRecordBuilder extends BanyanDBStorageDataBuilder<AlarmRecord> {
- public static final List<String> INDEXED_TAGS = ImmutableList.of(
- "level"
- );
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(AlarmRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
- searchable.add(TagAndValue.longField(entity.getScope()));
- searchable.add(TagAndValue.longField(entity.getStartTime()));
- searchable.addAll(filterSearchableTags(entity.getTags(), INDEXED_TAGS));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(AlarmRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(6);
- data.add(TagAndValue.stringField(entity.getName()));
- data.add(TagAndValue.stringField(entity.getId0()));
- data.add(TagAndValue.stringField(entity.getId1()));
- data.add(TagAndValue.stringField(entity.getAlarmMessage()));
- data.add(TagAndValue.stringField(entity.getRuleName()));
- data.add(TagAndValue.binaryField(entity.getTagsRawData()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
deleted file mode 100644
index 814f3149f7..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
+++ /dev/null
@@ -1,88 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
-import org.apache.skywalking.oap.server.core.storage.StorageData;
-import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-public abstract class BanyanDBStorageDataBuilder<T extends StorageData> implements StorageBuilder<T, StreamWrite.StreamWriteBuilder> {
- @Override
- public T storage2Entity(StreamWrite.StreamWriteBuilder storageData) {
- return null;
- }
-
- @Override
- public StreamWrite.StreamWriteBuilder entity2Storage(T entity) {
- StreamWrite.StreamWriteBuilder b = StreamWrite.builder()
- .elementId(this.elementID(entity))
- .searchableTags(this.searchableTags(entity))
- .dataTags(this.dataTags(entity));
- Long ts = this.extractTimestamp(entity);
- if (ts != null) {
- b.timestamp(ts);
- }
- return b;
- }
-
- protected Long extractTimestamp(T entity) {
- return null;
- }
-
- protected List<SerializableTag<BanyandbModel.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
- if (rawTags == null) {
- return Collections.emptyList();
- }
- Map<String, SerializableTag<BanyandbModel.TagValue>> map = new HashMap<>();
- for (final Tag tag : rawTags) {
- map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
- }
- final List<SerializableTag<BanyandbModel.TagValue>> tags = new ArrayList<>();
- for (String indexedTag : indexTags) {
- SerializableTag<BanyandbModel.TagValue> tag = map.get(indexedTag);
- if (tag == null) {
- tags.add(TagAndValue.nullField());
- } else {
- tags.add(tag);
- }
- }
-
- return tags;
- }
-
- protected String elementID(T entity) {
- return entity.id();
- }
-
- abstract protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(T entity);
-
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(T entity) {
- return Collections.emptyList();
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
deleted file mode 100644
index 034565249b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class BrowserErrorLogRecordBuilder extends BanyanDBStorageDataBuilder<BrowserErrorLogRecord> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(BrowserErrorLogRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
- searchable.add(TagAndValue.stringField(entity.getUniqueId()));
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- searchable.add(TagAndValue.stringField(entity.getServiceVersionId()));
- searchable.add(TagAndValue.stringField(entity.getPagePathId()));
- searchable.add(TagAndValue.longField(entity.getErrorCategory()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(BrowserErrorLogRecord entity) {
- return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
deleted file mode 100644
index c6cef854e4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
+++ /dev/null
@@ -1,48 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class EndpointRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<EndpointRelationServerSideMetrics> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointRelationServerSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
- // 0 - source_endpoint
- searchable.add(TagAndValue.stringField(entity.getSourceEndpoint()));
- // 1 - dest_endpoint
- searchable.add(TagAndValue.stringField(entity.getDestEndpoint()));
- // 2 - entity_id
- searchable.add(TagAndValue.stringField(entity.getEntityId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(EndpointRelationServerSideMetrics entity) {
- // 0 - component_id
- return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
deleted file mode 100644
index 512fe2aa5c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
+++ /dev/null
@@ -1,52 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.source.Event;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class EventBuilder extends BanyanDBStorageDataBuilder<Event> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(Event entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(8);
- searchable.add(TagAndValue.stringField(entity.getUuid()));
- searchable.add(TagAndValue.stringField(entity.getService()));
- searchable.add(TagAndValue.stringField(entity.getServiceInstance()));
- searchable.add(TagAndValue.stringField(entity.getEndpoint()));
- searchable.add(TagAndValue.stringField(entity.getName()));
- searchable.add(TagAndValue.stringField(entity.getType()));
- searchable.add(TagAndValue.longField(entity.getStartTime()));
- searchable.add(TagAndValue.longField(entity.getEndTime()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(Event entity) {
- return ImmutableList.of(
- TagAndValue.stringField(entity.getMessage()),
- TagAndValue.stringField(entity.getParameters())
- );
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
deleted file mode 100644
index 8fddd927a4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class LogRecordBuilder extends BanyanDBStorageDataBuilder<LogRecord> {
- public static final List<String> INDEXED_TAGS = ImmutableList.of(
- "level"
- );
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(LogRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
- searchable.add(TagAndValue.stringField(entity.getUniqueId()));
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- searchable.add(TagAndValue.stringField(entity.getServiceInstanceId()));
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- searchable.add(TagAndValue.stringField(entity.getEndpointId()));
- searchable.add(TagAndValue.stringField(entity.getTraceId()));
- searchable.add(TagAndValue.stringField(entity.getTraceSegmentId()));
- searchable.add(TagAndValue.longField(entity.getSpanId()));
- searchable.addAll(filterSearchableTags(entity.getTags(), INDEXED_TAGS));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(LogRecord entity) {
- return ImmutableList.of(
- TagAndValue.stringField(entity.getContent()),
- TagAndValue.longField(entity.getContentType()),
- TagAndValue.binaryField(entity.getTagsRawData())
- );
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
deleted file mode 100644
index 9f0fda0ac4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
+++ /dev/null
@@ -1,100 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.Layer;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-import static org.apache.skywalking.oap.server.core.Const.DOUBLE_COLONS_SPLIT;
-
-public class Metadata {
- public static class ServiceTrafficBuilder extends BanyanDBStorageDataBuilder<ServiceTraffic> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceTraffic entity) {
- final String serviceName = entity.getName();
- entity.setShortName(serviceName);
- if (entity.isNormal()) {
- int groupIdx = serviceName.indexOf(DOUBLE_COLONS_SPLIT);
- if (groupIdx > 0) {
- entity.setGroup(serviceName.substring(0, groupIdx));
- entity.setShortName(serviceName.substring(groupIdx + 2));
- }
- }
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(4);
- // 0 - serviceName
- searchable.add(TagAndValue.stringField(serviceName));
- // 1 - serviceID
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- // 2 - group
- searchable.add(TagAndValue.stringField(entity.getGroup()));
- // 3 - layer
- Layer layer = entity.getLayer();
- searchable.add(TagAndValue.longField(layer != null ? layer.value() : Layer.UNDEFINED.value()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceTraffic entity) {
- // 0 - shortName
- return Collections.singletonList(TagAndValue.stringField(entity.getShortName()));
- }
- }
-
- public static class EndpointTrafficBuilder extends BanyanDBStorageDataBuilder<EndpointTraffic> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointTraffic entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
- // 0 - serviceID
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- // 1 - name
- searchable.add(TagAndValue.stringField(entity.getName()));
- return searchable;
- }
- }
-
- public static class InstanceTrafficBuilder extends BanyanDBStorageDataBuilder<InstanceTraffic> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(InstanceTraffic entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
- // serviceID
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- // lastPingTimestamp
- searchable.add(TagAndValue.longField(entity.getLastPingTimestamp()));
- // ID: we have to duplicate "ID" here for query
- searchable.add(TagAndValue.stringField(entity.id()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(InstanceTraffic entity) {
- return Collections.singletonList(TagAndValue.binaryField(
- entity.serialize().build().toByteArray()
- ));
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
deleted file mode 100644
index e429f80b9c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ /dev/null
@@ -1,49 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(NetworkAddressAlias entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(4);
- // 0 - time_bucket
- data.add(TagAndValue.longField(entity.getTimeBucket()));
- // 1 - address
- data.add(TagAndValue.stringField(entity.getAddress()));
- // 2 - represent_service_id
- data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
- // 3 - represent_service_instance_id
- data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
deleted file mode 100644
index 1f861ab4cd..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class ProfileTaskLogRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskLogRecord> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
- searchable.add(TagAndValue.longField(entity.getOperationTime()));
- searchable.add(TagAndValue.stringField(entity.getInstanceId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ProfileTaskLogRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(2);
- data.add(TagAndValue.stringField(entity.getTaskId()));
- data.add(TagAndValue.longField(entity.getOperationType()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
deleted file mode 100644
index 23a5bf8fdf..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class ProfileTaskRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskRecord> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(9);
- // 0 - id
- searchable.add(TagAndValue.stringField(entity.id()));
- // 1 - service_id
- searchable.add(TagAndValue.stringField(entity.getServiceId()));
- // 2 - endpoint_name
- searchable.add(TagAndValue.stringField(entity.getEndpointName()));
- // 3 - start_time
- searchable.add(TagAndValue.longField(entity.getStartTime()));
- // 4 - duration
- searchable.add(TagAndValue.longField(entity.getDuration()));
- // 5 - min_duration_threshold
- searchable.add(TagAndValue.longField(entity.getMinDurationThreshold()));
- // 6 - dump_period
- searchable.add(TagAndValue.longField(entity.getDumpPeriod()));
- // 7 - create_time
- searchable.add(TagAndValue.longField(entity.getCreateTime()));
- // 8 - max_sampling_count
- searchable.add(TagAndValue.longField(entity.getMaxSamplingCount()));
- return searchable;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
deleted file mode 100644
index 28c0ba5f38..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class ProfileThreadSnapshotRecordBuilder extends BanyanDBStorageDataBuilder<ProfileThreadSnapshotRecord> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileThreadSnapshotRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(4);
- searchable.add(TagAndValue.stringField(entity.getTaskId()));
- searchable.add(TagAndValue.stringField(entity.getSegmentId()));
- searchable.add(TagAndValue.longField(entity.getDumpTime()));
- searchable.add(TagAndValue.longField(entity.getSequence()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ProfileThreadSnapshotRecord entity) {
- return Collections.singletonList(TagAndValue.binaryField(entity.getStackBinary()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
deleted file mode 100644
index 729ad90e15..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
+++ /dev/null
@@ -1,68 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class SegmentRecordBuilder extends BanyanDBStorageDataBuilder<SegmentRecord> {
- public static final List<String> INDEXED_TAGS = ImmutableList.of(
- "http.method",
- "status_code",
- "db.type",
- "db.instance",
- "mq.queue",
- "mq.topic",
- "mq.broker"
- );
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(SegmentRecord segmentRecord) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(10);
- // 0 - trace_id
- searchable.add(TagAndValue.stringField(segmentRecord.getTraceId()));
- // 1 - is_error
- searchable.add(TagAndValue.longField(segmentRecord.getIsError()));
- // 2 - service_id
- searchable.add(TagAndValue.stringField(segmentRecord.getServiceId()));
- // 3 - service_instance_id
- searchable.add(TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
- // 4 - endpoint_id
- searchable.add(TagAndValue.stringField(segmentRecord.getEndpointId()));
- // 5 - latency
- searchable.add(TagAndValue.longField(segmentRecord.getLatency()));
- // 6 - start_time
- searchable.add(TagAndValue.longField(segmentRecord.getStartTime()));
- // 7 ~ 13: indexed tags
- searchable.addAll(filterSearchableTags(segmentRecord.getTagsRawData(), INDEXED_TAGS));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(SegmentRecord entity) {
- return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
deleted file mode 100644
index dc4de6ddcb..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationClientSideMetrics> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
- searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getEntityId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(3);
- data.add(TagAndValue.longField(entity.getComponentId()));
- data.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
- data.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
deleted file mode 100644
index 7796265f8d..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class ServiceInstanceRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationServerSideMetrics> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationServerSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
- searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getEntityId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationServerSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.longField(entity.getComponentId()));
- data.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
- data.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
deleted file mode 100644
index cf8ae73fdf..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class ServiceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceRelationClientSideMetrics> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
- searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getEntityId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceRelationClientSideMetrics entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java
deleted file mode 100644
index 92a9a13fb9..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationServerSideMetricsBuilder.java
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-public class ServiceRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceRelationServerSideMetrics> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceRelationServerSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
- searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getEntityId()));
- return searchable;
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceRelationServerSideMetrics entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
deleted file mode 100644
index 6b53a63174..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class UITemplateBuilder extends BanyanDBStorageDataBuilder<UITemplate> {
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(UITemplate entity) {
- return ImmutableList.of(
- TagAndValue.stringField(entity.getName()),
- TagAndValue.longField(entity.getDisabled())
- );
- }
-
- @Override
- protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(UITemplate entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
- data.add(TagAndValue.stringField(entity.getType()));
- data.add(TagAndValue.stringField(entity.getConfiguration()));
- data.add(TagAndValue.longField(entity.getActivated()));
- return data;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index 04690a30cb..0f093e396b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -19,17 +19,15 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.time.Instant;
import java.util.List;
-import java.util.function.Function;
public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageClient> {
private static final Instant UPPER_BOUND = Instant.ofEpochSecond(0, Long.MAX_VALUE);
@@ -40,17 +38,17 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
super(client);
}
- protected StreamQueryResponse query(String indexName, List<String> searchableTags, QueryBuilder builder) {
- return this.query(indexName, searchableTags, null, builder);
+ protected StreamQueryResponse query(StreamMetadata metadata, List<String> searchableTags, QueryBuilder builder) {
+ return this.query(metadata, searchableTags, null, builder);
}
- protected StreamQueryResponse query(String indexName, List<String> searchableTags, TimestampRange timestampRange,
+ protected StreamQueryResponse query(StreamMetadata metadata, List<String> searchableTags, TimestampRange timestampRange,
QueryBuilder builder) {
final StreamQuery query;
if (timestampRange == null) {
- query = new StreamQuery(indexName, LARGEST_TIME_RANGE, searchableTags);
+ query = new StreamQuery(metadata.getGroup(), metadata.getModel().getName(), LARGEST_TIME_RANGE, searchableTags);
} else {
- query = new StreamQuery(indexName, timestampRange, searchableTags);
+ query = new StreamQuery(metadata.getGroup(), metadata.getModel().getName(), timestampRange, searchableTags);
}
builder.apply(query);
@@ -62,22 +60,19 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
abstract void apply(final StreamQuery query);
protected PairQueryCondition<Long> eq(String name, long value) {
- return PairQueryCondition.LongQueryCondition.eq(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.eq(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<Long> lte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.le(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.le(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<Long> gte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.ge(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.ge(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<String> eq(String name, String value) {
- return PairQueryCondition.StringQueryCondition.eq(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.StringQueryCondition.eq(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
}
}
-
- interface RowEntityDeserializer<T> extends Function<RowEntity, T> {
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index 08225e51e1..0b7693d16c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -18,14 +18,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.base.Charsets;
import com.google.common.collect.ImmutableList;
-import com.google.gson.reflect.TypeToken;
-import com.google.protobuf.ByteString;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
@@ -33,22 +29,25 @@ import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Scope;
import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
import org.apache.skywalking.oap.server.core.query.type.Alarms;
-import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.alarm.AlarmRecord} is a stream,
* which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
*/
public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarmQueryDAO {
+ private final StreamMetadata alarmRecordMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(AlarmRecord.INDEX_NAME);
+
public BanyanDBAlarmQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -60,7 +59,7 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
}
- StreamQueryResponse resp = query(AlarmRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(alarmRecordMetadata,
ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
tsRange,
new QueryBuilder() {
@@ -76,9 +75,8 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
- if (AlarmRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(eq(tag.getKey(), tag.getValue()));
- }
+ // TODO: check whether tags in the alarm are indexed
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
}
}
query.setLimit(limit);
@@ -86,39 +84,27 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
}
});
- List<AlarmMessage> messages = resp.getElements().stream().map(new AlarmMessageDeserializer())
- .collect(Collectors.toList());
-
Alarms alarms = new Alarms();
- alarms.setTotal(messages.size());
- alarms.getMsgs().addAll(messages);
- return alarms;
- }
+ alarms.setTotal(resp.size());
- public static class AlarmMessageDeserializer implements RowEntityDeserializer<AlarmMessage> {
- @Override
- public AlarmMessage apply(RowEntity row) {
- AlarmMessage alarmMessage = new AlarmMessage();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- int scopeID = ((Number) searchable.get(0).getValue()).intValue();
- alarmMessage.setScopeId(scopeID);
- alarmMessage.setScope(Scope.Finder.valueOf(scopeID));
- alarmMessage.setStartTime(((Number) searchable.get(1).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- alarmMessage.setId((String) data.get(0).getValue());
- alarmMessage.setId1((String) data.get(1).getValue());
- alarmMessage.setMessage((String) data.get(2).getValue());
- Object o = data.get(3).getValue();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- this.parseDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
- }
- return alarmMessage;
- }
+ for (final RowEntity rowEntity : resp.getElements()) {
+ AlarmRecord.Builder builder = new AlarmRecord.Builder();
+ AlarmRecord alarmRecord = builder.storage2Entity(
+ new BanyanDBConverter.StreamToEntity(this.alarmRecordMetadata, rowEntity)
+ );
- void parseDataBinary(byte[] dataBinary, List<KeyValue> tags) {
- List<Tag> tagList = GSON.fromJson(new String(dataBinary, Charsets.UTF_8), new TypeToken<List<Tag>>() {
- }.getType());
- tagList.forEach(pair -> tags.add(new KeyValue(pair.getKey(), pair.getValue())));
+ AlarmMessage message = new AlarmMessage();
+ message.setId(String.valueOf(alarmRecord.getId0()));
+ message.setId1(String.valueOf(alarmRecord.getId1()));
+ message.setMessage(alarmRecord.getAlarmMessage());
+ message.setStartTime(alarmRecord.getStartTime());
+ message.setScope(Scope.Finder.valueOf(alarmRecord.getScope()));
+ message.setScopeId(alarmRecord.getScope());
+ if (!CollectionUtils.isEmpty(alarmRecord.getTagsRawData())) {
+ parserDataBinary(alarmRecord.getTagsRawData(), message.getTags());
+ }
+ alarms.getMsgs().add(message);
}
+ return alarms;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index c1bbc16941..d384210cd9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -19,12 +19,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
import com.google.protobuf.InvalidProtocolBufferException;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
@@ -35,17 +33,20 @@ import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.Collections;
-import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
*/
public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements IBrowserLogQueryDAO {
+ private final StreamMetadata browserErrorLogRecordMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(BrowserErrorLogRecord.INDEX_NAME);
+
public BanyanDBBrowserLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -57,8 +58,7 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
- final BrowserErrorLogs logs = new BrowserErrorLogs();
- StreamQueryResponse resp = query(BrowserErrorLogRecord.INDEX_NAME, ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
+ StreamQueryResponse resp = query(browserErrorLogRecordMetadata, ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
BrowserErrorLogRecord.SERVICE_VERSION_ID,
BrowserErrorLogRecord.PAGE_PATH_ID,
BrowserErrorLogRecord.ERROR_CATEGORY), tsRange, new QueryBuilder() {
@@ -83,40 +83,47 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
query.setLimit(limit);
}
});
- logs.getLogs().addAll(resp.getElements().stream().map(new BrowserErrorLogDeserializer()).collect(Collectors.toList()));
- logs.setTotal(logs.getLogs().size());
+
+ BrowserErrorLogs logs = new BrowserErrorLogs();
+ logs.setTotal(resp.size());
+
+ for (final RowEntity rowEntity : resp.getElements()) {
+ final byte[] dataBinary =
+ rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, BrowserErrorLogRecord.DATA_BINARY);
+ if (dataBinary != null && dataBinary.length > 0) {
+ BrowserErrorLog log = parserDataBinary(dataBinary);
+ logs.getLogs().add(log);
+ }
+ }
return logs;
}
- public static class BrowserErrorLogDeserializer implements RowEntityDeserializer<BrowserErrorLog> {
- @Override
- public BrowserErrorLog apply(RowEntity row) {
- // FIXME: use protobuf directly
+ /**
+ * TODO: merge the default method in the interface
+ */
+ private BrowserErrorLog parserDataBinary(
+ byte[] dataBinary) {
+ try {
BrowserErrorLog log = new BrowserErrorLog();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- log.setService((String) searchable.get(0).getValue());
- log.setServiceVersion((String) searchable.get(1).getValue());
- log.setPagePath((String) searchable.get(2).getValue());
- log.setCategory(ErrorCategory.valueOf((String) searchable.get(3).getValue()));
- log.setTime(row.getTimestamp());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- Object o = data.get(0).getValue();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- try {
- org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog browserErrorLog = org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog
- .parseFrom((ByteString) o);
- log.setGrade(browserErrorLog.getGrade());
- log.setCol(browserErrorLog.getCol());
- log.setLine(browserErrorLog.getLine());
- log.setMessage(browserErrorLog.getMessage());
- log.setErrorUrl(browserErrorLog.getErrorUrl());
- log.setStack(browserErrorLog.getStack());
- log.setFirstReportedError(browserErrorLog.getFirstReportedError());
- } catch (InvalidProtocolBufferException ex) {
- throw new RuntimeException("fail to parse proto buffer", ex);
- }
- }
+ org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog browserErrorLog = org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog
+ .parseFrom(dataBinary);
+
+ log.setService(browserErrorLog.getService());
+ log.setServiceVersion(browserErrorLog.getServiceVersion());
+ log.setTime(browserErrorLog.getTime());
+ log.setPagePath(browserErrorLog.getPagePath());
+ log.setCategory(ErrorCategory.valueOf(browserErrorLog.getCategory().name().toUpperCase()));
+ log.setGrade(browserErrorLog.getGrade());
+ log.setMessage(browserErrorLog.getMessage());
+ log.setLine(browserErrorLog.getLine());
+ log.setCol(browserErrorLog.getCol());
+ log.setStack(browserErrorLog.getStack());
+ log.setErrorUrl(browserErrorLog.getErrorUrl());
+ log.setFirstReportedError(browserErrorLog.getFirstReportedError());
+
return log;
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException(e);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
deleted file mode 100644
index 44299c38e9..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
+++ /dev/null
@@ -1,148 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.base.Strings;
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.query.PaginationUtils;
-import org.apache.skywalking.oap.server.core.query.input.Duration;
-import org.apache.skywalking.oap.server.core.query.type.event.EventQueryCondition;
-import org.apache.skywalking.oap.server.core.query.type.event.EventType;
-import org.apache.skywalking.oap.server.core.query.type.event.Events;
-import org.apache.skywalking.oap.server.core.query.type.event.Source;
-import org.apache.skywalking.oap.server.core.source.Event;
-import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-
-import java.util.List;
-import java.util.stream.Collectors;
-
-/**
- * {@link org.apache.skywalking.oap.server.core.source.Event} is a stream
- */
-public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEventQueryDAO {
- public BanyanDBEventQueryDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public Events queryEvents(EventQueryCondition condition) throws Exception {
- StreamQueryResponse resp = query(Event.INDEX_NAME,
- ImmutableList.of(Event.UUID, Event.SERVICE, Event.SERVICE_INSTANCE, Event.ENDPOINT, Event.NAME, Event.TYPE, Event.START_TIME, Event.END_TIME),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(Event.MESSAGE, Event.PARAMETERS));
-
- buildConditions(condition, query);
-
- PaginationUtils.Page page = PaginationUtils.INSTANCE.exchange(condition.getPaging());
- query.setLimit(page.getLimit());
- query.setOffset(page.getFrom());
- switch (condition.getOrder()) {
- case ASC:
- query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.ASC));
- break;
- case DES:
- query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
- }
- }
-
- private void buildConditions(EventQueryCondition condition, final StreamQuery query) {
- if (!Strings.isNullOrEmpty(condition.getUuid())) {
- query.appendCondition(eq(Event.UUID, condition.getUuid()));
- }
- final Source source = condition.getSource();
- if (source != null) {
- if (!Strings.isNullOrEmpty(source.getService())) {
- query.appendCondition(eq(Event.SERVICE, source.getService()));
- }
- if (!Strings.isNullOrEmpty(source.getServiceInstance())) {
- query.appendCondition(eq(Event.SERVICE_INSTANCE, source.getServiceInstance()));
- }
- if (!Strings.isNullOrEmpty(source.getEndpoint())) {
- query.appendCondition(eq(Event.ENDPOINT, source.getEndpoint()));
- }
- }
- if (!Strings.isNullOrEmpty(condition.getName())) {
- query.appendCondition(eq(Event.NAME, condition.getName()));
- }
- if (condition.getType() != null) {
- query.appendCondition(eq(Event.TYPE, condition.getType().name()));
- }
- final Duration time = condition.getTime();
- if (time != null) {
- if (time.getStartTimestamp() > 0) {
- query.appendCondition(gte(Event.START_TIME, time.getStartTimestamp()));
- }
- if (time.getEndTimestamp() > 0) {
- query.appendCondition(lte(Event.END_TIME, time.getEndTimestamp()));
- }
- }
- }
- });
-
- List<org.apache.skywalking.oap.server.core.query.type.event.Event> eventList = resp.getElements().stream().map(new EventDeserializer()).collect(Collectors.toList());
-
- Events events = new Events();
- events.setEvents(eventList);
- events.setTotal(eventList.size());
- return events;
- }
-
- @Override
- public Events queryEvents(List<EventQueryCondition> conditionList) throws Exception {
- Events events = new Events();
- for (final EventQueryCondition condition : conditionList) {
- Events subEvents = this.queryEvents(condition);
- if (subEvents.getEvents().size() == 0) {
- continue;
- }
-
- events.getEvents().addAll(subEvents.getEvents());
- events.setTotal(events.getTotal() + subEvents.getTotal());
- }
-
- return events;
- }
-
- public static class EventDeserializer implements RowEntityDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
- @Override
- public org.apache.skywalking.oap.server.core.query.type.event.Event apply(RowEntity row) {
- final org.apache.skywalking.oap.server.core.query.type.event.Event resultEvent = new org.apache.skywalking.oap.server.core.query.type.event.Event();
- // searchable
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- resultEvent.setUuid((String) searchable.get(0).getValue());
- resultEvent.setSource(new Source((String) searchable.get(1).getValue(), (String) searchable.get(2).getValue(), (String) searchable.get(3).getValue()));
- resultEvent.setName((String) searchable.get(4).getValue());
- resultEvent.setType(EventType.parse((String) searchable.get(5).getValue()));
- resultEvent.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- resultEvent.setEndTime(((Number) searchable.get(7).getValue()).longValue());
- // data
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- resultEvent.setMessage((String) data.get(0).getValue());
- resultEvent.setParameters((String) data.get(1).getValue());
- return resultEvent;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index 6bce5767e7..5cef08a726 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -19,21 +19,20 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.apm.network.common.v3.KeyStringValuePair;
import org.apache.skywalking.apm.network.logging.v3.LogTags;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Order;
import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.ContentType;
import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import org.apache.skywalking.oap.server.core.query.type.Log;
import org.apache.skywalking.oap.server.core.query.type.Logs;
@@ -41,17 +40,20 @@ import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
*/
public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQueryDAO {
+ private final StreamMetadata logRecordMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(LogRecord.INDEX_NAME);
+
public BanyanDBLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -90,9 +92,8 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
- if (LogRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(eq(tag.getKey(), tag.getValue()));
- }
+ // TODO: check log indexed tags
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
}
}
}
@@ -103,45 +104,51 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
}
- StreamQueryResponse resp = query(LogRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(logRecordMetadata,
ImmutableList.of(AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
AbstractLogRecord.SPAN_ID), tsRange, query);
- List<Log> logEntities = resp.getElements().stream().map(new LogDeserializer()).collect(Collectors.toList());
-
Logs logs = new Logs();
- logs.getLogs().addAll(logEntities);
- logs.setTotal(logEntities.size());
+ logs.setTotal(resp.size());
+ for (final RowEntity rowEntity : resp.getElements()) {
+ Log log = new Log();
+ log.setServiceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.SERVICE_ID));
+ log.setServiceInstanceId(
+ rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.SERVICE_INSTANCE_ID));
+ log.setEndpointId(
+ rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.ENDPOINT_ID));
+ if (log.getEndpointId() != null) {
+ log.setEndpointName(
+ IDManager.EndpointID.analysisId(log.getEndpointId()).getEndpointName());
+ }
+ log.setTraceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.TRACE_ID));
+ log.setTimestamp(((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE,
+ AbstractLogRecord.TIMESTAMP)).longValue());
+ log.setContentType(ContentType.instanceOf(
+ ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA,
+ AbstractLogRecord.CONTENT_TYPE)).intValue()));
+ log.setContent(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.CONTENT));
+ byte[] dataBinary = rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, AbstractLogRecord.TAGS_RAW_DATA);
+ if (dataBinary != null && dataBinary.length > 0) {
+ parserDataBinary(dataBinary, log.getTags());
+ }
+ logs.getLogs().add(log);
+ }
return logs;
}
- public static class LogDeserializer implements RowEntityDeserializer<Log> {
- @Override
- public Log apply(RowEntity row) {
- Log log = new Log();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- log.setServiceId((String) searchable.get(0).getValue());
- log.setServiceInstanceId((String) searchable.get(1).getValue());
- log.setEndpointId((String) searchable.get(2).getValue());
- log.setTraceId((String) searchable.get(3).getValue());
- log.setTimestamp(row.getTimestamp());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- if (data.get(2).getValue() == null || ((ByteString) data.get(2).getValue()).isEmpty()) {
- log.setContent("");
- } else {
- try {
- // Don't read the tags as they have been in the data binary already.
- LogTags logTags = LogTags.parseFrom((ByteString) data.get(2).getValue());
- for (final KeyStringValuePair pair : logTags.getDataList()) {
- log.getTags().add(new KeyValue(pair.getKey(), pair.getValue()));
- }
- } catch (InvalidProtocolBufferException e) {
- throw new RuntimeException(e);
- }
- }
- return log;
+ /**
+ * Parser the raw tags.
+ * TODO: merge default method
+ */
+ private void parserDataBinary(byte[] dataBinary, List<KeyValue> tags) {
+ try {
+ LogTags logTags = LogTags.parseFrom(dataBinary);
+ logTags.getDataList().forEach(pair -> tags.add(new KeyValue(pair.getKey(), pair.getValue())));
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException(e);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
deleted file mode 100644
index b719090919..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
+++ /dev/null
@@ -1,70 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
-
-import java.io.IOException;
-import java.time.Instant;
-import java.util.Collections;
-
-/**
- * UITemplate insertion DAO
- *
- * @param <T> The only ManagementData we have now is {@link UITemplate}
- */
-public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBanyanDBDAO implements IManagementDAO {
- private final BanyanDBStorageDataBuilder<T> storageBuilder;
-
- public BanyanDBManagementDAO(BanyanDBStorageClient client, BanyanDBStorageDataBuilder<T> storageBuilder) {
- super(client);
- this.storageBuilder = storageBuilder;
- }
-
- @Override
- public void insert(Model model, ManagementData storageData) throws IOException {
- // ensure only insert once
- StreamQueryResponse resp = query(UITemplate.INDEX_NAME,
- Collections.singletonList(UITemplate.NAME),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(eq(UITemplate.NAME, storageData.id()));
- }
- });
-
- if (resp != null && resp.getElements().size() > 0) {
- return;
- }
-
- StreamWrite.StreamWriteBuilder streamWrite = this.storageBuilder
- .entity2Storage((T) storageData)
- .name(model.getName())
- .timestamp(Instant.now().toEpochMilli());
- getClient().write(streamWrite.build());
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
deleted file mode 100644
index 85ea280b72..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ /dev/null
@@ -1,214 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.collect.ImmutableList;
-import com.google.gson.JsonElement;
-import com.google.protobuf.ByteString;
-import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.Layer;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.query.enumeration.Language;
-import org.apache.skywalking.oap.server.core.query.type.Attribute;
-import org.apache.skywalking.oap.server.core.query.type.Endpoint;
-import org.apache.skywalking.oap.server.core.query.type.Service;
-import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
-import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
-import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
-import org.apache.skywalking.oap.server.library.util.StringUtil;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
-
-import java.io.IOException;
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
-import java.util.stream.Collectors;
-
-/**
- * {@link org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic},
- * {@link org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic}
- * {@link org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic}
- * are all streams.
- */
-public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMetadataQueryDAO {
- public BanyanDBMetadataQueryDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public List<Service> listServices(final String layer, final String group) throws IOException {
- StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.SERVICE_ID, ServiceTraffic.GROUP, ServiceTraffic.LAYER),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(ServiceTraffic.SHORT_NAME));
- if (StringUtil.isNotEmpty(layer)) {
- query.appendCondition(eq(ServiceTraffic.LAYER, Layer.valueOf(layer).value()));
- }
- if (StringUtil.isNotEmpty(group)) {
- query.appendCondition(eq(ServiceTraffic.GROUP, group));
- }
- }
- });
-
- return resp.getElements().stream().map(new ServiceDeserializer()).collect(Collectors.toList());
- }
-
- @Override
- public List<Service> getServices(String serviceId) throws IOException {
- StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.SERVICE_ID, ServiceTraffic.GROUP, ServiceTraffic.LAYER),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(ServiceTraffic.SHORT_NAME));
-
- query.appendCondition(eq(ServiceTraffic.SERVICE_ID, serviceId));
- }
- });
-
- return resp.getElements().stream().map(new ServiceDeserializer()).collect(Collectors.toList());
- }
-
- @Override
- public List<ServiceInstance> listInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
- StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList("data_binary"));
-
- final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
- final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
-
- query.appendCondition(gte(InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
- query.appendCondition(lte(InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
- query.appendCondition(eq(InstanceTraffic.SERVICE_ID, serviceId));
- }
- });
-
- return resp.getElements().stream().map(new ServiceInstanceDeserializer()).collect(Collectors.toList());
- }
-
- @Override
- public ServiceInstance getInstance(String instanceId) throws IOException {
- StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(StreamMetaInfo.ID), new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList("data_binary"));
-
- query.appendCondition(eq(StreamMetaInfo.ID, instanceId));
- }
- });
-
- return resp.getElements().stream().map(new ServiceInstanceDeserializer()).findFirst().orElse(null);
- }
-
- @Override
- public List<Endpoint> findEndpoint(String keyword, String serviceId, int limit) throws IOException {
- StreamQueryResponse resp = query(EndpointTraffic.INDEX_NAME,
- ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID), new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(eq(EndpointTraffic.SERVICE_ID, serviceId));
- }
- });
-
- return resp.getElements().stream().map(new EndpointDeserializer()).filter(e -> e.getName().contains(keyword))
- .limit(limit).collect(Collectors.toList());
- }
-
- public static class EndpointDeserializer implements RowEntityDeserializer<Endpoint> {
- @Override
- public Endpoint apply(RowEntity row) {
- Endpoint endpoint = new Endpoint();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- endpoint.setId(row.getId());
- endpoint.setName((String) searchable.get(0).getValue()); // 0 - name
- return endpoint;
- }
- }
-
- public static class ServiceDeserializer implements RowEntityDeserializer<Service> {
- @Override
- public Service apply(RowEntity row) {
- Service service = new Service();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- service.setName((String) searchable.get(0).getValue());
- service.setId((String) searchable.get(1).getValue());
- service.setGroup((String) searchable.get(2).getValue());
- String layerName = Layer.valueOf(((Number) searchable.get(3).getValue()).intValue()).name();
- service.getLayers().add(layerName);
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- service.setShortName((String) data.get(0).getValue());
- return service;
- }
- }
-
- public static class ServiceInstanceDeserializer implements RowEntityDeserializer<ServiceInstance> {
- @Override
- public ServiceInstance apply(RowEntity row) {
- InstanceTraffic instanceTraffic = new InstanceTraffic();
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- Object o = data.get(0).getValue();
- ServiceInstance serviceInstance = new ServiceInstance();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- try {
- RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
- instanceTraffic.deserialize(remoteData);
- serviceInstance.setName(instanceTraffic.getName());
- serviceInstance.setId(row.getId());
- serviceInstance.setInstanceUUID(serviceInstance.getId());
- serviceInstance.setLayer(instanceTraffic.getLayer().name());
-
- if (instanceTraffic.getProperties() != null) {
- for (Map.Entry<String, JsonElement> property : instanceTraffic.getProperties().entrySet()) {
- String key = property.getKey();
- String value = property.getValue().getAsString();
- if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
- serviceInstance.setLanguage(Language.value(value));
- } else {
- serviceInstance.getAttributes().add(new Attribute(key, value));
- }
- }
- } else {
- serviceInstance.setLanguage(Language.UNKNOWN);
- }
- } catch (InvalidProtocolBufferException ex) {
- throw new RuntimeException("fail to parse remote data", ex);
- }
- } else {
- throw new RuntimeException("unable to parse binary data");
- }
-
- return serviceInstance;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
deleted file mode 100644
index f1ed5b8d61..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
+++ /dev/null
@@ -1,57 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
-
-import java.io.IOException;
-import java.util.Collections;
-import java.util.List;
-
-@RequiredArgsConstructor
-public class BanyanDBMetricsDAO<T extends Metrics> implements IMetricsDAO {
- private final BanyanDBStorageDataBuilder<T> storageBuilder;
-
- @Override
- public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
- return Collections.emptyList();
- }
-
- @Override
- public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
- StreamWrite.StreamWriteBuilder builder = this.storageBuilder.entity2Storage((T) metrics)
- .name(model.getName())
- .timestamp(TimeBucket.getTimestamp(metrics.getTimeBucket(), model.getDownsampling()));
- return new BanyanDBStreamInsertRequest(builder.build());
- }
-
- @Override
- public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
- return new UpdateRequest() {
- };
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
deleted file mode 100644
index d5006ea689..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ /dev/null
@@ -1,76 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-
-import java.util.List;
-import java.util.stream.Collectors;
-
-/**
- * {@link NetworkAddressAlias} is a stream
- */
-public class BanyanDBNetworkAddressAliasDAO extends AbstractBanyanDBDAO implements INetworkAddressAliasDAO {
- public BanyanDBNetworkAddressAliasDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
- StreamQueryResponse resp = query(NetworkAddressAlias.INDEX_NAME,
- ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id"));
- query.appendCondition(gte(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
- }
- });
-
- return resp.getElements().stream().map(new NetworkAddressAliasDeserializer()).collect(Collectors.toList());
- }
-
- public static class NetworkAddressAliasDeserializer implements RowEntityDeserializer<NetworkAddressAlias> {
- @Override
- public NetworkAddressAlias apply(RowEntity row) {
- NetworkAddressAlias model = new NetworkAddressAlias();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // searchable - last_update_time_bucket
- model.setLastUpdateTimeBucket(((Number) searchable.get(0).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // data 0 - time_bucket
- model.setTimeBucket(((Number) data.get(0).getValue()).longValue());
- // data 1 - address
- model.setAddress((String) data.get(1).getValue());
- // data 2 - represent_service_id
- model.setRepresentServiceId((String) data.get(2).getValue());
- // data 3 - represent_service_instance_id
- model.setRepresentServiceInstanceId((String) data.get(3).getValue());
- return model;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 1da585bb4b..4411957f69 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -22,22 +22,25 @@ import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationType;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
-import java.util.Comparator;
+import java.util.LinkedList;
import java.util.List;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
*/
public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskLogQueryDAO {
+ private final StreamMetadata profileTaskLogRecord =
+ MetadataRegistry.INSTANCE.findStreamMetadata(ProfileTaskLogRecord.INDEX_NAME);
+
private final int queryMaxSize;
public BanyanDBProfileTaskLogQueryDAO(BanyanDBStorageClient client, int queryMaxSize) {
@@ -47,7 +50,7 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
- StreamQueryResponse resp = query(ProfileTaskLogRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(profileTaskLogRecord,
ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME, ProfileTaskLogRecord.INSTANCE_ID),
new QueryBuilder() {
@Override
@@ -58,26 +61,26 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
}
});
- return resp.getElements().stream().map(new ProfileTaskLogDeserializer())
- .sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
- .collect(Collectors.toList());
+ final LinkedList<ProfileTaskLog> tasks = new LinkedList<>();
+ for (final RowEntity rowEntity : resp.getElements()) {
+ tasks.add(parseTaskLog(rowEntity));
+ }
+
+ return tasks;
}
- public static class ProfileTaskLogDeserializer implements RowEntityDeserializer<ProfileTaskLog> {
- @Override
- public ProfileTaskLog apply(RowEntity row) {
- ProfileTaskLog profileTaskLog = new ProfileTaskLog();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // searchable - operation_time
- profileTaskLog.setOperationTime(((Number) searchable.get(0).getValue()).longValue());
- // searchable - instance_id
- profileTaskLog.setInstanceId((String) searchable.get(1).getValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // data - task_id
- profileTaskLog.setTaskId((String) data.get(0).getValue());
- // data - operation_type
- profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(1).getValue()).intValue()));
- return profileTaskLog;
- }
+ private ProfileTaskLog parseTaskLog(RowEntity data) {
+ return ProfileTaskLog.builder()
+ .id(data.getId())
+ .taskId(data.getValue(StreamMetadata.TAG_FAMILY_DATA, ProfileTaskLogRecord.TASK_ID))
+ .instanceId(
+ data.getValue(StreamMetadata.TAG_FAMILY_DATA, ProfileTaskLogRecord.INSTANCE_ID))
+ .operationType(ProfileTaskLogOperationType.parse(
+ ((Number) data.getValue(StreamMetadata.TAG_FAMILY_DATA,
+ ProfileTaskLogRecord.OPERATION_TYPE)).intValue()))
+ .operationTime(
+ ((Number) data.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE,
+ ProfileTaskLogRecord.OPERATION_TIME)).longValue())
+ .build();
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
deleted file mode 100644
index 4bf0996eb8..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ /dev/null
@@ -1,120 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
-import org.apache.skywalking.oap.server.library.util.StringUtil;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
-
-import java.io.IOException;
-import java.util.List;
-import java.util.Objects;
-import java.util.stream.Collectors;
-
-/**
- * {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
- */
-public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskQueryDAO {
- public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
- StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
- ImmutableList.of(StreamMetaInfo.ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
- ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
- ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT), new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(eq(ProfileTaskRecord.SERVICE_ID, serviceId));
- }
-
- if (StringUtil.isNotEmpty(endpointName)) {
- query.appendCondition(eq(ProfileTaskRecord.ENDPOINT_NAME, endpointName));
- }
-
- if (Objects.nonNull(startTimeBucket)) {
- query.appendCondition(gte(ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
- }
-
- if (Objects.nonNull(endTimeBucket)) {
- query.appendCondition(lte(ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
- }
-
- if (Objects.nonNull(limit)) {
- query.setLimit(limit);
- }
-
-// query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
- }
- });
-
- return resp.getElements().stream().map(new ProfileTaskDeserializer()).collect(Collectors.toList());
- }
-
- @Override
- public ProfileTask getById(String id) throws IOException {
- if (StringUtil.isEmpty(id)) {
- return null;
- }
-
- StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
- ImmutableList.of(StreamMetaInfo.ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
- ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
- ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(eq(StreamMetaInfo.ID, id));
- query.setLimit(1);
- }
- });
-
- return resp.getElements().stream().map(new ProfileTaskDeserializer()).findAny().orElse(null);
- }
-
- public static class ProfileTaskDeserializer implements RowEntityDeserializer<ProfileTask> {
- @Override
- public ProfileTask apply(RowEntity row) {
- ProfileTask profileTask = new ProfileTask();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- profileTask.setId((String) searchable.get(0).getValue());
- profileTask.setServiceId((String) searchable.get(1).getValue());
- profileTask.setEndpointName((String) searchable.get(2).getValue());
- profileTask.setStartTime(((Number) searchable.get(3).getValue()).longValue());
- profileTask.setDuration(((Number) searchable.get(4).getValue()).intValue());
- profileTask.setMinDurationThreshold(((Number) searchable.get(5).getValue()).intValue());
- profileTask.setDumpPeriod(((Number) searchable.get(6).getValue()).intValue());
- profileTask.setCreateTime(((Number) searchable.get(7).getValue()).intValue());
- profileTask.setMaxSamplingCount(((Number) searchable.get(8).getValue()).intValue());
- return profileTask;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index 6df30bb465..4c6ac7ddcd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -19,22 +19,25 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
+import java.util.LinkedList;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Collectors;
@@ -43,13 +46,22 @@ import java.util.stream.Collectors;
* {@link ProfileThreadSnapshotRecord} is a stream
*/
public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO implements IProfileThreadSnapshotQueryDAO {
+ protected final ProfileThreadSnapshotRecord.Builder builder =
+ new ProfileThreadSnapshotRecord.Builder();
+
+ private final StreamMetadata profileThreadSnapshotMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(ProfileThreadSnapshotRecord.INDEX_NAME);
+
+ private final StreamMetadata segmentRecordMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(SegmentRecord.INDEX_NAME);
+
public BanyanDBProfileThreadSnapshotQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
- StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
new QueryBuilder() {
@@ -64,22 +76,39 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
return Collections.emptyList();
}
- List<String> segmentIDs = resp.getElements().stream()
- .map(new ProfileThreadSnapshotRecordDeserializer())
- .map(ProfileThreadSnapshotRecord::getSegmentId)
- .collect(Collectors.toList());
+ final List<String> segmentIds = new LinkedList<>();
+ for (final RowEntity rowEntity : resp.getElements()) {
+ segmentIds.add(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, ProfileThreadSnapshotRecord.SEGMENT_ID));
+ }
// TODO: support `IN` or `OR` logic operation in BanyanDB
List<BasicTrace> basicTraces = new ArrayList<>();
- for (String segmentID : segmentIDs) {
- final StreamQueryResponse segmentRecordResp = query(SegmentRecord.INDEX_NAME, ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"),
+ for (String segmentID : segmentIds) {
+ final StreamQueryResponse segmentRecordResp = query(segmentRecordMetadata,
+ ImmutableList.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery traceQuery) {
traceQuery.appendCondition(eq(SegmentRecord.SEGMENT_ID, segmentID));
}
});
- basicTraces.addAll(segmentRecordResp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList()));
+
+ for (final RowEntity row : segmentRecordResp.getElements()) {
+ BasicTrace basicTrace = new BasicTrace();
+
+ basicTrace.setSegmentId(row.getId());
+ basicTrace.setStart(String.valueOf(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)));
+ basicTrace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.ENDPOINT_ID)
+ ).getEndpointName());
+ basicTrace.setDuration(((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ basicTrace.setError(BooleanUtils.valueToBoolean(
+ ((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue()
+ ));
+ basicTrace.getTraceIds().add(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
+
+ basicTraces.add(basicTrace);
+ }
}
// TODO: Sort in DB with DESC
@@ -104,7 +133,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
- StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
new QueryBuilder() {
@@ -118,26 +147,52 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
}
});
- return resp.getElements().stream().map(new ProfileThreadSnapshotRecordDeserializer()).collect(Collectors.toList());
+ List<ProfileThreadSnapshotRecord> result = new ArrayList<>(maxSequence - minSequence);
+ for (final RowEntity rowEntity : resp.getElements()) {
+ ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
+ new BanyanDBConverter.StreamToEntity(profileThreadSnapshotMetadata, rowEntity));
+
+ result.add(record);
+ }
+ return result;
}
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
- StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(segmentRecordMetadata,
ImmutableList.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.SERVICE_ID, SegmentRecord.SERVICE_INSTANCE_ID, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList("data_binary"));
+ query.setDataProjections(Collections.singletonList(SegmentRecord.DATA_BINARY));
query.appendCondition(eq(SegmentRecord.INDEX_NAME, segmentId));
}
});
- return resp.getElements().stream().map(new SegmentRecordDeserializer()).findFirst().orElse(null);
+ if (resp.size() == 0) {
+ return null;
+ }
+
+ final RowEntity rowEntity = resp.getElements().iterator().next();
+ final SegmentRecord segmentRecord = new SegmentRecord();
+ segmentRecord.setSegmentId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.SEGMENT_ID));
+ segmentRecord.setTraceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
+ segmentRecord.setServiceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.SERVICE_ID));
+ segmentRecord.setStartTime(
+ ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)).longValue());
+ segmentRecord.setLatency(
+ ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ segmentRecord.setIsError(
+ ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue());
+ byte[] dataBinary = rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, SegmentRecord.DATA_BINARY);
+ if (dataBinary != null && dataBinary.length > 0) {
+ segmentRecord.setDataBinary(dataBinary);
+ }
+ return segmentRecord;
}
private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) {
- StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
new QueryBuilder() {
@@ -151,7 +206,13 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
}
});
- List<ProfileThreadSnapshotRecord> records = resp.getElements().stream().map(new ProfileThreadSnapshotRecordDeserializer()).collect(Collectors.toList());
+ List<ProfileThreadSnapshotRecord> records = new ArrayList<>();
+ for (final RowEntity rowEntity : resp.getElements()) {
+ ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
+ new BanyanDBConverter.StreamToEntity(profileThreadSnapshotMetadata, rowEntity));
+
+ records.add(record);
+ }
switch (aggType) {
case MIN:
@@ -176,55 +237,4 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
enum AggType {
MIN, MAX
}
-
- public static class ProfileThreadSnapshotRecordDeserializer implements RowEntityDeserializer<ProfileThreadSnapshotRecord> {
- @Override
- public ProfileThreadSnapshotRecord apply(RowEntity row) {
- ProfileThreadSnapshotRecord record = new ProfileThreadSnapshotRecord();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- record.setTaskId((String) searchable.get(0).getValue());
- record.setSegmentId((String) searchable.get(1).getValue());
- record.setDumpTime(((Number) searchable.get(2).getValue()).longValue());
- record.setSequence(((Number) searchable.get(3).getValue()).intValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- record.setStackBinary(((ByteString) data.get(0).getValue()).toByteArray());
- return record;
- }
- }
-
- public static class SegmentRecordDeserializer implements RowEntityDeserializer<SegmentRecord> {
- @Override
- public SegmentRecord apply(RowEntity row) {
- SegmentRecord record = new SegmentRecord();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- record.setSegmentId(row.getId());
- record.setTraceId((String) searchable.get(0).getValue());
- record.setIsError(((Number) searchable.get(1).getValue()).intValue());
- record.setServiceId((String) searchable.get(2).getValue());
- record.setServiceInstanceId((String) searchable.get(3).getValue());
- record.setEndpointId((String) searchable.get(4).getValue());
- record.setLatency(((Number) searchable.get(5).getValue()).intValue());
- record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
- return record;
- }
- }
-
- public static class BasicTraceDeserializer implements RowEntityDeserializer<BasicTrace> {
- @Override
- public BasicTrace apply(RowEntity row) {
- BasicTrace trace = new BasicTrace();
- trace.setSegmentId(row.getId());
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- trace.getTraceIds().add((String) searchable.get(0).getValue());
- trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
- trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- (String) searchable.get(2).getValue()
- ).getEndpointName());
- trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
- trace.setStart(String.valueOf(searchable.get(4).getValue()));
- return trace;
- }
- }
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index d028fa91c3..a1cf73edcd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -24,21 +24,34 @@ import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.Convert2Storage;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
@RequiredArgsConstructor
-public class BanyanDBRecordDAO<T extends Record> implements IRecordDAO {
- private final BanyanDBStorageDataBuilder<T> storageBuilder;
+public class BanyanDBRecordDAO implements IRecordDAO {
+ private final StorageBuilder<Record> storageBuilder;
@Override
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- StreamWrite.StreamWriteBuilder builder = storageBuilder.entity2Storage((T) record)
- .name(model.getName())
- .timestamp(TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling()));
+ StreamMetadata metadata = MetadataRegistry.INSTANCE.findStreamMetadata(model.getName());
+ if (metadata == null) {
+ throw new IOException(model.getName() + " is not registered");
+ }
+ StreamWrite streamWrite = new StreamWrite(metadata.getGroup(), // group name
+ model.getName(), // index-name
+ record.id(), // identity
+ TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling()), // timestamp
+ metadata.getDataFamilySize(), // length of the "data" tag family
+ metadata.getSearchableFamilySize()); // length of the "searchable" tag family
+ Convert2Storage<StreamWrite> convert2Storage = new BanyanDBConverter.StreamToStorage(metadata, streamWrite);
+ storageBuilder.entity2Storage(record, convert2Storage);
- return new BanyanDBStreamInsertRequest(builder.build());
+ return new BanyanDBStreamInsertRequest(convert2Storage.obtain());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
index 3c3b4b4a6c..c3f75c8cef 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -23,24 +23,17 @@ import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
-import org.apache.skywalking.oap.server.core.analysis.topn.TopN;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.StorageDAO;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBManagementDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBMetricsDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBNoneStreamDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
-
-import java.io.IOException;
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
@Slf4j
public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> implements StorageDAO {
@@ -50,69 +43,21 @@ public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> imple
@Override
public IMetricsDAO newMetricsDao(StorageBuilder storageBuilder) {
- // SKIP:
- // 1. OAL runtime metrics builder
- // 2. Analysis Function builder
- if (storageBuilder.getClass().getName().startsWith("org.apache.skywalking.oap.server.core.")) {
- log.warn("metrics builder {} is not supported yet", storageBuilder.getClass());
- return new IMetricsDAO() {
- @Override
- public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
- return Collections.emptyList();
- }
-
- @Override
- public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
- return new InsertRequest() {
- };
- }
-
- @Override
- public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
- return new UpdateRequest() {
- };
- }
- };
- }
- return new BanyanDBMetricsDAO<>((BanyanDBStorageDataBuilder<Metrics>) storageBuilder);
+ return new BanyanDBMetricsDAO((StorageBuilder<Metrics>) storageBuilder);
}
@Override
public IRecordDAO newRecordDao(StorageBuilder storageBuilder) {
- try {
- final Class<?> returnType = storageBuilder.getClass().getDeclaredMethod("storage2Entity", Map.class).getReturnType();
- // FIXME: this is currently a hack to avoid TopN insertion since we will impl TopN later in BanyanDB side
- if (TopN.class.isAssignableFrom(returnType)) {
- return new IRecordDAO() {
- @Override
- public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- return new InsertRequest() {
- };
- }
- };
- } else if (returnType.getName().equals("org.apache.skywalking.oap.server.storage.plugin.zipkin.ZipkinSpanRecord")) {
- // SKIP ZipkinSpanRecord
- return new IRecordDAO() {
- @Override
- public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- return new InsertRequest() {
- };
- }
- };
- }
- } catch (NoSuchMethodException ex) {
- log.error("fail to get declared method");
- }
- return new BanyanDBRecordDAO<>((BanyanDBStorageDataBuilder<Record>) storageBuilder);
+ return new BanyanDBRecordDAO((StorageBuilder<Record>) storageBuilder);
}
@Override
public INoneStreamDAO newNoneStreamDao(StorageBuilder storageBuilder) {
- return new BanyanDBNoneStreamDAO<>(getClient(), (BanyanDBStorageDataBuilder<NoneStream>) storageBuilder);
+ return new BanyanDBNoneStreamDAO(getClient(), (StorageBuilder<NoneStream>) storageBuilder);
}
@Override
public IManagementDAO newManagementDao(StorageBuilder storageBuilder) {
- return new BanyanDBManagementDAO<>(getClient(), (BanyanDBStorageDataBuilder<ManagementData>) storageBuilder);
+ return new BanyanDBManagementDAO(getClient(), (StorageBuilder<ManagementData>) storageBuilder);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
index f8287973bc..b04e4422d5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
@@ -24,7 +24,7 @@ import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
@RequiredArgsConstructor
+@Getter
public class BanyanDBStreamInsertRequest implements InsertRequest {
- @Getter
private final StreamWrite streamWrite;
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
deleted file mode 100644
index 660190dd10..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
+++ /dev/null
@@ -1,276 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableSet;
-import io.vavr.Tuple2;
-import io.vavr.Tuple4;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TimestampRange;
-import org.apache.skywalking.oap.server.core.Const;
-import org.apache.skywalking.oap.server.core.UnexpectedException;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
-import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
-import org.apache.skywalking.oap.server.core.query.type.Call;
-import org.apache.skywalking.oap.server.core.source.DetectPoint;
-import org.apache.skywalking.oap.server.core.storage.query.ITopologyQueryDAO;
-import org.apache.skywalking.oap.server.library.util.CollectionUtils;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-import java.util.Set;
-import java.util.stream.Collectors;
-
-public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITopologyQueryDAO {
- public BanyanDBTopologyQueryDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
- if (CollectionUtils.isEmpty(serviceIds)) {
- throw new UnexpectedException("Service id is empty");
- }
-
- return loadServiceCalls(
- ServiceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceRelationServerSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationServerSideMetrics.DEST_SERVICE_ID, serviceIds, DetectPoint.SERVER
- );
- }
-
- @Override
- public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB, List<String> serviceIds) throws IOException {
- if (CollectionUtils.isEmpty(serviceIds)) {
- throw new UnexpectedException("Service id is empty");
- }
-
- return loadServiceCalls(
- ServiceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceRelationClientSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationClientSideMetrics.DEST_SERVICE_ID, serviceIds, DetectPoint.CLIENT
- );
- }
-
- @Override
- public List<Call.CallDetail> loadServiceRelationsDetectedAtServerSide(long startTB, long endTB) throws IOException {
- return loadServiceCalls(
- ServiceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceRelationServerSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationServerSideMetrics.DEST_SERVICE_ID, Collections.emptyList(), DetectPoint.SERVER
- );
- }
-
- @Override
- public List<Call.CallDetail> loadServiceRelationDetectedAtClientSide(long startTB, long endTB) throws IOException {
- return loadServiceCalls(
- ServiceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceRelationClientSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationClientSideMetrics.DEST_SERVICE_ID, Collections.emptyList(), DetectPoint.CLIENT
- );
- }
-
- @Override
- public List<Call.CallDetail> loadInstanceRelationDetectedAtServerSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
- return loadServiceInstanceCalls(
- ServiceInstanceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceInstanceRelationServerSideMetrics.SOURCE_SERVICE_ID,
- ServiceInstanceRelationServerSideMetrics.DEST_SERVICE_ID, clientServiceId, serverServiceId,
- DetectPoint.SERVER
- );
- }
-
- @Override
- public List<Call.CallDetail> loadInstanceRelationDetectedAtClientSide(String clientServiceId, String serverServiceId, long startTB, long endTB) throws IOException {
- return loadServiceInstanceCalls(
- ServiceInstanceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
- ServiceInstanceRelationClientSideMetrics.SOURCE_SERVICE_ID,
- ServiceInstanceRelationClientSideMetrics.DEST_SERVICE_ID, clientServiceId, serverServiceId,
- DetectPoint.CLIENT
- );
- }
-
- @Override
- public List<Call.CallDetail> loadEndpointRelation(long startTB, long endTB, String destEndpointId) throws IOException {
- return loadEndpointCalls(
- EndpointRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
- EndpointRelationServerSideMetrics.SOURCE_ENDPOINT,
- EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId
- );
- }
-
- private List<Call.CallDetail> loadServiceCalls(String tableName,
- long startTB,
- long endTB,
- String sourceCName,
- String destCName,
- List<String> serviceIds,
- DetectPoint detectPoint) throws IOException {
- TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
- List<Call.CallDetail> calls = new ArrayList<>();
- if (serviceIds.isEmpty()) {
- StreamQueryResponse resp = query(tableName, Collections.emptyList(), timeRange, new QueryBuilder() {
- @Override
- void apply(StreamQuery query) {
- // query component_id
- query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
- }
- });
-
- calls.addAll(resp.getElements().stream().map(new ServiceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
- } else {
- for (String fieldOfInterest : ImmutableList.of(sourceCName, destCName)) {
- for (String serviceID : serviceIds) {
- StreamQueryResponse resp = query(tableName, ImmutableList.of(fieldOfInterest), timeRange, new QueryBuilder() {
- @Override
- void apply(StreamQuery query) {
- // query component_id
- query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
-
- query.appendCondition(eq(fieldOfInterest, serviceID));
- }
- });
-
- calls.addAll(resp.getElements().stream().map(new ServiceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
- }
- }
- }
-
- return calls;
- }
-
- private List<Call.CallDetail> loadServiceInstanceCalls(String tableName,
- long startTB,
- long endTB,
- String sourceCName,
- String descCName,
- String sourceServiceId,
- String destServiceId,
- DetectPoint detectPoint) throws IOException {
- TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
-
- Set<Tuple4<String, String, String, String>> productQuerySet = ImmutableSet.of(
- new Tuple4<>(sourceCName, sourceServiceId, descCName, destServiceId),
- new Tuple4<>(sourceCName, destServiceId, descCName, sourceServiceId)
- );
-
- List<Call.CallDetail> calls = new ArrayList<>();
-
- for (Tuple4<String, String, String, String> querySet : productQuerySet) {
- StreamQueryResponse resp = query(tableName, ImmutableList.of(querySet._1(), querySet._3()), timeRange, new QueryBuilder() {
- @Override
- void apply(StreamQuery query) {
- // query component_id
- query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
-
- query.appendCondition(eq(querySet._1(), querySet._2()));
- query.appendCondition(eq(querySet._3(), querySet._4()));
- }
- });
-
- calls.addAll(resp.getElements().stream().map(new InstanceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
- }
-
- return calls;
- }
-
- private List<Call.CallDetail> loadEndpointCalls(String tableName,
- long startTB,
- long endTB,
- String sourceCName,
- String destCName,
- String id) throws IOException {
- TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
-
- Set<Tuple2<String, String>> allPossibleQuerySet = ImmutableSet.of(
- new Tuple2<>(sourceCName, id),
- new Tuple2<>(destCName, id)
- );
-
- List<Call.CallDetail> calls = new ArrayList<>();
-
- for (Tuple2<String, String> querySet : allPossibleQuerySet) {
- StreamQueryResponse resp = query(tableName, ImmutableList.of(querySet._1()), timeRange, new QueryBuilder() {
- @Override
- void apply(StreamQuery query) {
- query.appendCondition(eq(querySet._1(), querySet._2()));
- }
- });
-
- calls.addAll(resp.getElements().stream().map(new EndpointCallDetailDeserializer()).collect(Collectors.toList()));
- }
-
- return calls;
- }
-
- @RequiredArgsConstructor
- public static class ServiceCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
- private final DetectPoint detectPoint;
-
- @Override
- public Call.CallDetail apply(RowEntity rowEntity) {
- Call.CallDetail call = new Call.CallDetail();
- String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
- String entityId = idsSlice[1];
- int componentId = ((Number) rowEntity.getTagFamilies().get(1) // Tag Family: "data"
- .get(0).getValue()).intValue();
- call.buildFromServiceRelation(entityId, componentId, this.detectPoint);
- return call;
- }
- }
-
- @RequiredArgsConstructor
- public static class InstanceCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
- private final DetectPoint detectPoint;
-
- @Override
- public Call.CallDetail apply(RowEntity rowEntity) {
- Call.CallDetail call = new Call.CallDetail();
- String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
- String entityId = idsSlice[1];
- int componentId = ((Number) rowEntity.getTagFamilies().get(1) // Tag Family: "data"
- .get(0).getValue()).intValue();
- call.buildFromInstanceRelation(entityId, componentId, this.detectPoint);
- return call;
- }
- }
-
- @RequiredArgsConstructor
- public static class EndpointCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
- @Override
- public Call.CallDetail apply(RowEntity rowEntity) {
- Call.CallDetail call = new Call.CallDetail();
- String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
- String entityId = idsSlice[1];
- call.buildFromEndpointRelation(entityId, DetectPoint.SERVER);
- return call;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 642ed519fc..fc0d458c29 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -20,11 +20,9 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
@@ -38,15 +36,20 @@ import org.apache.skywalking.oap.server.core.query.type.TraceState;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-import java.util.stream.Collectors;
public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
+ private final StreamMetadata segmentRecordMetadata =
+ MetadataRegistry.INSTANCE.findStreamMetadata(SegmentRecord.INDEX_NAME);
+
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -97,9 +100,8 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
- if (SegmentRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(eq(tag.getKey(), tag.getValue()));
- }
+ // TODO: check if we have this tag indexed?
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
}
}
@@ -114,7 +116,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
- StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
+ StreamQueryResponse resp = query(segmentRecordMetadata,
ImmutableList.of(SegmentRecord.TRACE_ID, // 0 - trace_id
SegmentRecord.IS_ERROR, // 1 - is_error
SegmentRecord.SERVICE_ID, // 2 - service_id
@@ -124,24 +126,39 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
SegmentRecord.START_TIME), // 6 - start_time
tsRange, q);
- List<BasicTrace> basicTraces = resp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList());
+ TraceBrief traceBrief = new TraceBrief();
+ traceBrief.setTotal(resp.getElements().size());
+
+ for (final RowEntity row : resp.getElements()) {
+ BasicTrace basicTrace = new BasicTrace();
+
+ basicTrace.setSegmentId(row.getId());
+ basicTrace.setStart(String.valueOf(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)));
+ basicTrace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.ENDPOINT_ID)
+ ).getEndpointName());
+ basicTrace.setDuration(((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ basicTrace.setError(BooleanUtils.valueToBoolean(
+ ((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue()
+ ));
+ basicTrace.getTraceIds().add(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
+
+ traceBrief.getTraces().add(basicTrace);
+ }
- TraceBrief brief = new TraceBrief();
- brief.setTotal(basicTraces.size());
- brief.getTraces().addAll(basicTraces);
- return brief;
+ return traceBrief;
}
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
- StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of(SegmentRecord.TRACE_ID, // 0 - trace_id
- SegmentRecord.IS_ERROR, // 1 - is_error
- SegmentRecord.SERVICE_ID, // 2 - service_id
- SegmentRecord.SERVICE_INSTANCE_ID, // 3 - service_instance_id
- SegmentRecord.ENDPOINT_ID, // 4 - endpoint_id
- SegmentRecord.LATENCY, // 5 - latency
- SegmentRecord.START_TIME), // 6 - start_time
+ StreamQueryResponse resp = query(segmentRecordMetadata,
+ ImmutableList.of(SegmentRecord.TRACE_ID,
+ SegmentRecord.IS_ERROR,
+ SegmentRecord.SERVICE_ID,
+ SegmentRecord.SERVICE_INSTANCE_ID,
+ SegmentRecord.ENDPOINT_ID,
+ SegmentRecord.LATENCY,
+ SegmentRecord.START_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
@@ -150,47 +167,19 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
}
});
- return resp.getElements().stream().map(new SegmentRecordDeserializer()).collect(Collectors.toList());
+ List<SegmentRecord> segmentRecords = new ArrayList<>(resp.getElements().size());
+
+ for (final RowEntity rowEntity : resp.getElements()) {
+ SegmentRecord segmentRecord = new SegmentRecord.Builder().storage2Entity(
+ new BanyanDBConverter.StreamToEntity(segmentRecordMetadata, rowEntity));
+ segmentRecords.add(segmentRecord);
+ }
+
+ return segmentRecords;
}
@Override
public List<Span> doFlexibleTraceQuery(String traceId) throws IOException {
return Collections.emptyList();
}
-
- public static class BasicTraceDeserializer implements RowEntityDeserializer<BasicTrace> {
- @Override
- public BasicTrace apply(RowEntity row) {
- BasicTrace trace = new BasicTrace();
- trace.setSegmentId(row.getId());
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- trace.getTraceIds().add((String) searchable.get(0).getValue());
- trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
- trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- (String) searchable.get(4).getValue()
- ).getEndpointName());
- trace.setDuration(((Long) searchable.get(5).getValue()).intValue());
- trace.setStart(String.valueOf(searchable.get(6).getValue()));
- return trace;
- }
- }
-
- public static class SegmentRecordDeserializer implements RowEntityDeserializer<SegmentRecord> {
- @Override
- public SegmentRecord apply(RowEntity row) {
- SegmentRecord record = new SegmentRecord();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- record.setSegmentId(row.getId());
- record.setTraceId((String) searchable.get(0).getValue());
- record.setIsError(((Number) searchable.get(1).getValue()).intValue());
- record.setServiceId((String) searchable.get(2).getValue());
- record.setServiceInstanceId((String) searchable.get(3).getValue());
- record.setEndpointId((String) searchable.get(4).getValue());
- record.setLatency(((Number) searchable.get(5).getValue()).intValue());
- record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
- return record;
- }
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
deleted file mode 100644
index 7888b0f56e..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ /dev/null
@@ -1,119 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.banyandb.v1.client.Tag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.query.enumeration.TemplateType;
-import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
-import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
-import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
-import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
-import org.apache.skywalking.oap.server.library.util.BooleanUtils;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-
-import java.io.IOException;
-import java.time.Instant;
-import java.util.List;
-import java.util.stream.Collectors;
-
-/**
- * {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
- */
-public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO implements UITemplateManagementDAO {
- public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
- super(client);
- }
-
- @Override
- public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- StreamQueryResponse resp = query(UITemplate.INDEX_NAME, ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED),
- new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE));
- query.setLimit(10000);
- if (!includingDisabled) {
- query.appendCondition(eq(UITemplate.DISABLED, BooleanUtils.FALSE));
- }
- }
- });
-
- return resp.getElements().stream().map(new DashboardConfigurationDeserializer()).collect(Collectors.toList());
- }
-
- @Override
- public TemplateChangeStatus addTemplate(DashboardSetting setting) throws IOException {
- final UITemplate uiTemplate = setting.toEntity();
-
- StreamWrite request = StreamWrite.builder()
- .name(UITemplate.INDEX_NAME)
- // searchable - name
- .searchableTag(Tag.stringField(uiTemplate.getName()))
- // searchable - disabled
- .searchableTag(Tag.longField(uiTemplate.getDisabled()))
- // data - type
- .dataTag(Tag.stringField(uiTemplate.getType()))
- // data - configuration
- .dataTag(Tag.stringField(uiTemplate.getConfiguration()))
- // data - activated
- .dataTag(Tag.longField(uiTemplate.getActivated()))
- .timestamp(Instant.now().toEpochMilli())
- .elementId(uiTemplate.id())
- .build();
- getClient().write(request);
- return TemplateChangeStatus.builder().status(true).build();
- }
-
- @Override
- public TemplateChangeStatus changeTemplate(DashboardSetting setting) throws IOException {
- return TemplateChangeStatus.builder().status(false).message("Can't update the template").build();
- }
-
- @Override
- public TemplateChangeStatus disableTemplate(String name) throws IOException {
- return TemplateChangeStatus.builder().status(false).message("Can't disable the template").build();
- }
-
- public static class DashboardConfigurationDeserializer implements RowEntityDeserializer<DashboardConfiguration> {
- @Override
- public DashboardConfiguration apply(RowEntity row) {
- DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // name
- dashboardConfiguration.setName((String) searchable.get(0).getValue());
- // disabled
- dashboardConfiguration.setDisabled(BooleanUtils.valueToBoolean(((Number) searchable.get(1).getValue()).intValue()));
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // activated
- dashboardConfiguration.setActivated(BooleanUtils.valueToBoolean(((Number) data.get(0).getValue()).intValue()));
- // configuration
- dashboardConfiguration.setConfiguration((String) data.get(1).getValue());
- // type
- dashboardConfiguration.setType(TemplateType.forName((String) data.get(2).getValue()));
- return dashboardConfiguration;
- }
- }
-}
[skywalking] 23/24: refactor converter and measure reigstration
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit e078a3bbc0ee17c46bb3529a648af31833d887cc
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Mon Apr 25 14:40:06 2022 +0800
refactor converter and measure reigstration
---
.../storage/plugin/banyandb/BanyanDBConverter.java | 114 ++++++++--
.../plugin/banyandb/BanyanDBIndexInstaller.java | 8 +-
.../plugin/banyandb/BanyanDBMetricsDAO.java | 15 ++
.../plugin/banyandb/BanyanDBStorageClient.java | 28 ++-
.../plugin/banyandb/BanyanDBStorageProvider.java | 3 +-
.../BanyanDBUITemplateManagementDAO.java | 2 +-
.../storage/plugin/banyandb/MetadataRegistry.java | 234 ++++++++++++++++-----
.../banyandb/measure/BanyanDBMetadataQueryDAO.java | 27 ++-
.../banyandb/stream/AbstractBanyanDBDAO.java | 46 +++-
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 9 +-
.../stream/BanyanDBBrowserLogQueryDAO.java | 7 +-
.../banyandb/stream/BanyanDBLogQueryDAO.java | 7 +-
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 7 +-
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 30 ++-
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 6 +-
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 13 +-
.../storage/plugin/banyandb/util/ByteUtil.java | 33 +++
.../metadata/endpoint_relation_server_side.json | 41 ----
.../main/resources/metadata/endpoint_traffic.json | 31 ---
.../src/main/resources/metadata/events.json | 64 ------
.../metadata/index_rules/component_id.json | 12 --
.../metadata/index_rules/create_time.json | 12 --
.../metadata/index_rules/db.instance.json | 12 --
.../resources/metadata/index_rules/db.type.json | 12 --
.../metadata/index_rules/dest_endpoint.json | 12 --
.../metadata/index_rules/dest_service_id.json | 12 --
.../resources/metadata/index_rules/disabled.json | 12 --
.../metadata/index_rules/dump_period.json | 12 --
.../resources/metadata/index_rules/dump_time.json | 12 --
.../resources/metadata/index_rules/duration.json | 12 --
.../resources/metadata/index_rules/end_time.json | 12 --
.../resources/metadata/index_rules/endpoint.json | 12 --
.../metadata/index_rules/endpoint_id.json | 12 --
.../metadata/index_rules/endpoint_name.json | 12 --
.../resources/metadata/index_rules/entity_id.json | 12 --
.../metadata/index_rules/http.method.json | 12 --
.../main/resources/metadata/index_rules/id.json | 12 --
.../resources/metadata/index_rules/last_ping.json | 12 --
.../index_rules/last_update_time_bucket.json | 12 --
.../resources/metadata/index_rules/latency.json | 12 --
.../main/resources/metadata/index_rules/layer.json | 12 --
.../metadata/index_rules/max_sampling_count.json | 12 --
.../index_rules/min_duration_threshold.json | 12 --
.../resources/metadata/index_rules/mq.broker.json | 12 --
.../resources/metadata/index_rules/mq.queue.json | 12 --
.../resources/metadata/index_rules/mq.topic.json | 12 --
.../main/resources/metadata/index_rules/name.json | 12 --
.../metadata/index_rules/operation_time.json | 12 --
.../resources/metadata/index_rules/segment_id.json | 12 --
.../resources/metadata/index_rules/sequence.json | 12 --
.../resources/metadata/index_rules/service.json | 12 --
.../metadata/index_rules/service_group.json | 12 --
.../resources/metadata/index_rules/service_id.json | 12 --
.../metadata/index_rules/service_instance.json | 12 --
.../metadata/index_rules/source_endpoint.json | 12 --
.../metadata/index_rules/source_service_id.json | 12 --
.../resources/metadata/index_rules/start_time.json | 12 --
.../metadata/index_rules/status_code.json | 12 --
.../resources/metadata/index_rules/trace_id.json | 12 --
.../main/resources/metadata/index_rules/type.json | 12 --
.../main/resources/metadata/index_rules/uuid.json | 12 --
.../main/resources/metadata/instance_traffic.json | 40 ----
.../resources/metadata/network_address_alias.json | 45 ----
.../src/main/resources/metadata/profile_task.json | 59 ------
.../main/resources/metadata/profile_task_log.json | 40 ----
.../metadata/profile_task_segment_snapshot.json | 44 ----
.../src/main/resources/metadata/segment.json | 86 --------
.../service_instance_relation_client_side.json | 56 -----
.../service_instance_relation_server_side.json | 56 -----
.../metadata/service_relation_client_side.json | 41 ----
.../metadata/service_relation_server_side.json | 41 ----
.../main/resources/metadata/service_traffic.json | 44 ----
.../src/main/resources/metadata/ui_template.json | 44 ----
73 files changed, 452 insertions(+), 1361 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
index 41237f1f3b..c512b5653c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
@@ -21,15 +21,17 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
+import org.apache.skywalking.banyandb.v1.client.MeasureWrite;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.banyandb.v1.client.metadata.Serializable;
-import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.type.Convert2Entity;
import org.apache.skywalking.oap.server.core.storage.type.Convert2Storage;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataComplexObject;
import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.util.ByteUtil;
import java.util.List;
import java.util.function.Function;
@@ -37,11 +39,17 @@ import java.util.function.Function;
public class BanyanDBConverter {
@RequiredArgsConstructor
public static class StreamToEntity implements Convert2Entity {
+ private final MetadataRegistry.Schema schema;
private final RowEntity rowEntity;
@Override
public Object get(String fieldName) {
- return rowEntity.getTagValue(fieldName);
+ MetadataRegistry.ColumnSpec spec = schema.getSpec(fieldName);
+ if (double.class.equals(spec.getModelColumn().getType())) {
+ return ByteUtil.bytes2Double(rowEntity.getTagValue(fieldName));
+ } else {
+ return rowEntity.getTagValue(fieldName);
+ }
}
@Override
@@ -57,10 +65,7 @@ public class BanyanDBConverter {
@Override
public void accept(String fieldName, Object fieldValue) {
- // skip "time_bucket"
- if (Record.TIME_BUCKET.equals(fieldName)) {
- return;
- }
+ // TODO: skip "time_bucket"
try {
this.streamWrite.tag(fieldName, buildTag(fieldValue));
} catch (BanyanDBException ex) {
@@ -68,15 +73,6 @@ public class BanyanDBConverter {
}
}
- private Serializable<BanyandbModel.TagValue> buildTag(Object value) {
- if (Integer.class.equals(value.getClass()) || Long.class.equals(value.getClass())) {
- return TagAndValue.longTagValue((long) value);
- } else if (String.class.equals(value.getClass())) {
- return TagAndValue.stringTagValue((String) value);
- }
- throw new IllegalStateException(value.getClass() + " is not supported");
- }
-
@Override
public void accept(String fieldName, byte[] fieldValue) {
try {
@@ -104,7 +100,6 @@ public class BanyanDBConverter {
@Override
public Object get(String fieldName) {
- // TODO: get an unmodifiable view of tag
return null;
}
@@ -113,4 +108,91 @@ public class BanyanDBConverter {
return this.streamWrite;
}
}
+
+ @Slf4j
+ @RequiredArgsConstructor
+ public static class MeasureToStorage implements Convert2Storage<MeasureWrite> {
+ private final MetadataRegistry.Schema schema;
+ private final MeasureWrite measureWrite;
+
+ @Override
+ public void accept(String fieldName, Object fieldValue) {
+ MetadataRegistry.ColumnSpec columnSpec = this.schema.getSpec(fieldName);
+ try {
+ if (columnSpec.getColumnType() == MetadataRegistry.ColumnType.TAG) {
+ this.measureWrite.tag(fieldName, buildTag(fieldValue));
+ } else {
+ this.measureWrite.field(fieldName, buildField(fieldValue));
+ }
+ } catch (BanyanDBException ex) {
+ log.error("fail to add tag", ex);
+ }
+ }
+
+ @Override
+ public void accept(String fieldName, byte[] fieldValue) {
+ MetadataRegistry.ColumnSpec columnSpec = this.schema.getSpec(fieldName);
+ try {
+ if (columnSpec.getColumnType() == MetadataRegistry.ColumnType.TAG) {
+ this.measureWrite.tag(fieldName, TagAndValue.binaryTagValue(fieldValue));
+ } else {
+ this.measureWrite.field(fieldName, TagAndValue.binaryFieldValue(fieldValue));
+ }
+ } catch (BanyanDBException ex) {
+ log.error("fail to add binary tag/field", ex);
+ }
+ }
+
+ @Override
+ public void accept(String fieldName, List<String> fieldValue) {
+ for (final String tagKeyAndValue : fieldValue) {
+ if (StringUtil.isEmpty(tagKeyAndValue)) {
+ continue;
+ }
+ int pos = tagKeyAndValue.indexOf("=");
+ if (pos == -1) {
+ continue;
+ }
+ String key = tagKeyAndValue.substring(0, pos);
+ String value = tagKeyAndValue.substring(pos + 1);
+ this.accept(key, value);
+ }
+ }
+
+ @Override
+ public Object get(String fieldName) {
+ return null;
+ }
+
+ @Override
+ public MeasureWrite obtain() {
+ return this.measureWrite;
+ }
+ }
+
+ private static Serializable<BanyandbModel.TagValue> buildTag(Object value) {
+ if (Integer.class.equals(value.getClass()) || Long.class.equals(value.getClass())) {
+ return TagAndValue.longTagValue((long) value);
+ } else if (String.class.equals(value.getClass())) {
+ return TagAndValue.stringTagValue((String) value);
+ } else if (Double.class.equals(value.getClass())) {
+ return TagAndValue.binaryTagValue(ByteUtil.double2Bytes((double) value));
+ } else if (value instanceof StorageDataComplexObject) {
+ return TagAndValue.stringTagValue(((StorageDataComplexObject<?>) value).toStorageData());
+ }
+ throw new IllegalStateException(value.getClass() + " is not supported");
+ }
+
+ private static Serializable<BanyandbModel.FieldValue> buildField(Object value) {
+ if (Integer.class.equals(value.getClass()) || Long.class.equals(value.getClass())) {
+ return TagAndValue.longFieldValue((long) value);
+ } else if (String.class.equals(value.getClass())) {
+ return TagAndValue.stringFieldValue((String) value);
+ } else if (Double.class.equals(value.getClass())) {
+ return TagAndValue.binaryFieldValue(ByteUtil.double2Bytes((double) value));
+ } else if (value instanceof StorageDataComplexObject) {
+ return TagAndValue.stringFieldValue(((StorageDataComplexObject<?>) value).toStorageData());
+ }
+ throw new IllegalStateException(value.getClass() + " is not supported");
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index 32f3480769..75109aa781 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -22,6 +22,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.banyandb.v1.client.metadata.Group;
+import org.apache.skywalking.banyandb.v1.client.metadata.Measure;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
@@ -72,8 +73,11 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
((BanyanDBStorageClient) client).define(stream);
}
} else if (model.isTimeSeries() && !model.isRecord()) { // measure
- // TODO: dynamically register Measure
- log.info("skip measure index {}", model.getName());
+ Measure measure = (Measure) MetadataRegistry.INSTANCE.registerModel(model, this.configService);
+ if (measure != null) {
+ log.info("install measure schema {}", model.getName());
+ ((BanyanDBStorageClient) client).define(measure);
+ }
} else if (!model.isTimeSeries()) { // UITemplate
log.info("skip property index {}", model.getName());
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
index 4e84979498..20a300c3b0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
@@ -1,9 +1,13 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.v1.client.MeasureWrite;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.Convert2Storage;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
@@ -13,6 +17,7 @@ import java.util.Collections;
import java.util.List;
@RequiredArgsConstructor
+@Slf4j
public class BanyanDBMetricsDAO implements IMetricsDAO {
private final StorageBuilder<Metrics> storageBuilder;
@@ -23,6 +28,16 @@ public class BanyanDBMetricsDAO implements IMetricsDAO {
@Override
public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
+ log.info("prepare to insert {}", model.getName());
+ MetadataRegistry.Schema schema = MetadataRegistry.INSTANCE.findMetadata(model.getName());
+ if (schema == null) {
+ throw new IOException(model.getName() + " is not registered");
+ }
+ MeasureWrite measureWrite = new MeasureWrite(schema.getMetadata().getGroup(), // group name
+ model.getName(), // index-name
+ TimeBucket.getTimestamp(metrics.getTimeBucket(), model.getDownsampling())); // timestamp
+ final Convert2Storage<MeasureWrite> toStorage = new BanyanDBConverter.MeasureToStorage(schema, measureWrite);
+ storageBuilder.entity2Storage(metrics, toStorage);
return new InsertRequest() {
};
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 2df53a2ed9..a92c270023 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -20,11 +20,14 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import io.grpc.Status;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
+import org.apache.skywalking.banyandb.v1.client.MeasureQuery;
+import org.apache.skywalking.banyandb.v1.client.MeasureQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
+import org.apache.skywalking.banyandb.v1.client.metadata.Measure;
import org.apache.skywalking.banyandb.v1.client.metadata.Property;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.library.client.Client;
@@ -90,9 +93,9 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
- public StreamQueryResponse query(StreamQuery streamQuery) throws IOException {
+ public StreamQueryResponse query(StreamQuery q) throws IOException {
try {
- StreamQueryResponse response = this.client.query(streamQuery);
+ StreamQueryResponse response = this.client.query(q);
this.healthChecker.health();
return response;
} catch (BanyanDBException ex) {
@@ -101,6 +104,17 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
+ public MeasureQueryResponse query(MeasureQuery q) throws IOException {
+ try {
+ MeasureQueryResponse response = this.client.query(q);
+ this.healthChecker.health();
+ return response;
+ } catch (BanyanDBException ex) {
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to query measure", ex);
+ }
+ }
+
public void define(Property property) throws IOException {
try {
this.client.save(property);
@@ -121,6 +135,16 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
+ public void define(Measure measure) throws IOException {
+ try {
+ this.client.define(measure);
+ this.healthChecker.health();
+ } catch (BanyanDBException ex) {
+ healthChecker.unHealth(ex);
+ throw new IOException("fail to define stream", ex);
+ }
+ }
+
public void write(StreamWrite streamWrite) {
this.client.write(streamWrite);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 26c0e8d501..26c6b2529a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -56,7 +56,6 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDB
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBServiceLabelDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBTopologyQueryDAO;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBAlarmQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBatchDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBrowserLogQueryDAO;
@@ -109,7 +108,7 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.registerServiceImplementation(INetworkAddressAliasDAO.class, new BanyanDBNetworkAddressAliasDAO());
this.registerServiceImplementation(ITraceQueryDAO.class, new BanyanDBTraceQueryDAO(client));
this.registerServiceImplementation(IBrowserLogQueryDAO.class, new BanyanDBBrowserLogQueryDAO(client));
- this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO());
+ this.registerServiceImplementation(IMetadataQueryDAO.class, new BanyanDBMetadataQueryDAO(client));
this.registerServiceImplementation(IAlarmQueryDAO.class, new BanyanDBAlarmQueryDAO(client));
this.registerServiceImplementation(ILogQueryDAO.class, new BanyanDBLogQueryDAO(client));
this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBUITemplateManagementDAO.java
similarity index 99%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBUITemplateManagementDAO.java
index dba7844413..b97e71ccb7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBUITemplateManagementDAO.java
@@ -16,7 +16,7 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
index b8925c7529..6c6fbb7658 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
@@ -18,9 +18,14 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+import com.google.gson.JsonObject;
import io.grpc.Status;
+import lombok.Builder;
import lombok.Data;
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
import lombok.RequiredArgsConstructor;
+import lombok.Singular;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
@@ -28,14 +33,19 @@ import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
import org.apache.skywalking.banyandb.v1.client.metadata.Group;
import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.Measure;
import org.apache.skywalking.banyandb.v1.client.metadata.NamedSchema;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.DownSampling;
+import org.apache.skywalking.oap.server.core.analysis.Layer;
import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.analysis.metrics.DataTable;
+import org.apache.skywalking.oap.server.core.analysis.metrics.IntList;
import org.apache.skywalking.oap.server.core.config.ConfigService;
+import org.apache.skywalking.oap.server.core.storage.annotation.ValueColumnMetadata;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.model.ModelColumn;
import org.apache.skywalking.oap.server.library.util.StringUtil;
@@ -45,9 +55,9 @@ import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
-import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
@@ -57,53 +67,105 @@ import java.util.stream.Collectors;
public enum MetadataRegistry {
INSTANCE;
- private final Map<String, PartialMetadata> registry = new ConcurrentHashMap<>();
+ private final Map<String, Schema> registry = new ConcurrentHashMap<>();
public NamedSchema<?> registerModel(Model model, ConfigService configService) {
- PartialMetadata partialMetadata = parseMetadata(model);
- final Stream.Builder builder = Stream.create(partialMetadata.getGroup(), partialMetadata.getName());
+ final PartialMetadata partialMetadata = parseMetadata(model);
+ Schema.SchemaBuilder schemaBuilder = Schema.builder().metadata(partialMetadata);
Map<String, ModelColumn> modelColumnMap = model.getColumns().stream()
.collect(Collectors.toMap(modelColumn -> modelColumn.getColumnName().getStorageName(), Function.identity()));
// parse and set sharding keys
- builder.setEntityRelativeTags(parseEntityNames(modelColumnMap));
- // parse and set tag families, which contains tag specs
- List<TagFamilySpec> specs = parseTagFamilySpecs(model, partialMetadata, configService);
- builder.addTagFamilies(specs);
- // parse and add index definition
- builder.addIndexes(parseIndexRules(specs, partialMetadata.indexFamily(), modelColumnMap));
-
- registry.put(model.getName(), partialMetadata);
- return builder.build();
+ List<String> entities = parseEntityNames(modelColumnMap);
+ // parse tag metadata
+ // this can be used to build both
+ // 1) a list of TagFamilySpec,
+ // 2) a list of IndexRule,
+ List<TagMetadata> tags = parseTagMetadata(model, configService, schemaBuilder);
+ List<TagFamilySpec> tagFamilySpecs = partialMetadata.extractTagFamilySpec(tags);
+ List<IndexRule> indexRules = tags.stream()
+ .map(TagMetadata::getIndexRule)
+ .filter(Objects::nonNull)
+ .collect(Collectors.toList());
+
+
+ if (partialMetadata.getKind() == Kind.STREAM) {
+ final Stream.Builder builder = Stream.create(partialMetadata.getGroup(), partialMetadata.getName());
+ builder.setEntityRelativeTags(entities);
+ builder.addTagFamilies(tagFamilySpecs);
+ builder.addIndexes(indexRules);
+
+ registry.put(model.getName(), schemaBuilder.build());
+ return builder.build();
+ } else {
+ final Measure.Builder builder = Measure.create(partialMetadata.getGroup(), partialMetadata.getName(),
+ downSamplingDuration(model.getDownsampling()));
+ builder.setEntityRelativeTags(entities);
+ builder.addTagFamilies(tagFamilySpecs);
+ builder.addIndexes(indexRules);
+ // parse and set field
+ Optional<ValueColumnMetadata.ValueColumn> valueColumnOpt = ValueColumnMetadata.INSTANCE
+ .readValueColumnDefinition(model.getName());
+ valueColumnOpt.ifPresent(valueColumn -> builder.addField(parseFieldSpec(modelColumnMap.get(valueColumn.getValueCName()), valueColumn)));
+
+ registry.put(model.getName(), schemaBuilder.build());
+ return builder.build();
+ }
}
- public PartialMetadata findSchema(final String name) {
+ public Schema findMetadata(final String name) {
return this.registry.get(name);
}
- List<IndexRule> parseIndexRules(List<TagFamilySpec> specs, String indexTagFamily, Map<String, ModelColumn> modelColumnMap) {
- List<IndexRule> indexRules = new ArrayList<>();
- for (final TagFamilySpec spec : specs) {
- if (!indexTagFamily.equals(spec.tagFamilyName())) {
- continue;
- }
- for (final TagFamilySpec.TagSpec tagSpec : spec.tagSpecs()) {
- final String tagName = tagSpec.getTagName();
- // TODO: we need to add support index type in the OAP core
- // Currently, we only register INVERTED type
- final ModelColumn modelColumn = modelColumnMap.get(tagName);
- // if it is null, it must be a user-defined tag
- if (modelColumn == null) {
- indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES));
- continue;
- }
- if (modelColumn.getBanyanDBExtension().isGlobalIndexing()) {
- indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.GLOBAL));
- } else {
- indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES));
- }
- }
+ private Measure.FieldSpec parseFieldSpec(ModelColumn modelColumn, ValueColumnMetadata.ValueColumn valueColumn) {
+ if (String.class.equals(modelColumn.getType())) {
+ return Measure.FieldSpec.newIntField(valueColumn.getValueCName())
+ .compressWithZSTD()
+ .build();
+ } else if (long.class.equals(modelColumn.getType()) || int.class.equals(modelColumn.getType())) {
+ return Measure.FieldSpec.newIntField(valueColumn.getValueCName())
+ .compressWithZSTD()
+ .encodeWithGorilla()
+ .build();
+ } else if (DataTable.class.equals(modelColumn.getType())) {
+ return Measure.FieldSpec.newBinaryField(valueColumn.getValueCName())
+ .compressWithZSTD()
+ .build();
+ } else if (double.class.equals(modelColumn.getType())) {
+ // TODO: natively support double/float in BanyanDB
+ log.warn("Double is stored as binary");
+ return Measure.FieldSpec.newBinaryField(valueColumn.getValueCName())
+ .compressWithZSTD()
+ .build();
+ } else {
+ throw new UnsupportedOperationException(modelColumn.getType().getSimpleName() + " is not supported for field");
+ }
+ }
+
+ Duration downSamplingDuration(DownSampling downSampling) {
+ switch (downSampling) {
+ case Hour:
+ return Duration.ofHours(1);
+ case Minute:
+ return Duration.ofMinutes(1);
+ case Day:
+ return Duration.ofDays(1);
+ default:
+ throw new UnsupportedOperationException("unsupported downSampling interval");
+ }
+ }
+
+ IndexRule parseIndexRule(String tagName, ModelColumn modelColumn) {
+ // TODO: we need to add support index type in the OAP core
+ // Currently, we only register INVERTED type
+ // if it is null, it must be a user-defined tag
+ if (modelColumn == null) {
+ return IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+ }
+ if (modelColumn.getBanyanDBExtension().isGlobalIndexing()) {
+ return IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.GLOBAL);
+ } else {
+ return IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
}
- return indexRules;
}
/**
@@ -125,30 +187,40 @@ public enum MetadataRegistry {
.collect(Collectors.toList());
}
- List<TagFamilySpec> parseTagFamilySpecs(Model model, PartialMetadata metadata, ConfigService configService) {
- Map<String, TagFamilySpec.Builder> builderMap = new HashMap<>();
+ List<TagMetadata> parseTagMetadata(Model model, ConfigService configService, Schema.SchemaBuilder builder) {
+ List<TagMetadata> tagMetadataList = new ArrayList<>();
+ // skip metric
+ Optional<ValueColumnMetadata.ValueColumn> valueColumnOpt = ValueColumnMetadata.INSTANCE
+ .readValueColumnDefinition(model.getName());
for (final ModelColumn col : model.getColumns()) {
+ if (valueColumnOpt.isPresent() && valueColumnOpt.get().getValueCName().equals(col.getColumnName().getStorageName())) {
+ builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.FIELD, col));
+ continue;
+ }
final TagFamilySpec.TagSpec tagSpec = parseTagSpec(col);
if (tagSpec == null) {
continue;
}
+ builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.TAG, col));
if (col.shouldIndex()) {
- builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpec(tagSpec);
+ // build indexRule
+ IndexRule indexRule = parseIndexRule(tagSpec.getTagName(), col);
+ tagMetadataList.add(new TagMetadata(indexRule, tagSpec));
} else {
- builderMap.computeIfAbsent(metadata.nonIndexFamily(), TagFamilySpec::create).addTagSpec(tagSpec);
+ tagMetadataList.add(new TagMetadata(null, tagSpec));
}
}
// add all user-defined indexed tags to the end of the "searchable" family
if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
- builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableTracesTags()));
+ tagMetadataList.addAll(parseExtraTagSpecs(configService.getSearchableTracesTags()));
} else if (LogRecord.INDEX_NAME.equals(model.getName())) {
- builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableLogsTags()));
+ tagMetadataList.addAll(parseExtraTagSpecs(configService.getSearchableLogsTags()));
} else if (AlarmRecord.INDEX_NAME.equals(model.getName())) {
- builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableAlarmTags()));
+ tagMetadataList.addAll(parseExtraTagSpecs(configService.getSearchableAlarmTags()));
}
- return builderMap.values().stream().map(TagFamilySpec.Builder::build).collect(Collectors.toList());
+ return tagMetadataList;
}
/**
@@ -158,7 +230,7 @@ public enum MetadataRegistry {
* @param tags a series of tags joint by comma
* @return a list of {@link org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec.TagSpec} generated from input
*/
- private List<TagFamilySpec.TagSpec> parseExtraTagSpecs(String tags) {
+ private List<TagMetadata> parseExtraTagSpecs(String tags) {
if (StringUtil.isEmpty(tags)) {
return Collections.emptyList();
}
@@ -167,7 +239,8 @@ public enum MetadataRegistry {
return Collections.emptyList();
}
return Arrays.stream(tagsArray)
- .map(TagFamilySpec.TagSpec::newStringTag)
+ .map(tagName -> new TagMetadata(parseIndexRule(tagName, null),
+ TagFamilySpec.TagSpec.newStringTag(tagName)))
.collect(Collectors.toList());
}
@@ -180,15 +253,23 @@ public enum MetadataRegistry {
@Nullable
private TagFamilySpec.TagSpec parseTagSpec(ModelColumn modelColumn) {
final Class<?> clazz = modelColumn.getType();
- if (String.class.equals(clazz)) {
- return TagFamilySpec.TagSpec.newStringTag(modelColumn.getColumnName().getStorageName());
+ final String colName = modelColumn.getColumnName().getStorageName();
+ if (String.class.equals(clazz) || DataTable.class.equals(clazz) || JsonObject.class.equals(clazz)) {
+ return TagFamilySpec.TagSpec.newStringTag(colName);
} else if (int.class.equals(clazz) || long.class.equals(clazz)) {
- return TagFamilySpec.TagSpec.newIntTag(modelColumn.getColumnName().getStorageName());
- } else if (byte[].class.equals(clazz) || DataTable.class.equals(clazz)) {
- return TagFamilySpec.TagSpec.newBinaryTag(modelColumn.getColumnName().getStorageName());
- } else {
+ return TagFamilySpec.TagSpec.newIntTag(colName);
+ } else if (byte[].class.equals(clazz)) {
+ return TagFamilySpec.TagSpec.newBinaryTag(colName);
+ } else if (Layer.class.equals(clazz)) {
+ return TagFamilySpec.TagSpec.newIntTag(colName);
+ } else if (double.class.equals(clazz) || Double.class.equals(clazz)) {
+ // serialize double as binary
+ return TagFamilySpec.TagSpec.newBinaryTag(colName);
+ } else if (IntList.class.isAssignableFrom(clazz)) {
+ return TagFamilySpec.TagSpec.newIntArrayTag(colName);
+ } else { // handle exceptions
// TODO: we skip all tags with type of List<String>
- if ("tags".equals(modelColumn.getColumnName().getStorageName())) {
+ if ("tags".equals(colName)) {
return null;
}
throw new IllegalStateException("type " + modelColumn.getType().toString() + " is not supported");
@@ -207,6 +288,7 @@ public enum MetadataRegistry {
return new PartialMetadata("measure-default", model.getName(), Kind.MEASURE);
}
+
@RequiredArgsConstructor
@Data
public static class PartialMetadata {
@@ -233,6 +315,14 @@ public enum MetadataRegistry {
}
}
+ private List<TagFamilySpec> extractTagFamilySpec(List<TagMetadata> tagMetadataList) {
+ return tagMetadataList.stream().collect(Collectors.groupingBy(tagMetadata -> tagMetadata.isIndex() ? PartialMetadata.this.indexFamily() : PartialMetadata.this.nonIndexFamily()))
+ .entrySet().stream()
+ .map(entry -> TagFamilySpec.create(entry.getKey())
+ .addTagSpecs(entry.getValue().stream().map(TagMetadata::getTagSpec).collect(Collectors.toList())).build())
+ .collect(Collectors.toList());
+ }
+
public Group getOrCreateGroup(BanyanDBClient client) throws BanyanDBException {
Group g = client.findGroup(this.group);
if (g != null) {
@@ -262,7 +352,6 @@ public enum MetadataRegistry {
public String nonIndexFamily() {
switch (kind) {
case MEASURE:
- return null;
case STREAM:
return "binary";
default:
@@ -274,4 +363,39 @@ public enum MetadataRegistry {
public enum Kind {
MEASURE, STREAM;
}
+
+ @RequiredArgsConstructor
+ @Getter
+ private static class TagMetadata {
+ private final IndexRule indexRule;
+ private final TagFamilySpec.TagSpec tagSpec;
+
+ boolean isIndex() {
+ return this.indexRule != null;
+ }
+ }
+
+ @Builder
+ @EqualsAndHashCode
+ public static class Schema {
+ @Getter
+ private final PartialMetadata metadata;
+ @Singular
+ private final Map<String, ColumnSpec> specs;
+
+ public ColumnSpec getSpec(String columnName) {
+ return this.specs.get(columnName);
+ }
+ }
+
+ @RequiredArgsConstructor
+ @Getter
+ public static class ColumnSpec {
+ private final ColumnType columnType;
+ private final ModelColumn modelColumn;
+ }
+
+ public enum ColumnType {
+ TAG, FIELD;
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
index bff4fc00be..23828464da 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
@@ -18,19 +18,44 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+import com.google.common.collect.ImmutableSet;
+import org.apache.skywalking.banyandb.v1.client.MeasureQuery;
+import org.apache.skywalking.banyandb.v1.client.MeasureQueryResponse;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
import org.apache.skywalking.oap.server.core.query.type.Endpoint;
import org.apache.skywalking.oap.server.core.query.type.Process;
import org.apache.skywalking.oap.server.core.query.type.Service;
import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.AbstractBanyanDBDAO;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-public class BanyanDBMetadataQueryDAO implements IMetadataQueryDAO {
+public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMetadataQueryDAO {
+ public BanyanDBMetadataQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<Service> listServices(String layer, String group) throws IOException {
+ MeasureQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
+ ImmutableSet.of(ServiceTraffic.NAME, ServiceTraffic.SHORT_NAME),
+ Collections.emptySet(), new QueryBuilder<MeasureQuery>() {
+ @Override
+ protected void apply(MeasureQuery query) {
+ if (StringUtil.isNotEmpty(group)) {
+ query.appendCondition(eq(ServiceTraffic.GROUP, group));
+ }
+ if (StringUtil.isNotEmpty(layer)) {
+ query.appendCondition(eq(ServiceTraffic.LAYER, layer));
+ }
+ }
+ });
+
return Collections.emptyList();
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index 53e8cc4f80..c7d4f930ee 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -18,6 +18,9 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.AbstractQuery;
+import org.apache.skywalking.banyandb.v1.client.MeasureQuery;
+import org.apache.skywalking.banyandb.v1.client.MeasureQueryResponse;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
@@ -39,17 +42,44 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
super(client);
}
- protected StreamQueryResponse query(MetadataRegistry.PartialMetadata metadata, Set<String> searchableTags, QueryBuilder builder) throws IOException {
- return this.query(metadata, searchableTags, null, builder);
+ protected StreamQueryResponse query(String modelName, Set<String> tags, QueryBuilder<StreamQuery> builder) throws IOException {
+ return this.query(modelName, tags, null, builder);
}
- protected StreamQueryResponse query(MetadataRegistry.PartialMetadata metadata, Set<String> searchableTags, TimestampRange timestampRange,
- QueryBuilder builder) throws IOException {
+ protected StreamQueryResponse query(String modelName, Set<String> tags, TimestampRange timestampRange,
+ QueryBuilder<StreamQuery> builder) throws IOException {
+ MetadataRegistry.Schema schema = MetadataRegistry.INSTANCE.findMetadata(modelName);
+ if (schema == null) {
+ throw new IllegalStateException("schema is not registered");
+ }
final StreamQuery query;
if (timestampRange == null) {
- query = new StreamQuery(metadata.getGroup(), metadata.getName(), LARGEST_TIME_RANGE, searchableTags);
+ query = new StreamQuery(schema.getMetadata().getGroup(), schema.getMetadata().getName(), LARGEST_TIME_RANGE, tags);
+ } else {
+ query = new StreamQuery(schema.getMetadata().getGroup(), schema.getMetadata().getName(), timestampRange, tags);
+ }
+
+ builder.apply(query);
+
+ return getClient().query(query);
+ }
+
+ protected MeasureQueryResponse query(String modelName, Set<String> tags, Set<String> fields,
+ QueryBuilder<MeasureQuery> builder) throws IOException {
+ return this.query(modelName, tags, fields, null, builder);
+ }
+
+ protected MeasureQueryResponse query(String modelName, Set<String> tags, Set<String> fields,
+ TimestampRange timestampRange, QueryBuilder<MeasureQuery> builder) throws IOException {
+ MetadataRegistry.Schema schema = MetadataRegistry.INSTANCE.findMetadata(modelName);
+ if (schema == null) {
+ throw new IllegalStateException("schema is not registered");
+ }
+ final MeasureQuery query;
+ if (timestampRange == null) {
+ query = new MeasureQuery(schema.getMetadata().getGroup(), schema.getMetadata().getName(), LARGEST_TIME_RANGE, tags, fields);
} else {
- query = new StreamQuery(metadata.getGroup(), metadata.getName(), timestampRange, searchableTags);
+ query = new MeasureQuery(schema.getMetadata().getGroup(), schema.getMetadata().getName(), timestampRange, tags, fields);
}
builder.apply(query);
@@ -57,8 +87,8 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
return getClient().query(query);
}
- protected abstract static class QueryBuilder {
- abstract void apply(final StreamQuery query);
+ protected abstract static class QueryBuilder<T extends AbstractQuery<? extends com.google.protobuf.GeneratedMessageV3>> {
+ protected abstract void apply(final T query);
protected PairQueryCondition<Long> eq(String name, long value) {
return PairQueryCondition.LongQueryCondition.eq(name, value);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index b2a43abff7..55f2cf9ec7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -44,9 +44,6 @@ import java.util.Objects;
* which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
*/
public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarmQueryDAO {
- private final MetadataRegistry.PartialMetadata alarmRecordMetadata =
- MetadataRegistry.INSTANCE.findSchema(AlarmRecord.INDEX_NAME);
-
public BanyanDBAlarmQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -58,10 +55,10 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
}
- StreamQueryResponse resp = query(alarmRecordMetadata,
+ StreamQueryResponse resp = query(AlarmRecord.INDEX_NAME,
ImmutableSet.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME, AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA),
tsRange,
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
if (Objects.nonNull(scopeId)) {
@@ -87,7 +84,7 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
for (final RowEntity rowEntity : resp.getElements()) {
AlarmRecord.Builder builder = new AlarmRecord.Builder();
AlarmRecord alarmRecord = builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(rowEntity)
+ new BanyanDBConverter.StreamToEntity(MetadataRegistry.INSTANCE.findMetadata(AlarmRecord.INDEX_NAME), rowEntity)
);
AlarmMessage message = new AlarmMessage();
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index 2c412cf7dc..e692a8d3b3 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -42,9 +42,6 @@ import java.util.Objects;
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
*/
public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements IBrowserLogQueryDAO {
- private final MetadataRegistry.PartialMetadata browserErrorLogRecordMetadata =
- MetadataRegistry.INSTANCE.findSchema(BrowserErrorLogRecord.INDEX_NAME);
-
public BanyanDBBrowserLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -56,10 +53,10 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
- StreamQueryResponse resp = query(browserErrorLogRecordMetadata, ImmutableSet.of(BrowserErrorLogRecord.SERVICE_ID,
+ StreamQueryResponse resp = query(BrowserErrorLogRecord.INDEX_NAME, ImmutableSet.of(BrowserErrorLogRecord.SERVICE_ID,
BrowserErrorLogRecord.SERVICE_VERSION_ID,
BrowserErrorLogRecord.PAGE_PATH_ID,
- BrowserErrorLogRecord.ERROR_CATEGORY, BrowserErrorLogRecord.DATA_BINARY), tsRange, new QueryBuilder() {
+ BrowserErrorLogRecord.ERROR_CATEGORY, BrowserErrorLogRecord.DATA_BINARY), tsRange, new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(BrowserErrorLogRecord.SERVICE_ID, serviceId));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index c6097539c9..d4e3214a36 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -50,9 +50,6 @@ import java.util.Objects;
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
*/
public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQueryDAO {
- private final MetadataRegistry.PartialMetadata logRecordMetadata =
- MetadataRegistry.INSTANCE.findSchema(LogRecord.INDEX_NAME);
-
public BanyanDBLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -62,7 +59,7 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit,
long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent,
List<String> excludingKeywordsOfContent) throws IOException {
- final QueryBuilder query = new QueryBuilder() {
+ final QueryBuilder<StreamQuery> query = new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
if (StringUtil.isNotEmpty(serviceId)) {
@@ -101,7 +98,7 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
}
- StreamQueryResponse resp = query(logRecordMetadata,
+ StreamQueryResponse resp = query(LogRecord.INDEX_NAME,
ImmutableSet.of(AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
AbstractLogRecord.SPAN_ID, AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT,
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 6749118a25..9d22ea36ba 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -37,9 +37,6 @@ import java.util.List;
* {@link ProfileTaskLogRecord} is a stream
*/
public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskLogQueryDAO {
- private final MetadataRegistry.PartialMetadata profileTaskLogRecord =
- MetadataRegistry.INSTANCE.findSchema(ProfileTaskLogRecord.INDEX_NAME);
-
private final int queryMaxSize;
public BanyanDBProfileTaskLogQueryDAO(BanyanDBStorageClient client, int queryMaxSize) {
@@ -49,10 +46,10 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
- StreamQueryResponse resp = query(profileTaskLogRecord,
+ StreamQueryResponse resp = query(ProfileTaskLogRecord.INDEX_NAME,
ImmutableSet.of(ProfileTaskLogRecord.OPERATION_TIME, ProfileTaskLogRecord.INSTANCE_ID,
ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.OPERATION_TYPE),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index a0cacd0c39..d60484b6f0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -48,22 +48,16 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
protected final ProfileThreadSnapshotRecord.Builder builder =
new ProfileThreadSnapshotRecord.Builder();
- private final MetadataRegistry.PartialMetadata profileThreadSnapshotMetadata =
- MetadataRegistry.INSTANCE.findSchema(ProfileThreadSnapshotRecord.INDEX_NAME);
-
- private final MetadataRegistry.PartialMetadata segmentRecordMetadata =
- MetadataRegistry.INSTANCE.findSchema(SegmentRecord.INDEX_NAME);
-
public BanyanDBProfileThreadSnapshotQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
- StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(ProfileThreadSnapshotRecord.TASK_ID, taskId))
@@ -83,9 +77,9 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
// TODO: support `IN` or `OR` logic operation in BanyanDB
List<BasicTrace> basicTraces = new ArrayList<>();
for (String segmentID : segmentIds) {
- final StreamQueryResponse segmentRecordResp = query(segmentRecordMetadata,
+ final StreamQueryResponse segmentRecordResp = query(SegmentRecord.INDEX_NAME,
ImmutableSet.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery traceQuery) {
traceQuery.appendCondition(eq(SegmentRecord.SEGMENT_ID, segmentID));
@@ -132,11 +126,11 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
- StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
ProfileThreadSnapshotRecord.STACK_BINARY),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
@@ -148,7 +142,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
List<ProfileThreadSnapshotRecord> result = new ArrayList<>(maxSequence - minSequence);
for (final RowEntity rowEntity : resp.getElements()) {
ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(rowEntity));
+ new BanyanDBConverter.StreamToEntity(MetadataRegistry.INSTANCE.findMetadata(ProfileThreadSnapshotRecord.INDEX_NAME), rowEntity));
result.add(record);
}
@@ -157,11 +151,11 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
- StreamQueryResponse resp = query(segmentRecordMetadata,
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
ImmutableSet.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.SERVICE_ID,
SegmentRecord.SERVICE_INSTANCE_ID, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY,
SegmentRecord.START_TIME, SegmentRecord.DATA_BINARY),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(SegmentRecord.INDEX_NAME, segmentId));
@@ -191,11 +185,11 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
}
private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) throws IOException {
- StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
ProfileThreadSnapshotRecord.STACK_BINARY),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
@@ -207,7 +201,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
List<ProfileThreadSnapshotRecord> records = new ArrayList<>();
for (final RowEntity rowEntity : resp.getElements()) {
ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(rowEntity));
+ new BanyanDBConverter.StreamToEntity(MetadataRegistry.INSTANCE.findMetadata(ProfileThreadSnapshotRecord.INDEX_NAME), rowEntity));
records.add(record);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index 3fe0e03ef7..48443b9e51 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -38,11 +38,11 @@ public class BanyanDBRecordDAO implements IRecordDAO {
@Override
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- MetadataRegistry.PartialMetadata metadata = MetadataRegistry.INSTANCE.findSchema(model.getName());
- if (metadata == null) {
+ MetadataRegistry.Schema schema = MetadataRegistry.INSTANCE.findMetadata(model.getName());
+ if (schema == null) {
throw new IOException(model.getName() + " is not registered");
}
- StreamWrite streamWrite = new StreamWrite(metadata.getGroup(), // group name
+ StreamWrite streamWrite = new StreamWrite(schema.getMetadata().getGroup(), // group name
model.getName(), // index-name
record.id(), // identity
TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling())); // timestamp
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index e149b5ab27..b7e7d215ae 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -46,16 +46,13 @@ import java.util.Collections;
import java.util.List;
public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
- private final MetadataRegistry.PartialMetadata segmentRecordMetadata =
- MetadataRegistry.INSTANCE.findSchema(SegmentRecord.INDEX_NAME);
-
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public TraceBrief queryBasicTraces(long startSecondTB, long endSecondTB, long minDuration, long maxDuration, String serviceId, String serviceInstanceId, String endpointId, String traceId, int limit, int from, TraceState traceState, QueryOrder queryOrder, List<Tag> tags) throws IOException {
- final QueryBuilder q = new QueryBuilder() {
+ final QueryBuilder<StreamQuery> q = new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
if (minDuration != 0) {
@@ -115,7 +112,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
- StreamQueryResponse resp = query(segmentRecordMetadata,
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
ImmutableSet.of(SegmentRecord.TRACE_ID, // 0 - trace_id
SegmentRecord.IS_ERROR, // 1 - is_error
SegmentRecord.SERVICE_ID, // 2 - service_id
@@ -150,7 +147,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
- StreamQueryResponse resp = query(segmentRecordMetadata,
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
ImmutableSet.of(SegmentRecord.TRACE_ID,
SegmentRecord.IS_ERROR,
SegmentRecord.SERVICE_ID,
@@ -159,7 +156,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
SegmentRecord.LATENCY,
SegmentRecord.START_TIME,
SegmentRecord.DATA_BINARY),
- new QueryBuilder() {
+ new QueryBuilder<StreamQuery>() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(eq(SegmentRecord.TRACE_ID, traceId));
@@ -170,7 +167,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
for (final RowEntity rowEntity : resp.getElements()) {
SegmentRecord segmentRecord = new SegmentRecord.Builder().storage2Entity(
- new BanyanDBConverter.StreamToEntity(rowEntity));
+ new BanyanDBConverter.StreamToEntity(MetadataRegistry.INSTANCE.findMetadata(SegmentRecord.INDEX_NAME), rowEntity));
segmentRecords.add(segmentRecord);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/util/ByteUtil.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/util/ByteUtil.java
new file mode 100644
index 0000000000..5ae1102e38
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/util/ByteUtil.java
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.util;
+
+import java.nio.ByteBuffer;
+
+public class ByteUtil {
+ private static final ThreadLocal<ByteBuffer> BYTE_BUFFER = ThreadLocal.withInitial(() -> ByteBuffer.allocate(8));
+
+ public static Double bytes2Double(byte[] bytes) {
+ return BYTE_BUFFER.get().put(bytes).getDouble();
+ }
+
+ public static byte[] double2Bytes(double number) {
+ return BYTE_BUFFER.get().putDouble(number).array();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
deleted file mode 100644
index 2a203c1679..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
+++ /dev/null
@@ -1,41 +0,0 @@
-{
- "metadata": {
- "name": "endpoint_relation_server_side",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "component_id",
- "type": "TAG_TYPE_INT"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "source_endpoint",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_endpoint",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "entity_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "source_endpoint",
- "dest_endpoint"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
deleted file mode 100644
index 459b77d808..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
+++ /dev/null
@@ -1,31 +0,0 @@
-{
- "metadata": {
- "name": "endpoint_traffic",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": []
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "name",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
deleted file mode 100644
index 37a02eca24..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
+++ /dev/null
@@ -1,64 +0,0 @@
-{
- "metadata": {
- "name": "events",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "message",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "parameters",
- "type": "TAG_TYPE_STRING"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "uuid",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service_instance",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "endpoint",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "name",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "type",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "end_time",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "uuid"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json
deleted file mode 100644
index 9b52f22ba2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "component_id",
- "group": "default"
- },
- "tags": [
- "component_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json
deleted file mode 100644
index addab64e17..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "create_time",
- "group": "default"
- },
- "tags": [
- "create_time"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json
deleted file mode 100644
index 5d5c159c39..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "db.instance",
- "group": "default"
- },
- "tags": [
- "db.instance"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json
deleted file mode 100644
index 67914ee464..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "db.type",
- "group": "default"
- },
- "tags": [
- "db.type"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json
deleted file mode 100644
index affb1fdf17..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "dest_endpoint",
- "group": "default"
- },
- "tags": [
- "dest_endpoint"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json
deleted file mode 100644
index 3bb400392e..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "dest_service_id",
- "group": "default"
- },
- "tags": [
- "dest_service_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
deleted file mode 100644
index 19a1c1b204..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "disabled",
- "group": "default"
- },
- "tags": [
- "disabled"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json
deleted file mode 100644
index 25ae87f411..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "dump_period",
- "group": "default"
- },
- "tags": [
- "dump_period"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json
deleted file mode 100644
index 6073ee7740..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "dump_time",
- "group": "default"
- },
- "tags": [
- "dump_time"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json
deleted file mode 100644
index 6f18db1359..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "duration",
- "group": "default"
- },
- "tags": [
- "duration"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json
deleted file mode 100644
index 951a1c88b9..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "end_time",
- "group": "default"
- },
- "tags": [
- "end_time"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json
deleted file mode 100644
index f9fefdddf0..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "endpoint",
- "group": "default"
- },
- "tags": [
- "endpoint"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json
deleted file mode 100644
index 7da3b0d2f5..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "endpoint_id",
- "group": "default"
- },
- "tags": [
- "endpoint_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json
deleted file mode 100644
index 0678ae4691..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "endpoint_name",
- "group": "default"
- },
- "tags": [
- "endpoint_name"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json
deleted file mode 100644
index 31ece658a6..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "entity_id",
- "group": "default"
- },
- "tags": [
- "entity_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json
deleted file mode 100644
index 3001154bfc..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "http.method",
- "group": "default"
- },
- "tags": [
- "http.method"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json
deleted file mode 100644
index c90b39ac62..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "id",
- "group": "default"
- },
- "tags": [
- "id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json
deleted file mode 100644
index 7d063b2dd4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "last_ping",
- "group": "default"
- },
- "tags": [
- "last_ping"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json
deleted file mode 100644
index bbde5cf37b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "last_update_time_bucket",
- "group": "default"
- },
- "tags": [
- "last_update_time_bucket"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json
deleted file mode 100644
index 4eb445055f..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "latency",
- "group": "default"
- },
- "tags": [
- "latency"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json
deleted file mode 100644
index 05455e1d6b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "layer",
- "group": "default"
- },
- "tags": [
- "layer"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json
deleted file mode 100644
index 1fb14a84c2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "max_sampling_count",
- "group": "default"
- },
- "tags": [
- "max_sampling_count"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json
deleted file mode 100644
index f359e2d5ba..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "min_duration_threshold",
- "group": "default"
- },
- "tags": [
- "min_duration_threshold"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json
deleted file mode 100644
index 0cfbd278dd..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "mq.broker",
- "group": "default"
- },
- "tags": [
- "mq.broker"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json
deleted file mode 100644
index 0c589fc6e4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "mq.queue",
- "group": "default"
- },
- "tags": [
- "mq.queue"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json
deleted file mode 100644
index 5dc51fcf66..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "mq.topic",
- "group": "default"
- },
- "tags": [
- "mq.topic"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json
deleted file mode 100644
index db25d029f2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "name",
- "group": "default"
- },
- "tags": [
- "name"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json
deleted file mode 100644
index 30222bc484..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "operation_time",
- "group": "default"
- },
- "tags": [
- "operation_time"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json
deleted file mode 100644
index a58129eab2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "segment_id",
- "group": "default"
- },
- "tags": [
- "segment_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json
deleted file mode 100644
index 2bbd960809..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "sequence",
- "group": "default"
- },
- "tags": [
- "sequence"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json
deleted file mode 100644
index 4764f16eca..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "service",
- "group": "default"
- },
- "tags": [
- "service"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json
deleted file mode 100644
index 3cecbefa13..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "service_group",
- "group": "default"
- },
- "tags": [
- "service_group"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json
deleted file mode 100644
index 76c3f67d22..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "service_id",
- "group": "default"
- },
- "tags": [
- "service_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json
deleted file mode 100644
index 73d84e7151..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "service_instance",
- "group": "default"
- },
- "tags": [
- "service_instance"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json
deleted file mode 100644
index 164962f91c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "source_endpoint",
- "group": "default"
- },
- "tags": [
- "source_endpoint"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json
deleted file mode 100644
index f54b191164..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "source_service_id",
- "group": "default"
- },
- "tags": [
- "component_id"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json
deleted file mode 100644
index 5a73ad12f4..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "start_time",
- "group": "default"
- },
- "tags": [
- "start_time"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json
deleted file mode 100644
index ad034773f5..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "status_code",
- "group": "default"
- },
- "tags": [
- "status_code"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json
deleted file mode 100644
index 2f84bfb489..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "trace_id",
- "group": "default"
- },
- "tags": [
- "trace_id"
- ],
- "type": "TYPE_TREE",
- "location": "LOCATION_GLOBAL",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json
deleted file mode 100644
index db25d029f2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "name",
- "group": "default"
- },
- "tags": [
- "name"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json
deleted file mode 100644
index 4cd401c3dc..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
- "metadata": {
- "name": "uuid",
- "group": "default"
- },
- "tags": [
- "uuid"
- ],
- "type": "TYPE_INVERTED",
- "location": "LOCATION_SERIES",
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
deleted file mode 100644
index fc728af744..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
+++ /dev/null
@@ -1,40 +0,0 @@
-{
- "metadata": {
- "name": "instance_traffic",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "data_binary",
- "type": "TAG_TYPE_DATA_BINARY"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "last_ping",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
deleted file mode 100644
index 0dc6f344c0..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
+++ /dev/null
@@ -1,45 +0,0 @@
-{
- "metadata": {
- "name": "network_address_alias",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "time_bucket",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "address",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "represent_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "represent_service_instance_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "last_update_time_bucket",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "represent_service_id",
- "represent_service_instance_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
deleted file mode 100644
index 85c5901c8c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ /dev/null
@@ -1,59 +0,0 @@
-{
- "metadata": {
- "name": "profile_task",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": []
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "endpoint_name",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "duration",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "min_duration_threshold",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "dump_period",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "create_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "max_sampling_count",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json
deleted file mode 100644
index 6fa6ccb244..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json
+++ /dev/null
@@ -1,40 +0,0 @@
-{
- "metadata": {
- "name": "profile_task_log",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "task_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "operation_type",
- "type": "TAG_TYPE_INT"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "operation_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "instance_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "instance_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json
deleted file mode 100644
index a5fcbcbf5c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json
+++ /dev/null
@@ -1,44 +0,0 @@
-{
- "metadata": {
- "name": "profile_task_segment_snapshot",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "stack_binary",
- "type": "TAG_TYPE_DATA_BINARY"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "task_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "segment_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dump_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "sequence",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "task_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
deleted file mode 100644
index 7a8343d45c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
+++ /dev/null
@@ -1,86 +0,0 @@
-{
- "metadata": {
- "name": "segment",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "data_binary",
- "type": "TAG_TYPE_DATA_BINARY"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "trace_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "is_error",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service_instance_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "endpoint_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "latency",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "http.method",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "status_code",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "db.type",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "db.instance",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "mq.queue",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "mq.topic",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "mq.broker",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "service_id",
- "service_instance_id",
- "is_error"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
deleted file mode 100644
index edd49ef287..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
+++ /dev/null
@@ -1,56 +0,0 @@
-{
- "metadata": {
- "name": "service_instance_relation_client_side",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "component_id",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "source_service_instance_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_instance_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "source_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "entity_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "source_service_id",
- "dest_service_id"
- ]
- },
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
deleted file mode 100644
index a8c6089421..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
+++ /dev/null
@@ -1,56 +0,0 @@
-{
- "metadata": {
- "name": "service_instance_relation_server_side",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "component_id",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "source_service_instance_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_instance_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "source_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "entity_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "source_service_id",
- "dest_service_id"
- ]
- },
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
deleted file mode 100644
index 1f74420c2c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
+++ /dev/null
@@ -1,41 +0,0 @@
-{
- "metadata": {
- "name": "service_relation_client_side",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "component_id",
- "type": "TAG_TYPE_INT"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "source_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "entity_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "source_service_id",
- "dest_service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
deleted file mode 100644
index 13c2cc6177..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
+++ /dev/null
@@ -1,41 +0,0 @@
-{
- "metadata": {
- "name": "service_relation_server_side",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "component_id",
- "type": "TAG_TYPE_INT"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "source_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "dest_service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "entity_id",
- "type": "TAG_TYPE_STRING"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "source_service_id",
- "dest_service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
deleted file mode 100644
index b009a199cf..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
+++ /dev/null
@@ -1,44 +0,0 @@
-{
- "metadata": {
- "name": "service_traffic",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "short_name",
- "type": "TAG_TYPE_STRING"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "name",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "service_group",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "layer",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "service_id"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
deleted file mode 100644
index b01e8baae0..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
+++ /dev/null
@@ -1,44 +0,0 @@
-{
- "metadata": {
- "name": "ui_template",
- "group": "default-stream"
- },
- "tag_families": [
- {
- "name": "data",
- "tags": [
- {
- "name": "type",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "configuration",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "activated",
- "type": "TAG_TYPE_INT"
- }
- ]
- },
- {
- "name": "searchable",
- "tags": [
- {
- "name": "name",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "disabled",
- "type": "TAG_TYPE_INT"
- }
- ]
- }
- ],
- "entity": {
- "tag_names": [
- "name"
- ]
- },
- "updated_at": "2021-04-15T01:30:15.01Z"
-}
\ No newline at end of file
[skywalking] 06/24: complete basic logic
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 799608dd59a61c28827560c3086a8ef8cbaa13cd
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Sun Dec 5 22:44:22 2021 +0800
complete basic logic
---
.../storage/plugin/banyandb/BanyanDBSchema.java | 66 ------------
.../plugin/banyandb/BanyanDBStorageProvider.java | 6 +-
.../deserializer/AbstractBanyanDBDeserializer.java | 20 +++-
...geMapper.java => AlarmMessageDeserializer.java} | 22 +++-
.../deserializer/BanyanDBDeserializerFactory.java | 48 ++++++---
...raceMapper.java => BasicTraceDeserializer.java} | 22 +++-
...apper.java => BrowserErrorLogDeserializer.java} | 27 ++++-
...ava => DashboardConfigurationDeserializer.java} | 22 +++-
.../deserializer/DatabaseDeserializer.java | 43 ++++++++
.../banyandb/deserializer/DatabaseMapper.java | 25 -----
.../deserializer/EndpointDeserializer.java | 43 ++++++++
.../banyandb/deserializer/EndpointMapper.java | 25 -----
.../{EventMapper.java => EventDeserializer.java} | 22 +++-
.../{LogMapper.java => LogDeserializer.java} | 26 ++++-
...r.java => NetworkAddressAliasDeserializer.java} | 22 +++-
...askMapper.java => ProfileTaskDeserializer.java} | 22 +++-
...Mapper.java => ProfileTaskLogDeserializer.java} | 22 +++-
...> ProfileThreadSnapshotRecordDeserializer.java} | 22 +++-
.../RowEntityDeserializer.java} | 17 ++-
.../banyandb/deserializer/RowEntityMapper.java | 8 --
...dMapper.java => SegmentRecordDeserializer.java} | 22 +++-
...ServiceMapper.java => ServiceDeserializer.java} | 22 +++-
.../deserializer/ServiceInstanceDeserializer.java | 79 ++++++++++++++
.../deserializer/ServiceInstanceMapper.java | 70 ------------
.../plugin/banyandb/schema/AlarmRecordBuilder.java | 55 ++++++++++
.../BanyanDBMetricsBuilder.java} | 21 ++--
.../banyandb/schema/BanyanDBRecordBuilder.java | 61 +++++++++++
.../schema/BanyanDBStorageDataBuilder.java | 52 +++++++++
.../schema/BrowserErrorLogRecordBuilder.java | 46 ++++++++
.../plugin/banyandb/schema/EventBuilder.java | 41 ++++++++
.../plugin/banyandb/schema/LogRecordBuilder.java | 58 ++++++++++
.../storage/plugin/banyandb/schema/Metadata.java | 70 ++++++++++++
.../schema/NetworkAddressAliasBuilder.java | 44 ++++++++
.../schema/ProfileTaskLogRecordBuilder.java | 44 ++++++++
.../banyandb/schema/ProfileTaskRecordBuilder.java | 44 ++++++++
.../schema/ProfileThreadSnapshotRecordBuilder.java | 45 ++++++++
.../banyandb/schema/SegmentRecordBuilder.java | 60 +++++++++++
.../plugin/banyandb/schema/UITemplateBuilder.java | 55 ++++++++++
.../banyandb/stream/AbstractBanyanDBDAO.java | 18 ++++
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 29 ++++-
.../stream/BanyanDBBrowserLogQueryDAO.java | 36 +++++--
.../banyandb/stream/BanyanDBEventQueryDAO.java | 19 +++-
.../banyandb/stream/BanyanDBLogQueryDAO.java | 49 ++++++---
...sertRequest.java => BanyanDBManagementDAO.java} | 22 ++--
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 18 ++++
.../plugin/banyandb/stream/BanyanDBMetricsDAO.java | 53 ++++++++++
.../stream/BanyanDBNetworkAddressAliasDAO.java | 18 ++++
...sertRequest.java => BanyanDBNoneStreamDAO.java} | 22 ++--
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 22 +++-
.../stream/BanyanDBProfileTaskQueryDAO.java | 25 ++++-
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 26 +++--
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 48 +--------
.../stream/BanyanDBSegmentRecordBuilder.java | 58 ----------
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 117 +++++++++++++++------
.../stream/BanyanDBStreamInsertRequest.java | 1 -
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 32 +++---
.../stream/BanyanDBUITemplateManagementDAO.java | 23 +++-
57 files changed, 1588 insertions(+), 467 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java
deleted file mode 100644
index 3ebf87776b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBSchema.java
+++ /dev/null
@@ -1,66 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableSet;
-import lombok.Getter;
-
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.Set;
-
-public class BanyanDBSchema {
- public static final String NAME = "sw";
- public static final String GROUP = "default";
- public static final List<String> FIELD_NAMES;
-
- public static final Set<String> INDEX_FIELDS = ImmutableSet.of("http.method", "status_code", "db.type",
- "db.instance", "mq.queue", "mq.topic", "mq.broker");
-
- static {
- Set<String> fields = new LinkedHashSet<>();
- fields.add("trace_id");
- fields.add("state");
- fields.add("service_id");
- fields.add("service_instance_id");
- fields.add("endpoint_id");
- fields.add("duration");
- fields.add("start_time");
- fields.add("http.method");
- fields.add("status_code");
- fields.add("db.type");
- fields.add("db.instance");
- fields.add("mq.queue");
- fields.add("mq.topic");
- fields.add("mq.broker");
- FIELD_NAMES = ImmutableList.copyOf(fields);
- }
-
- public enum TraceState {
- ALL(0), SUCCESS(1), ERROR(2);
-
- @Getter
- private final int state;
-
- TraceState(int state) {
- this.state = state;
- }
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index c4aa1016ca..09b72509a8 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -22,7 +22,11 @@ import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
import org.apache.skywalking.oap.server.core.storage.StorageModule;
-import org.apache.skywalking.oap.server.library.module.*;
+import org.apache.skywalking.oap.server.library.module.ModuleConfig;
+import org.apache.skywalking.oap.server.library.module.ModuleDefine;
+import org.apache.skywalking.oap.server.library.module.ModuleProvider;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
import org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics;
import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
index 44adcb8869..0b2f0644ad 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
@@ -6,7 +24,7 @@ import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import java.util.Collections;
import java.util.List;
-public abstract class AbstractBanyanDBDeserializer<T> implements RowEntityMapper<T> {
+public abstract class AbstractBanyanDBDeserializer<T> implements RowEntityDeserializer<T> {
private final String indexName;
private final List<String> searchableProjection;
private final List<String> dataProjection;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
similarity index 71%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
index 939496e061..3165d60afa 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.base.Charsets;
@@ -15,10 +33,10 @@ import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import java.util.List;
-public class AlarmMessageMapper extends AbstractBanyanDBDeserializer<AlarmMessage> {
+public class AlarmMessageDeserializer extends AbstractBanyanDBDeserializer<AlarmMessage> {
private final Gson GSON = new Gson();
- public AlarmMessageMapper() {
+ public AlarmMessageDeserializer() {
super(AlarmRecord.INDEX_NAME,
ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
ImmutableList.of(AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
index dd4af41736..859812530e 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
@@ -26,21 +44,21 @@ public enum BanyanDBDeserializerFactory {
BanyanDBDeserializerFactory() {
registry = new HashMap<>(10);
- register(AlarmMessage.class, new AlarmMessageMapper());
- register(BasicTrace.class, new BasicTraceMapper());
- register(BrowserErrorLog.class, new BrowserErrorLogMapper());
- register(DashboardConfiguration.class, new DashboardConfigurationMapper());
- register(Database.class, new DatabaseMapper());
- register(Endpoint.class, new EndpointMapper());
- register(Event.class, new EventMapper());
- register(Log.class, new LogMapper());
- register(NetworkAddressAlias.class, new NetworkAddressAliasMapper());
- register(ProfileTaskLog.class, new ProfileTaskLogMapper());
- register(ProfileTask.class, new ProfileTaskMapper());
- register(ProfileThreadSnapshotRecord.class, new ProfileThreadSnapshotRecordMapper());
- register(SegmentRecord.class, new SegmentRecordMapper());
- register(ServiceInstance.class, new ServiceInstanceMapper());
- register(Service.class, new ServiceMapper());
+ register(AlarmMessage.class, new AlarmMessageDeserializer());
+ register(BasicTrace.class, new BasicTraceDeserializer());
+ register(BrowserErrorLog.class, new BrowserErrorLogDeserializer());
+ register(DashboardConfiguration.class, new DashboardConfigurationDeserializer());
+ register(Database.class, new DatabaseDeserializer());
+ register(Endpoint.class, new EndpointDeserializer());
+ register(Event.class, new EventDeserializer());
+ register(Log.class, new LogDeserializer());
+ register(NetworkAddressAlias.class, new NetworkAddressAliasDeserializer());
+ register(ProfileTaskLog.class, new ProfileTaskLogDeserializer());
+ register(ProfileTask.class, new ProfileTaskDeserializer());
+ register(ProfileThreadSnapshotRecord.class, new ProfileThreadSnapshotRecordDeserializer());
+ register(SegmentRecord.class, new SegmentRecordDeserializer());
+ register(ServiceInstance.class, new ServiceInstanceDeserializer());
+ register(Service.class, new ServiceDeserializer());
}
private <T> void register(Class<T> clazz, AbstractBanyanDBDeserializer<T> mapper) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java
similarity index 58%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java
index 5d7ae36590..949ae43e16 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -9,8 +27,8 @@ import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
import java.util.List;
-public class BasicTraceMapper extends AbstractBanyanDBDeserializer<BasicTrace> {
- public BasicTraceMapper() {
+public class BasicTraceDeserializer extends AbstractBanyanDBDeserializer<BasicTrace> {
+ public BasicTraceDeserializer() {
super(SegmentRecord.INDEX_NAME, ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"));
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java
similarity index 70%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java
index 0189a77049..92605afa67 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -12,14 +30,13 @@ import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
import java.util.Collections;
import java.util.List;
-public class BrowserErrorLogMapper extends AbstractBanyanDBDeserializer<BrowserErrorLog> {
- public BrowserErrorLogMapper() {
+public class BrowserErrorLogDeserializer extends AbstractBanyanDBDeserializer<BrowserErrorLog> {
+ public BrowserErrorLogDeserializer() {
super(BrowserErrorLogRecord.INDEX_NAME,
ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
BrowserErrorLogRecord.SERVICE_VERSION_ID,
BrowserErrorLogRecord.PAGE_PATH_ID,
- BrowserErrorLogRecord.ERROR_CATEGORY,
- BrowserErrorLogRecord.TIMESTAMP),
+ BrowserErrorLogRecord.ERROR_CATEGORY),
Collections.singletonList(BrowserErrorLogRecord.DATA_BINARY));
}
@@ -32,7 +49,7 @@ public class BrowserErrorLogMapper extends AbstractBanyanDBDeserializer<BrowserE
log.setServiceVersion((String) searchable.get(1).getValue());
log.setPagePath((String) searchable.get(2).getValue());
log.setCategory(ErrorCategory.valueOf((String) searchable.get(3).getValue()));
- log.setTime(((Number) searchable.get(4).getValue()).longValue());
+ log.setTime(row.getTimestamp());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
Object o = data.get(0).getValue();
if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java
similarity index 64%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java
index cb6b4f3c2f..c46db9df8d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -10,8 +28,8 @@ import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import java.util.List;
-public class DashboardConfigurationMapper extends AbstractBanyanDBDeserializer<DashboardConfiguration> {
- public DashboardConfigurationMapper() {
+public class DashboardConfigurationDeserializer extends AbstractBanyanDBDeserializer<DashboardConfiguration> {
+ public DashboardConfigurationDeserializer() {
super(UITemplate.INDEX_NAME,
ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED),
ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java
new file mode 100644
index 0000000000..db6d875358
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.query.type.Database;
+
+import java.util.List;
+
+public class DatabaseDeserializer extends AbstractBanyanDBDeserializer<Database> {
+ public DatabaseDeserializer() {
+ super(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE));
+ }
+
+ @Override
+ public Database map(RowEntity row) {
+ Database database = new Database();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ database.setId(row.getId());
+ database.setName((String) searchable.get(0).getValue());
+ return database;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java
deleted file mode 100644
index 1c5a15f774..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java
+++ /dev/null
@@ -1,25 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.query.type.Database;
-
-import java.util.List;
-
-public class DatabaseMapper extends AbstractBanyanDBDeserializer<Database> {
- public DatabaseMapper() {
- super(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE));
- }
-
- @Override
- public Database map(RowEntity row) {
- Database database = new Database();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- database.setId(row.getId());
- database.setName((String) searchable.get(0).getValue());
- return database;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java
new file mode 100644
index 0000000000..e86867c771
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.query.type.Endpoint;
+
+import java.util.List;
+
+public class EndpointDeserializer extends AbstractBanyanDBDeserializer<Endpoint> {
+ public EndpointDeserializer() {
+ super(EndpointTraffic.INDEX_NAME,
+ ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID));
+ }
+
+ @Override
+ public Endpoint map(RowEntity row) {
+ Endpoint endpoint = new Endpoint();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ endpoint.setName((String) searchable.get(0).getValue());
+ endpoint.setId((String) searchable.get(1).getValue());
+ return endpoint;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java
deleted file mode 100644
index 8f81df039c..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java
+++ /dev/null
@@ -1,25 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.query.type.Endpoint;
-
-import java.util.List;
-
-public class EndpointMapper extends AbstractBanyanDBDeserializer<Endpoint> {
- public EndpointMapper() {
- super(EndpointTraffic.INDEX_NAME,
- ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID));
- }
-
- @Override
- public Endpoint map(RowEntity row) {
- Endpoint endpoint = new Endpoint();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- endpoint.setName((String) searchable.get(0).getValue());
- endpoint.setId((String) searchable.get(1).getValue());
- return endpoint;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java
similarity index 66%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java
index 63c9dc802c..39783228d2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -9,8 +27,8 @@ import org.apache.skywalking.oap.server.core.source.Event;
import java.util.List;
-public class EventMapper extends AbstractBanyanDBDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
- public EventMapper() {
+public class EventDeserializer extends AbstractBanyanDBDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
+ public EventDeserializer() {
super(Event.INDEX_NAME,
ImmutableList.of(Event.UUID, Event.SERVICE, Event.SERVICE_INSTANCE, Event.ENDPOINT, Event.NAME,
Event.TYPE, Event.START_TIME, Event.END_TIME),
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java
similarity index 70%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java
index 3f65a82cea..f1bb3bec5d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -14,12 +32,12 @@ import org.apache.skywalking.oap.server.core.query.type.Log;
import java.util.List;
-public class LogMapper extends AbstractBanyanDBDeserializer<Log> {
- public LogMapper() {
+public class LogDeserializer extends AbstractBanyanDBDeserializer<Log> {
+ public LogDeserializer() {
super(LogRecord.INDEX_NAME, ImmutableList.of(
AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
- AbstractLogRecord.SPAN_ID, AbstractLogRecord.TIMESTAMP),
+ AbstractLogRecord.SPAN_ID),
ImmutableList.of(AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT, AbstractLogRecord.TAGS_RAW_DATA));
}
@@ -31,7 +49,7 @@ public class LogMapper extends AbstractBanyanDBDeserializer<Log> {
log.setServiceInstanceId((String) searchable.get(1).getValue());
log.setEndpointId((String) searchable.get(2).getValue());
log.setTraceId((String) searchable.get(3).getValue());
- log.setTimestamp(((Number) searchable.get(6).getValue()).longValue());
+ log.setTimestamp(row.getTimestamp());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
if (data.get(2).getValue() == null || ((ByteString) data.get(2).getValue()).isEmpty()) {
log.setContent("");
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java
similarity index 62%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java
index 7c231df86a..6255a0ea40 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -8,8 +26,8 @@ import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import java.util.List;
-public class NetworkAddressAliasMapper extends AbstractBanyanDBDeserializer<NetworkAddressAlias> {
- public NetworkAddressAliasMapper() {
+public class NetworkAddressAliasDeserializer extends AbstractBanyanDBDeserializer<NetworkAddressAlias> {
+ public NetworkAddressAliasDeserializer() {
super(NetworkAddressAlias.INDEX_NAME,
ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET),
ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id"));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java
similarity index 66%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java
index 2626a91c7d..d3eddedec6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -8,10 +26,10 @@ import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
import java.util.List;
-public class ProfileTaskMapper extends AbstractBanyanDBDeserializer<ProfileTask> {
+public class ProfileTaskDeserializer extends AbstractBanyanDBDeserializer<ProfileTask> {
public static final String ID = "profile_task_query_id";
- public ProfileTaskMapper() {
+ public ProfileTaskDeserializer() {
super(ProfileTaskRecord.INDEX_NAME,
ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java
similarity index 62%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java
index de51d2f95c..79731bf5fb 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -9,8 +27,8 @@ import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationT
import java.util.List;
-public class ProfileTaskLogMapper extends AbstractBanyanDBDeserializer<ProfileTaskLog> {
- public ProfileTaskLogMapper() {
+public class ProfileTaskLogDeserializer extends AbstractBanyanDBDeserializer<ProfileTaskLog> {
+ public ProfileTaskLogDeserializer() {
super(ProfileTaskLogRecord.INDEX_NAME,
ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME),
ImmutableList.of(ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.INSTANCE_ID,
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java
similarity index 60%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java
index 2d966a5955..be815f57c0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -9,8 +27,8 @@ import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord
import java.util.Collections;
import java.util.List;
-public class ProfileThreadSnapshotRecordMapper extends AbstractBanyanDBDeserializer<ProfileThreadSnapshotRecord> {
- public ProfileThreadSnapshotRecordMapper() {
+public class ProfileThreadSnapshotRecordDeserializer extends AbstractBanyanDBDeserializer<ProfileThreadSnapshotRecord> {
+ public ProfileThreadSnapshotRecordDeserializer() {
super(ProfileThreadSnapshotRecord.INDEX_NAME,
ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java
similarity index 70%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java
index aef8ec94e7..2e33ed2373 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java
@@ -16,16 +16,11 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
-@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
-
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+@FunctionalInterface
+public interface RowEntityDeserializer<T> {
+ T map(RowEntity row);
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
deleted file mode 100644
index 51f9a5d687..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
+++ /dev/null
@@ -1,8 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-
-@FunctionalInterface
-public interface RowEntityMapper<T> {
- T map(RowEntity row);
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java
similarity index 62%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java
index da7be9de35..0665d614af 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -9,8 +27,8 @@ import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentReco
import java.util.Collections;
import java.util.List;
-public class SegmentRecordMapper extends AbstractBanyanDBDeserializer<SegmentRecord> {
- public SegmentRecordMapper() {
+public class SegmentRecordDeserializer extends AbstractBanyanDBDeserializer<SegmentRecord> {
+ public SegmentRecordDeserializer() {
super(SegmentRecord.INDEX_NAME,
ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
Collections.singletonList("data_binary"));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java
similarity index 50%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java
index e006a1107d..cf36fd9239 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
@@ -8,8 +26,8 @@ import org.apache.skywalking.oap.server.core.query.type.Service;
import java.util.List;
-public class ServiceMapper extends AbstractBanyanDBDeserializer<Service> {
- public ServiceMapper() {
+public class ServiceDeserializer extends AbstractBanyanDBDeserializer<Service> {
+ public ServiceDeserializer() {
super(ServiceTraffic.INDEX_NAME,
ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP));
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java
new file mode 100644
index 0000000000..a95867c42e
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.gson.JsonElement;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.query.enumeration.Language;
+import org.apache.skywalking.oap.server.core.query.type.Attribute;
+import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+public class ServiceInstanceDeserializer extends AbstractBanyanDBDeserializer<ServiceInstance> {
+ public ServiceInstanceDeserializer() {
+ super(InstanceTraffic.INDEX_NAME,
+ ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
+ Collections.singletonList("data_binary"));
+ }
+
+ @Override
+ public ServiceInstance map(RowEntity row) {
+ InstanceTraffic instanceTraffic = new InstanceTraffic();
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ Object o = data.get(0).getValue();
+ ServiceInstance serviceInstance = new ServiceInstance();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ try {
+ RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
+ instanceTraffic.deserialize(remoteData);
+ serviceInstance.setName(instanceTraffic.getName());
+ serviceInstance.setId(instanceTraffic.getServiceId());
+
+ if (instanceTraffic.getProperties() != null) {
+ for (Map.Entry<String, JsonElement> property : instanceTraffic.getProperties().entrySet()) {
+ String key = property.getKey();
+ String value = property.getValue().getAsString();
+ if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
+ serviceInstance.setLanguage(Language.value(value));
+ } else {
+ serviceInstance.getAttributes().add(new Attribute(key, value));
+ }
+ }
+ } else {
+ serviceInstance.setLanguage(Language.UNKNOWN);
+ }
+ } catch (InvalidProtocolBufferException ex) {
+ throw new RuntimeException("fail to parse remote data", ex);
+ }
+ } else {
+ throw new RuntimeException("unable to parse binary data");
+ }
+
+ return serviceInstance;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java
deleted file mode 100644
index f6925d3fd3..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java
+++ /dev/null
@@ -1,70 +0,0 @@
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.gson.Gson;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.protobuf.ByteString;
-import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.query.enumeration.Language;
-import org.apache.skywalking.oap.server.core.query.type.Attribute;
-import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
-import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
-import org.apache.skywalking.oap.server.library.util.StringUtil;
-
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
-
-public class ServiceInstanceMapper extends AbstractBanyanDBDeserializer<ServiceInstance> {
- private static final Gson GSON = new Gson();
-
- public ServiceInstanceMapper() {
- super(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
- Collections.singletonList("data_binary"));
- }
-
- @Override
- public ServiceInstance map(RowEntity row) {
- ServiceInstance serviceInstance = new ServiceInstance();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- serviceInstance.setId((String) searchable.get(0).getValue());
- serviceInstance.setInstanceUUID((String) searchable.get(0).getValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- Object o = data.get(0).getValue();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- try {
- RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
- serviceInstance.setName(remoteData.getDataStrings(1));
- final String propString = remoteData.getDataStrings(2);
- if (StringUtil.isNotEmpty(propString)) {
- JsonObject properties = GSON.fromJson(propString, JsonObject.class);
- if (properties != null) {
- for (Map.Entry<String, JsonElement> property : properties.entrySet()) {
- String key = property.getKey();
- String value = property.getValue().getAsString();
- if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
- serviceInstance.setLanguage(Language.value(value));
- } else {
- serviceInstance.getAttributes().add(new Attribute(key, value));
- }
- }
- } else {
- serviceInstance.setLanguage(Language.UNKNOWN);
- }
- } else {
- serviceInstance.setLanguage(Language.UNKNOWN);
- }
- } catch (InvalidProtocolBufferException ex) {
- throw new RuntimeException("fail to parse remote data", ex);
- }
- } else {
- serviceInstance.setLanguage(Language.UNKNOWN);
- }
- return serviceInstance;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
new file mode 100644
index 0000000000..b154c9280d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class AlarmRecordBuilder extends BanyanDBRecordBuilder<AlarmRecord> {
+ public static final List<String> INDEXED_TAGS = ImmutableList.of(
+ "level"
+ );
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(AlarmRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
+ searchable.add(TagAndValue.longField(entity.getScope()));
+ searchable.add(TagAndValue.longField(entity.getStartTime()));
+ searchable.addAll(filterSearchableTags(entity.getTags(), INDEXED_TAGS));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(AlarmRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>(6);
+ data.add(TagAndValue.stringField(entity.getName()));
+ data.add(TagAndValue.stringField(entity.getId0()));
+ data.add(TagAndValue.stringField(entity.getId1()));
+ data.add(TagAndValue.stringField(entity.getAlarmMessage()));
+ data.add(TagAndValue.stringField(entity.getRuleName()));
+ data.add(TagAndValue.binaryField(entity.getTagsRawData()));
+ return data;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java
similarity index 64%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java
index aef8ec94e7..28c3055f47 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java
@@ -16,16 +16,15 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
-@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
-
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+public abstract class BanyanDBMetricsBuilder<T extends Metrics> extends BanyanDBStorageDataBuilder<T> {
+ @Override
+ protected long timestamp(Model model, T entity) {
+ return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java
new file mode 100644
index 0000000000..0310c7173b
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public abstract class BanyanDBRecordBuilder<T extends Record> extends BanyanDBStorageDataBuilder<T> {
+ @Override
+ protected long timestamp(Model model, T entity) {
+ return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
+ }
+
+ protected List<SerializableTag<Banyandb.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
+ if (rawTags == null) {
+ return Collections.emptyList();
+ }
+ Map<String, SerializableTag<Banyandb.TagValue>> map = new HashMap<>();
+ for (final Tag tag : rawTags) {
+ map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
+ }
+ final List<SerializableTag<Banyandb.TagValue>> tags = new ArrayList<>();
+ for (String indexedTag : indexTags) {
+ SerializableTag<Banyandb.TagValue> tag = map.get(indexedTag);
+ if (tag == null) {
+ tags.add(TagAndValue.nullField());
+ } else {
+ tags.add(tag);
+ }
+ }
+
+ return tags;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
new file mode 100644
index 0000000000..6d545f8a9e
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.core.storage.StorageData;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.util.Collections;
+import java.util.List;
+
+public abstract class BanyanDBStorageDataBuilder<T extends StorageData> {
+ public StreamWrite entity2Storage(Model model, T entity) {
+ return StreamWrite.builder()
+ .elementId(this.elementID(entity))
+ .name(model.getName())
+ .timestamp(this.timestamp(model, entity))
+ .searchableTags(this.searchableTags(entity))
+ .dataTags(this.dataTags(entity))
+ .build();
+ }
+
+ protected String elementID(T entity) {
+ return entity.id();
+ }
+
+ abstract protected long timestamp(Model model, T entity);
+
+ abstract protected List<SerializableTag<Banyandb.TagValue>> searchableTags(T entity);
+
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(T entity) {
+ return Collections.emptyList();
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
new file mode 100644
index 0000000000..0d1923c6f0
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class BrowserErrorLogRecordBuilder extends BanyanDBRecordBuilder<BrowserErrorLogRecord> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(BrowserErrorLogRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getUniqueId()));
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getServiceVersionId()));
+ searchable.add(TagAndValue.stringField(entity.getPagePathId()));
+ searchable.add(TagAndValue.longField(entity.getErrorCategory()));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(BrowserErrorLogRecord entity) {
+ return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
new file mode 100644
index 0000000000..cce48e8b71
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
@@ -0,0 +1,41 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.source.Event;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class EventBuilder extends BanyanDBMetricsBuilder<Event> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(Event entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(8);
+ searchable.add(TagAndValue.stringField(entity.getUuid()));
+ searchable.add(TagAndValue.stringField(entity.getService()));
+ searchable.add(TagAndValue.stringField(entity.getServiceInstance()));
+ searchable.add(TagAndValue.stringField(entity.getEndpoint()));
+ searchable.add(TagAndValue.stringField(entity.getName()));
+ searchable.add(TagAndValue.stringField(entity.getType()));
+ searchable.add(TagAndValue.longField(entity.getStartTime()));
+ searchable.add(TagAndValue.longField(entity.getEndTime()));
+ return searchable;
+ }
+
+ @Override
+ protected long timestamp(Model model, Event entity) {
+ return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(Event entity) {
+ return ImmutableList.of(
+ TagAndValue.stringField(entity.getMessage()),
+ TagAndValue.stringField(entity.getParameters())
+ );
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
new file mode 100644
index 0000000000..1d88799fd9
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class LogRecordBuilder extends BanyanDBRecordBuilder<LogRecord> {
+ public static final List<String> INDEXED_TAGS = ImmutableList.of(
+ "level"
+ );
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(LogRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>();
+ searchable.add(TagAndValue.stringField(entity.getUniqueId()));
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getEndpointId()));
+ searchable.add(TagAndValue.stringField(entity.getTraceId()));
+ searchable.add(TagAndValue.stringField(entity.getTraceSegmentId()));
+ searchable.add(TagAndValue.longField(entity.getSpanId()));
+ searchable.addAll(filterSearchableTags(entity.getTags(), INDEXED_TAGS));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(LogRecord entity) {
+ return ImmutableList.of(
+ TagAndValue.stringField(entity.getContent()),
+ TagAndValue.longField(entity.getContentType()),
+ TagAndValue.binaryField(entity.getTagsRawData())
+ );
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
new file mode 100644
index 0000000000..1fcd5f1587
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class Metadata {
+ public static class ServiceTrafficBuilder extends BanyanDBMetricsBuilder<ServiceTraffic> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ServiceTraffic entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(3);
+ searchable.add(TagAndValue.stringField(entity.getName()));
+ searchable.add(TagAndValue.longField(entity.getNodeType().value()));
+ searchable.add(TagAndValue.stringField(entity.getGroup()));
+ return searchable;
+ }
+ }
+
+ public static class EndpointTrafficBuilder extends BanyanDBMetricsBuilder<EndpointTraffic> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(EndpointTraffic entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getName()));
+ return searchable;
+ }
+ }
+
+ public static class InstanceTrafficBuilder extends BanyanDBMetricsBuilder<InstanceTraffic> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(InstanceTraffic entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.longField(entity.getLastPingTimestamp()));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(InstanceTraffic entity) {
+ return Collections.singletonList(TagAndValue.binaryField(
+ entity.serialize().build().toByteArray()
+ ));
+ }
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
new file mode 100644
index 0000000000..1054bd922f
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class NetworkAddressAliasBuilder extends BanyanDBMetricsBuilder<NetworkAddressAlias> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(NetworkAddressAlias entity) {
+ List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ data.add(TagAndValue.stringField(entity.getAddress()));
+ data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
+ data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
+ return data;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
new file mode 100644
index 0000000000..ee0141dc6d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileTaskLogRecordBuilder extends BanyanDBRecordBuilder<ProfileTaskLogRecord> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
+ return Collections.singletonList(TagAndValue.longField(entity.getOperationTime()));
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(ProfileTaskLogRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ data.add(TagAndValue.stringField(entity.getTaskId()));
+ data.add(TagAndValue.stringField(entity.getInstanceId()));
+ data.add(TagAndValue.longField(entity.getOperationType()));
+ return data;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
new file mode 100644
index 0000000000..0ce548ffc6
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class ProfileTaskRecordBuilder extends BanyanDBRecordBuilder<ProfileTaskRecord> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(9);
+ searchable.add(TagAndValue.stringField(entity.id()));
+ searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ searchable.add(TagAndValue.stringField(entity.getEndpointName()));
+ searchable.add(TagAndValue.longField(entity.getStartTime()));
+ searchable.add(TagAndValue.longField(entity.getDuration()));
+ searchable.add(TagAndValue.longField(entity.getMinDurationThreshold()));
+ searchable.add(TagAndValue.longField(entity.getDuration()));
+ searchable.add(TagAndValue.longField(entity.getCreateTime()));
+ searchable.add(TagAndValue.longField(entity.getMaxSamplingCount()));
+ return searchable;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
new file mode 100644
index 0000000000..29f6a8b708
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class ProfileThreadSnapshotRecordBuilder extends BanyanDBRecordBuilder<ProfileThreadSnapshotRecord> {
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileThreadSnapshotRecord entity) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(4);
+ searchable.add(TagAndValue.stringField(entity.getTaskId()));
+ searchable.add(TagAndValue.stringField(entity.getSegmentId()));
+ searchable.add(TagAndValue.longField(entity.getDumpTime()));
+ searchable.add(TagAndValue.longField(entity.getSequence()));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(ProfileThreadSnapshotRecord entity) {
+ return Collections.singletonList(TagAndValue.binaryField(entity.getStackBinary()));
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
new file mode 100644
index 0000000000..e93335643c
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class SegmentRecordBuilder extends BanyanDBRecordBuilder<SegmentRecord> {
+ public static final List<String> INDEXED_TAGS = ImmutableList.of(
+ "http.method",
+ "status_code",
+ "db.type",
+ "db.instance",
+ "mq.queue",
+ "mq.topic",
+ "mq.broker"
+ );
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(SegmentRecord segmentRecord) {
+ List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(10);
+ searchable.add(TagAndValue.stringField(segmentRecord.getTraceId()));
+ searchable.add(TagAndValue.stringField(segmentRecord.getServiceId()));
+ searchable.add(TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
+ searchable.add(TagAndValue.stringField(segmentRecord.getEndpointId()));
+ searchable.add(TagAndValue.longField(segmentRecord.getStartTime()));
+ searchable.add(TagAndValue.longField(segmentRecord.getLatency()));
+ searchable.add(TagAndValue.longField(segmentRecord.getIsError()));
+ searchable.addAll(filterSearchableTags(segmentRecord.getTagsRawData(), INDEXED_TAGS));
+ return searchable;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(SegmentRecord entity) {
+ return Collections.singletonList(TagAndValue.binaryField(entity.getDataBinary()));
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
new file mode 100644
index 0000000000..1776e5af8d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.Banyandb;
+import org.apache.skywalking.banyandb.v1.client.SerializableTag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class UITemplateBuilder extends BanyanDBStorageDataBuilder<UITemplate> {
+ public static final long UI_TEMPLATE_TIMESTAMP = 1L;
+
+ @Override
+ protected long timestamp(Model model, UITemplate entity) {
+ return UI_TEMPLATE_TIMESTAMP;
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> searchableTags(UITemplate entity) {
+ return ImmutableList.of(
+ TagAndValue.stringField(entity.getName()),
+ TagAndValue.longField(entity.getDisabled())
+ );
+ }
+
+ @Override
+ protected List<SerializableTag<Banyandb.TagValue>> dataTags(UITemplate entity) {
+ List<SerializableTag<Banyandb.TagValue>> data = new ArrayList<>();
+ data.add(TagAndValue.stringField(entity.getType()));
+ data.add(TagAndValue.stringField(entity.getConfiguration()));
+ data.add(TagAndValue.longField(entity.getActivated()));
+ return data;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index b11cca6ed4..d0daa3375f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index 57f658a380..999d89db79 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -8,7 +26,9 @@ import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
import org.apache.skywalking.oap.server.core.query.type.Alarms;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
import java.io.IOException;
import java.util.List;
@@ -38,8 +58,13 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
// TODO: support keyword search
- // TODO: support tag search
-
+ if (CollectionUtils.isNotEmpty(tags)) {
+ for (final Tag tag : tags) {
+ if (AlarmRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ }
+ }
+ }
query.setLimit(limit);
query.setOffset(from);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index 3bd3af8ad5..803b05e255 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -25,16 +43,12 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
@Override
public BrowserErrorLogs queryBrowserErrorLogs(String serviceId, String serviceVersionId, String pagePathId, BrowserErrorCategory category, long startSecondTB, long endSecondTB, int limit, int from) throws IOException {
- final BrowserErrorLogs logs = new BrowserErrorLogs();
- List<BrowserErrorLog> browserErrorLogs = query(BrowserErrorLog.class, new QueryBuilder() {
+
+ final QueryBuilder qb = new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_ID, serviceId));
- if (startSecondTB != 0 && endSecondTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startSecondTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endSecondTB)));
- }
if (StringUtil.isNotEmpty(serviceVersionId)) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
}
@@ -48,7 +62,15 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
query.setOffset(from);
query.setLimit(limit);
}
- });
+ };
+
+ final BrowserErrorLogs logs = new BrowserErrorLogs();
+ final List<BrowserErrorLog> browserErrorLogs;
+ if (startSecondTB != 0 && endSecondTB != 0) {
+ browserErrorLogs = query(BrowserErrorLog.class, qb, TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
+ } else {
+ browserErrorLogs = query(BrowserErrorLog.class, qb);
+ }
logs.getLogs().addAll(browserErrorLogs);
logs.setTotal(logs.getLogs().size());
return logs;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
index 9b2f0d2be5..b85e07d860 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
@@ -45,7 +63,6 @@ public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEvent
Events events = new Events();
events.setEvents(eventList);
- // TODO: how to set total???
events.setTotal(eventList.size());
return events;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index 404d60017c..8325618f88 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -10,8 +28,10 @@ import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
import org.apache.skywalking.oap.server.core.query.type.Log;
import org.apache.skywalking.oap.server.core.query.type.Logs;
import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
import java.io.IOException;
import java.util.List;
@@ -30,18 +50,13 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit,
long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent,
List<String> excludingKeywordsOfContent) throws IOException {
- List<Log> entities = query(Log.class, new QueryBuilder() {
+ final QueryBuilder query = new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
if (StringUtil.isNotEmpty(serviceId)) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_ID, serviceId));
}
- if (startTB != 0 && endTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endTB)));
- }
-
if (StringUtil.isNotEmpty(serviceInstanceId)) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
}
@@ -60,14 +75,22 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
}
}
- // TODO: if we allow to index tags?
-// if (CollectionUtils.isNotEmpty(tags)) {
-// for (final Tag tag : tags) {
-// query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
-// }
-// }
+ if (CollectionUtils.isNotEmpty(tags)) {
+ for (final Tag tag : tags) {
+ if (LogRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ }
+ }
+ }
}
- });
+ };
+
+ final List<Log> entities;
+ if (startTB != 0 && endTB != 0) {
+ entities = query(Log.class, query, TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
+ } else {
+ entities = query(Log.class, query);
+ }
Logs logs = new Logs();
logs.getLogs().addAll(entities);
logs.setTotal(entities.size());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
similarity index 53%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
index aef8ec94e7..aecdd13dfd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
@@ -18,14 +18,24 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import lombok.Getter;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
+import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
+
+import java.io.IOException;
@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
+public class BanyanDBManagementDAO<T extends ManagementData> implements IManagementDAO {
+ private final BanyanDBStorageClient client;
+ private final BanyanDBStorageDataBuilder<T> storageBuilder;
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+ @Override
+ public void insert(Model model, ManagementData storageData) throws IOException {
+ StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) storageData);
+ client.write(streamWrite);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index dd3d9623a6..bee6508cd2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
new file mode 100644
index 0000000000..618ff86af9
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBMetricsBuilder;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+@RequiredArgsConstructor
+public class BanyanDBMetricsDAO<T extends Metrics> implements IMetricsDAO {
+ private final BanyanDBMetricsBuilder<T> storageBuilder;
+
+ @Override
+ public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
+ StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) metrics);
+ return new BanyanDBStreamInsertRequest(streamWrite);
+ }
+
+ @Override
+ public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
+ return null;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
index 6112593a55..ce36573617 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
similarity index 54%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
index aef8ec94e7..30ed279432 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
@@ -18,14 +18,24 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import lombok.Getter;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
+import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBRecordBuilder;
+
+import java.io.IOException;
@RequiredArgsConstructor
-public class BanyanDBStreamInsertRequest implements InsertRequest {
+public class BanyanDBNoneStreamDAO<T extends NoneStream> implements INoneStreamDAO {
+ private final BanyanDBStorageClient client;
+ private final BanyanDBRecordBuilder<T> storageBuilder;
- @Getter
- private final StreamWrite streamWrite;
-}
\ No newline at end of file
+ @Override
+ public void insert(Model model, NoneStream noneStream) throws IOException {
+ StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) noneStream);
+ this.client.write(streamWrite);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 0095965109..3a3c12e119 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -1,11 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskLogMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.Comparator;
@@ -16,8 +32,6 @@ import java.util.stream.Collectors;
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
*/
public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskLogQueryDAO {
- private static final RowEntityMapper<ProfileTaskLog> MAPPER = new ProfileTaskLogMapper();
-
private final int queryMaxSize;
public BanyanDBProfileTaskLogQueryDAO(BanyanDBStorageClient client, int queryMaxSize) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 23fb8c7d1b..47c7c57a00 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -8,8 +26,7 @@ import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskDeserializer;
import java.io.IOException;
import java.util.List;
@@ -19,8 +36,6 @@ import java.util.Objects;
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
*/
public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskQueryDAO {
- private static final RowEntityMapper<ProfileTask> MAPPER = new ProfileTaskMapper();
-
public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -68,7 +83,7 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements
return query(ProfileTask.class, new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskMapper.ID, id));
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskDeserializer.ID, id));
query.setLimit(1);
}
}).stream().findAny().orElse(null);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index e92a0b24ff..c6b9cdec9f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -7,10 +25,6 @@ import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileThreadSnapshotRecordMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
import java.io.IOException;
import java.util.Collections;
@@ -24,10 +38,6 @@ import java.util.stream.Collectors;
* {@link ProfileThreadSnapshotRecord} is a stream
*/
public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO implements IProfileThreadSnapshotQueryDAO {
- private static final RowEntityMapper<ProfileThreadSnapshotRecord> MAPPER = new ProfileThreadSnapshotRecordMapper();
- private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
- private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
-
public BanyanDBProfileThreadSnapshotQueryDAO(BanyanDBStorageClient client) {
super(client);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index 54870d98c6..0bd0e30a82 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -19,61 +19,23 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.RequiredArgsConstructor;
-import org.apache.skywalking.banyandb.v1.Banyandb;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.banyandb.v1.client.Tag;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
-import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBRecordBuilder;
import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
@RequiredArgsConstructor
-public class BanyanDBRecordDAO implements IRecordDAO {
- private final StorageHashMapBuilder<Record> storageBuilder;
+public class BanyanDBRecordDAO<T extends Record> implements IRecordDAO {
+ private final BanyanDBRecordBuilder<T> storageBuilder;
@Override
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
- SegmentRecord segmentRecord = (SegmentRecord) record;
- StreamWrite streamWrite = StreamWrite.builder()
- .name(SegmentRecord.INDEX_NAME)
- .dataTag(Tag.binaryField(segmentRecord.getDataBinary()))
- .timestamp(segmentRecord.getStartTime())
- .elementId(segmentRecord.getSegmentId())
- .searchableTags(buildFieldObjects(this.storageBuilder.entity2Storage(segmentRecord)))
- .build();
- return new BanyanDBStreamInsertRequest(streamWrite);
- }
- // TODO: support other stream types
- return new InsertRequest() {
- };
- }
+ StreamWrite streamWrite = storageBuilder.entity2Storage(model, (T) record);
- /**
- * Convert storageEntity in Map to an ordered list of {@link SerializableTag}
- *
- * @param segmentRecordMap which comes from {@link SegmentRecord}
- * @return an ordered list of {@link SerializableTag} which is accepted by BanyanDB Client
- */
- static List<SerializableTag<Banyandb.TagValue>> buildFieldObjects(Map<String, Object> segmentRecordMap) {
- List<SerializableTag<Banyandb.TagValue>> tagList = new ArrayList<>(BanyanDBSchema.FIELD_NAMES.size());
- for (String fieldName : BanyanDBSchema.FIELD_NAMES) {
- Object val = segmentRecordMap.get(fieldName);
- if (val == null) {
- tagList.add(Tag.nullField());
- } else {
- tagList.add((SerializableTag<Banyandb.TagValue>) val);
- }
- }
- return tagList;
+ return new BanyanDBStreamInsertRequest(streamWrite);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java
deleted file mode 100644
index b8c4744380..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBSegmentRecordBuilder.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.analysis.record.Record;
-import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
-
-import java.util.HashMap;
-import java.util.Map;
-
-public class BanyanDBSegmentRecordBuilder implements StorageHashMapBuilder<Record> {
- @Override
- public SegmentRecord storage2Entity(Map<String, Object> dbMap) {
- return new SegmentRecord();
- }
-
- /**
- * Map SegmentRecord to Skywalking-BanyanDB compatible Map with indexed tags and
- * without binaryData, entityId
- */
- @Override
- public Map<String, Object> entity2Storage(Record record) {
- final SegmentRecord segmentRecord = (SegmentRecord) record;
- Map<String, Object> map = new HashMap<>();
- map.put(SegmentRecord.TRACE_ID, TagAndValue.stringField(segmentRecord.getTraceId()));
- map.put(SegmentRecord.SERVICE_ID, TagAndValue.stringField(segmentRecord.getServiceId()));
- map.put(SegmentRecord.SERVICE_INSTANCE_ID, TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
- map.put(SegmentRecord.ENDPOINT_ID, TagAndValue.stringField(segmentRecord.getEndpointId()));
- map.put(SegmentRecord.START_TIME, TagAndValue.longField(segmentRecord.getStartTime()));
- map.put("duration", TagAndValue.longField(segmentRecord.getLatency()));
- map.put("state", TagAndValue.longField(segmentRecord.getIsError()));
- if (segmentRecord.getTagsRawData() != null) {
- for (final Tag tag : segmentRecord.getTagsRawData()) {
- map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
- }
- }
- return map;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
index be81608a37..76c0359153 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -19,55 +19,88 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.source.Event;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.StorageDAO;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
-import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BrowserErrorLogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EventBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.Metadata;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.NetworkAddressAliasBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskLogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileThreadSnapshotRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
-import java.io.IOException;
-import java.util.Collections;
-import java.util.List;
import java.util.Map;
@Slf4j
-public class BanyanDBStorageDAO implements StorageDAO {
+public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> implements StorageDAO {
+ public BanyanDBStorageDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public IMetricsDAO newMetricsDao(StorageBuilder storageBuilder) {
- return new IMetricsDAO() {
- @Override
- public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
- return Collections.emptyList();
- }
-
- @Override
- public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
- return new InsertRequest() {
- };
- }
-
- @Override
- public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
- return new UpdateRequest() {
- };
+ try {
+ Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
+ if (Event.class.equals(returnType)) {
+ return new BanyanDBMetricsDAO<>(new EventBuilder());
+ } else if (ServiceTraffic.class.equals(returnType)) {
+ return new BanyanDBMetricsDAO<>(new Metadata.ServiceTrafficBuilder());
+ } else if (InstanceTraffic.class.equals(returnType)) {
+ return new BanyanDBMetricsDAO<>(new Metadata.InstanceTrafficBuilder());
+ } else if (EndpointTraffic.class.equals(returnType)) {
+ return new BanyanDBMetricsDAO<>(new Metadata.EndpointTrafficBuilder());
+ } else if (NetworkAddressAlias.class.equals(returnType)) {
+ return new BanyanDBMetricsDAO<>(new NetworkAddressAliasBuilder());
+ } else {
+ throw new IllegalStateException("record type is not supported");
}
- };
+ } catch (NoSuchMethodException noSuchMethodException) {
+ log.error("cannot find method storage2Entity", noSuchMethodException);
+ throw new RuntimeException("cannot find method storage2Entity");
+ }
}
@Override
public IRecordDAO newRecordDao(StorageBuilder storageBuilder) {
try {
- if (SegmentRecord.class.equals(storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType())) {
- return new BanyanDBRecordDAO(new BanyanDBSegmentRecordBuilder());
+ Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
+ if (SegmentRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new SegmentRecordBuilder());
+ } else if (AlarmRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new AlarmRecordBuilder());
+ } else if (BrowserErrorLogRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new BrowserErrorLogRecordBuilder());
+ } else if (LogRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new LogRecordBuilder());
+ } else if (ProfileTaskLogRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new ProfileTaskLogRecordBuilder());
+ } else if (ProfileThreadSnapshotRecord.class.equals(returnType)) {
+ return new BanyanDBRecordDAO<>(new ProfileThreadSnapshotRecordBuilder());
} else {
- return (model, record) -> new InsertRequest() {
- };
+ throw new IllegalStateException("record type is not supported");
}
} catch (NoSuchMethodException noSuchMethodException) {
log.error("cannot find method storage2Entity", noSuchMethodException);
@@ -77,13 +110,31 @@ public class BanyanDBStorageDAO implements StorageDAO {
@Override
public INoneStreamDAO newNoneStreamDao(StorageBuilder storageBuilder) {
- return (model, noneStream) -> {
- };
+ try {
+ Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
+ if (ProfileTaskRecord.class.equals(returnType)) {
+ return new BanyanDBNoneStreamDAO<>(getClient(), new ProfileTaskRecordBuilder());
+ } else {
+ throw new IllegalStateException("record type is not supported");
+ }
+ } catch (NoSuchMethodException noSuchMethodException) {
+ log.error("cannot find method storage2Entity", noSuchMethodException);
+ throw new RuntimeException("cannot find method storage2Entity");
+ }
}
@Override
public IManagementDAO newManagementDao(StorageBuilder storageBuilder) {
- return (model, storageData) -> {
- };
+ try {
+ Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
+ if (UITemplate.class.equals(returnType)) {
+ return new BanyanDBManagementDAO<>(getClient(), new UITemplateBuilder());
+ } else {
+ throw new IllegalStateException("record type is not supported");
+ }
+ } catch (NoSuchMethodException noSuchMethodException) {
+ log.error("cannot find method storage2Entity", noSuchMethodException);
+ throw new RuntimeException("cannot find method storage2Entity");
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
index aef8ec94e7..f8287973bc 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStreamInsertRequest.java
@@ -25,7 +25,6 @@ import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
@RequiredArgsConstructor
public class BanyanDBStreamInsertRequest implements InsertRequest {
-
@Getter
private final StreamWrite streamWrite;
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index bbfd4e3150..73e80c8f94 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -19,6 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
+import lombok.Getter;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
@@ -31,24 +32,14 @@ import org.apache.skywalking.oap.server.core.query.type.TraceBrief;
import org.apache.skywalking.oap.server.core.query.type.TraceState;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
-import org.joda.time.format.DateTimeFormat;
-import org.joda.time.format.DateTimeFormatter;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
- private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
- private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
-
- private static final DateTimeFormatter YYYYMMDDHHMMSS = DateTimeFormat.forPattern("yyyyMMddHHmmss");
-
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -81,13 +72,13 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
switch (traceState) {
case ERROR:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ERROR.getState()));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.ERROR.getState()));
break;
case SUCCESS:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.SUCCESS.getState()));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.SUCCESS.getState()));
break;
default:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ALL.getState()));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.ALL.getState()));
break;
}
@@ -102,7 +93,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
- if (BanyanDBSchema.INDEX_FIELDS.contains(tag.getKey())) {
+ if (SegmentRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
}
}
@@ -140,4 +131,15 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
public List<Span> doFlexibleTraceQuery(String traceId) throws IOException {
return Collections.emptyList();
}
+
+ public enum TraceStateStorage {
+ ALL(0), SUCCESS(1), ERROR(2);
+
+ @Getter
+ private final int state;
+
+ TraceStateStorage(int state) {
+ this.state = state;
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index 95731220fd..9ed0af8447 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
@@ -11,6 +29,7 @@ import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
import java.io.IOException;
import java.util.List;
@@ -19,8 +38,6 @@ import java.util.List;
* {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
*/
public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO implements UITemplateManagementDAO {
- private static final long UI_TEMPLATE_TIMESTAMP = 1L;
-
public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -54,7 +71,7 @@ public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO impleme
.dataTag(Tag.stringField(uiTemplate.getConfiguration()))
// data - activated
.dataTag(Tag.longField(uiTemplate.getActivated()))
- .timestamp(UI_TEMPLATE_TIMESTAMP)
+ .timestamp(UITemplateBuilder.UI_TEMPLATE_TIMESTAMP)
.elementId(uiTemplate.id())
.build();
getClient().write(request);
[skywalking] 05/24: refactor deser
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 7fb9a6ad37e96be4fc4104ae6e022514683a0492
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 2 22:13:57 2021 +0800
refactor deser
---
.../deserializer/AbstractBanyanDBDeserializer.java | 35 +++++
.../banyandb/deserializer/AlarmMessageMapper.java | 36 ++---
.../deserializer/BanyanDBDeserializerFactory.java | 54 +++++++
.../banyandb/deserializer/BasicTraceMapper.java | 18 +--
.../deserializer/BrowserErrorLogMapper.java | 24 ++-
.../deserializer/DashboardConfigurationMapper.java | 18 +--
.../banyandb/deserializer/DatabaseMapper.java | 25 ++++
.../banyandb/deserializer/EndpointMapper.java | 25 ++++
.../plugin/banyandb/deserializer/EventMapper.java | 37 +++--
.../plugin/banyandb/deserializer/LogMapper.java | 26 +---
.../deserializer/NetworkAddressAliasMapper.java | 15 +-
.../deserializer/ProfileTaskLogMapper.java | 16 +-
.../banyandb/deserializer/ProfileTaskMapper.java | 18 +--
.../ProfileThreadSnapshotRecordMapper.java | 16 +-
.../banyandb/deserializer/RowEntityMapper.java | 7 +-
.../banyandb/deserializer/SegmentRecordMapper.java | 18 +--
.../deserializer/ServiceInstanceMapper.java | 70 +++++++++
.../banyandb/deserializer/ServiceMapper.java | 26 ++++
.../banyandb/stream/AbstractBanyanDBDAO.java | 42 ++++++
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 49 +++----
.../stream/BanyanDBBrowserLogQueryDAO.java | 59 ++++----
.../banyandb/stream/BanyanDBEventQueryDAO.java | 86 ++++++++++-
.../banyandb/stream/BanyanDBLogQueryDAO.java | 71 ++++-----
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 77 ++++++++--
.../stream/BanyanDBNetworkAddressAliasDAO.java | 20 +--
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 19 +--
.../stream/BanyanDBProfileTaskQueryDAO.java | 69 +++++----
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 77 +++++-----
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 6 +-
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 161 +++++++++------------
.../stream/BanyanDBUITemplateManagementDAO.java | 26 ++--
31 files changed, 782 insertions(+), 464 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
new file mode 100644
index 0000000000..44adcb8869
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
@@ -0,0 +1,35 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+
+import java.util.Collections;
+import java.util.List;
+
+public abstract class AbstractBanyanDBDeserializer<T> implements RowEntityMapper<T> {
+ private final String indexName;
+ private final List<String> searchableProjection;
+ private final List<String> dataProjection;
+
+ protected AbstractBanyanDBDeserializer(String indexName, List<String> searchableProjection) {
+ this(indexName, searchableProjection, Collections.emptyList());
+ }
+
+ protected AbstractBanyanDBDeserializer(String indexName, List<String> searchableProjection, List<String> dataProjection) {
+ this.indexName = indexName;
+ this.searchableProjection = searchableProjection;
+ this.dataProjection = dataProjection;
+ }
+
+ public StreamQuery buildStreamQuery() {
+ final StreamQuery query = new StreamQuery(this.indexName, this.searchableProjection);
+ query.setDataProjections(this.dataProjection);
+ return query;
+ }
+
+ public StreamQuery buildStreamQuery(long startTimestamp, long endTimestamp) {
+ final StreamQuery query = new StreamQuery(this.indexName, new TimestampRange(startTimestamp, endTimestamp), this.searchableProjection);
+ query.setDataProjections(this.dataProjection);
+ return query;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
index d906dbb78f..939496e061 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageMapper.java
@@ -1,33 +1,27 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+import com.google.common.base.Charsets;
import com.google.common.collect.ImmutableList;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
import com.google.protobuf.ByteString;
-import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Scope;
import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
-import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
+import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import java.util.List;
-@RequiredArgsConstructor
-public class AlarmMessageMapper implements RowEntityMapper<AlarmMessage> {
- private final IAlarmQueryDAO alarmQueryDAO;
+public class AlarmMessageMapper extends AbstractBanyanDBDeserializer<AlarmMessage> {
+ private final Gson GSON = new Gson();
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(AlarmRecord.SCOPE, // 0
- AlarmRecord.START_TIME); // 1
- }
-
- @Override
- public List<String> dataProjection() {
- return ImmutableList.of(AlarmRecord.ID0, // 0
- AlarmRecord.ID1, // 1
- AlarmRecord.ALARM_MESSAGE, // 2
- AlarmRecord.TAGS_RAW_DATA); // 3
+ public AlarmMessageMapper() {
+ super(AlarmRecord.INDEX_NAME,
+ ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
+ ImmutableList.of(AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA));
}
@Override
@@ -44,8 +38,14 @@ public class AlarmMessageMapper implements RowEntityMapper<AlarmMessage> {
alarmMessage.setMessage((String) data.get(2).getValue());
Object o = data.get(3).getValue();
if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- this.alarmQueryDAO.parserDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
+ this.parseDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
}
return alarmMessage;
}
+
+ void parseDataBinary(byte[] dataBinary, List<KeyValue> tags) {
+ List<Tag> tagList = GSON.fromJson(new String(dataBinary, Charsets.UTF_8), new TypeToken<List<Tag>>() {
+ }.getType());
+ tagList.forEach(pair -> tags.add(new KeyValue(pair.getKey(), pair.getValue())));
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
new file mode 100644
index 0000000000..dd4af41736
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
@@ -0,0 +1,54 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
+import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
+import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
+import org.apache.skywalking.oap.server.core.query.type.Database;
+import org.apache.skywalking.oap.server.core.query.type.Endpoint;
+import org.apache.skywalking.oap.server.core.query.type.Log;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
+import org.apache.skywalking.oap.server.core.query.type.Service;
+import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.query.type.event.Event;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public enum BanyanDBDeserializerFactory {
+ INSTANCE;
+
+ private final Map<Class<?>, AbstractBanyanDBDeserializer<?>> registry;
+
+ BanyanDBDeserializerFactory() {
+ registry = new HashMap<>(10);
+ register(AlarmMessage.class, new AlarmMessageMapper());
+ register(BasicTrace.class, new BasicTraceMapper());
+ register(BrowserErrorLog.class, new BrowserErrorLogMapper());
+ register(DashboardConfiguration.class, new DashboardConfigurationMapper());
+ register(Database.class, new DatabaseMapper());
+ register(Endpoint.class, new EndpointMapper());
+ register(Event.class, new EventMapper());
+ register(Log.class, new LogMapper());
+ register(NetworkAddressAlias.class, new NetworkAddressAliasMapper());
+ register(ProfileTaskLog.class, new ProfileTaskLogMapper());
+ register(ProfileTask.class, new ProfileTaskMapper());
+ register(ProfileThreadSnapshotRecord.class, new ProfileThreadSnapshotRecordMapper());
+ register(SegmentRecord.class, new SegmentRecordMapper());
+ register(ServiceInstance.class, new ServiceInstanceMapper());
+ register(Service.class, new ServiceMapper());
+ }
+
+ private <T> void register(Class<T> clazz, AbstractBanyanDBDeserializer<T> mapper) {
+ this.registry.put(clazz, mapper);
+ }
+
+ @SuppressWarnings({"unchecked"})
+ public <T> AbstractBanyanDBDeserializer<T> findDeserializer(Class<T> clazz) {
+ return (AbstractBanyanDBDeserializer<T>) registry.get(clazz);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
index bbcf7091d0..5d7ae36590 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceMapper.java
@@ -4,12 +4,16 @@ import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
-import java.util.Collections;
import java.util.List;
-public class BasicTraceMapper implements RowEntityMapper<BasicTrace> {
+public class BasicTraceMapper extends AbstractBanyanDBDeserializer<BasicTrace> {
+ public BasicTraceMapper() {
+ super(SegmentRecord.INDEX_NAME, ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"));
+ }
+
@Override
public BasicTrace map(RowEntity row) {
BasicTrace trace = new BasicTrace();
@@ -24,14 +28,4 @@ public class BasicTraceMapper implements RowEntityMapper<BasicTrace> {
trace.setStart(String.valueOf(searchable.get(4).getValue()));
return trace;
}
-
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time");
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.emptyList();
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
index 0807955376..0189a77049 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogMapper.java
@@ -9,22 +9,18 @@ import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErro
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
+import java.util.Collections;
import java.util.List;
-public class BrowserErrorLogMapper implements RowEntityMapper<BrowserErrorLog> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
- BrowserErrorLogRecord.SERVICE_VERSION_ID,
- BrowserErrorLogRecord.PAGE_PATH_ID,
- BrowserErrorLogRecord.ERROR_CATEGORY,
- BrowserErrorLogRecord.TIMESTAMP
- );
- }
-
- @Override
- public List<String> dataProjection() {
- return ImmutableList.of(BrowserErrorLogRecord.DATA_BINARY);
+public class BrowserErrorLogMapper extends AbstractBanyanDBDeserializer<BrowserErrorLog> {
+ public BrowserErrorLogMapper() {
+ super(BrowserErrorLogRecord.INDEX_NAME,
+ ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
+ BrowserErrorLogRecord.SERVICE_VERSION_ID,
+ BrowserErrorLogRecord.PAGE_PATH_ID,
+ BrowserErrorLogRecord.ERROR_CATEGORY,
+ BrowserErrorLogRecord.TIMESTAMP),
+ Collections.singletonList(BrowserErrorLogRecord.DATA_BINARY));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
index 6b9e0962e6..cb6b4f3c2f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationMapper.java
@@ -10,7 +10,13 @@ import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import java.util.List;
-public class DashboardConfigurationMapper implements RowEntityMapper<DashboardConfiguration> {
+public class DashboardConfigurationMapper extends AbstractBanyanDBDeserializer<DashboardConfiguration> {
+ public DashboardConfigurationMapper() {
+ super(UITemplate.INDEX_NAME,
+ ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED),
+ ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE));
+ }
+
@Override
public DashboardConfiguration map(RowEntity row) {
DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
@@ -28,14 +34,4 @@ public class DashboardConfigurationMapper implements RowEntityMapper<DashboardCo
dashboardConfiguration.setType(TemplateType.forName((String) data.get(2).getValue()));
return dashboardConfiguration;
}
-
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED);
- }
-
- @Override
- public List<String> dataProjection() {
- return ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE);
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java
new file mode 100644
index 0000000000..1c5a15f774
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseMapper.java
@@ -0,0 +1,25 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.query.type.Database;
+
+import java.util.List;
+
+public class DatabaseMapper extends AbstractBanyanDBDeserializer<Database> {
+ public DatabaseMapper() {
+ super(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE));
+ }
+
+ @Override
+ public Database map(RowEntity row) {
+ Database database = new Database();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ database.setId(row.getId());
+ database.setName((String) searchable.get(0).getValue());
+ return database;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java
new file mode 100644
index 0000000000..8f81df039c
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointMapper.java
@@ -0,0 +1,25 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.query.type.Endpoint;
+
+import java.util.List;
+
+public class EndpointMapper extends AbstractBanyanDBDeserializer<Endpoint> {
+ public EndpointMapper() {
+ super(EndpointTraffic.INDEX_NAME,
+ ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID));
+ }
+
+ @Override
+ public Endpoint map(RowEntity row) {
+ Endpoint endpoint = new Endpoint();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ endpoint.setName((String) searchable.get(0).getValue());
+ endpoint.setId((String) searchable.get(1).getValue());
+ return endpoint;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
index 1c94903cbd..63c9dc802c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventMapper.java
@@ -2,23 +2,36 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.oap.server.core.query.type.event.Event;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.query.type.event.EventType;
+import org.apache.skywalking.oap.server.core.query.type.event.Source;
+import org.apache.skywalking.oap.server.core.source.Event;
import java.util.List;
-public class EventMapper implements RowEntityMapper<Event> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of();
- }
-
- @Override
- public List<String> dataProjection() {
- return null;
+public class EventMapper extends AbstractBanyanDBDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
+ public EventMapper() {
+ super(Event.INDEX_NAME,
+ ImmutableList.of(Event.UUID, Event.SERVICE, Event.SERVICE_INSTANCE, Event.ENDPOINT, Event.NAME,
+ Event.TYPE, Event.START_TIME, Event.END_TIME),
+ ImmutableList.of(Event.MESSAGE, Event.PARAMETERS));
}
@Override
- public Event map(RowEntity row) {
- return null;
+ public org.apache.skywalking.oap.server.core.query.type.event.Event map(RowEntity row) {
+ final org.apache.skywalking.oap.server.core.query.type.event.Event resultEvent = new org.apache.skywalking.oap.server.core.query.type.event.Event();
+ // searchable
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ resultEvent.setUuid((String) searchable.get(0).getValue());
+ resultEvent.setSource(new Source((String) searchable.get(1).getValue(), (String) searchable.get(2).getValue(), (String) searchable.get(3).getValue()));
+ resultEvent.setName((String) searchable.get(4).getValue());
+ resultEvent.setType(EventType.parse((String) searchable.get(5).getValue()));
+ resultEvent.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ resultEvent.setEndTime(((Number) searchable.get(7).getValue()).longValue());
+ // data
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ resultEvent.setMessage((String) data.get(0).getValue());
+ resultEvent.setParameters((String) data.get(1).getValue());
+ return resultEvent;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
index f1c5aaf442..3f65a82cea 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogMapper.java
@@ -8,29 +8,19 @@ import org.apache.skywalking.apm.network.logging.v3.LogTags;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import org.apache.skywalking.oap.server.core.query.type.Log;
import java.util.List;
-public class LogMapper implements RowEntityMapper<Log> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(
- AbstractLogRecord.SERVICE_ID, // 0
- AbstractLogRecord.SERVICE_INSTANCE_ID, // 1
- AbstractLogRecord.ENDPOINT_ID, // 2
- AbstractLogRecord.TRACE_ID, // 3
- AbstractLogRecord.TRACE_SEGMENT_ID,
- AbstractLogRecord.SPAN_ID,
- AbstractLogRecord.TIMESTAMP); // 6
- }
-
- @Override
- public List<String> dataProjection() {
- return ImmutableList.of(AbstractLogRecord.CONTENT_TYPE,
- AbstractLogRecord.CONTENT,
- AbstractLogRecord.TAGS_RAW_DATA); // 2
+public class LogMapper extends AbstractBanyanDBDeserializer<Log> {
+ public LogMapper() {
+ super(LogRecord.INDEX_NAME, ImmutableList.of(
+ AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
+ AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
+ AbstractLogRecord.SPAN_ID, AbstractLogRecord.TIMESTAMP),
+ ImmutableList.of(AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT, AbstractLogRecord.TAGS_RAW_DATA));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
index aeb823c752..7c231df86a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasMapper.java
@@ -8,16 +8,11 @@ import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import java.util.List;
-public class NetworkAddressAliasMapper implements RowEntityMapper<NetworkAddressAlias> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET);
- }
-
- @Override
- public List<String> dataProjection() {
- // TODO: make these static fields public
- return ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id");
+public class NetworkAddressAliasMapper extends AbstractBanyanDBDeserializer<NetworkAddressAlias> {
+ public NetworkAddressAliasMapper() {
+ super(NetworkAddressAlias.INDEX_NAME,
+ ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET),
+ ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id"));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
index bc157253f0..de51d2f95c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogMapper.java
@@ -9,16 +9,12 @@ import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationT
import java.util.List;
-public class ProfileTaskLogMapper implements RowEntityMapper<ProfileTaskLog> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME);
- }
-
- @Override
- public List<String> dataProjection() {
- return ImmutableList.of(ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.INSTANCE_ID,
- ProfileTaskLogRecord.OPERATION_TYPE);
+public class ProfileTaskLogMapper extends AbstractBanyanDBDeserializer<ProfileTaskLog> {
+ public ProfileTaskLogMapper() {
+ super(ProfileTaskLogRecord.INDEX_NAME,
+ ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME),
+ ImmutableList.of(ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.INSTANCE_ID,
+ ProfileTaskLogRecord.OPERATION_TYPE));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
index 8a49404d23..2626a91c7d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskMapper.java
@@ -6,22 +6,16 @@ import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import java.util.Collections;
import java.util.List;
-public class ProfileTaskMapper implements RowEntityMapper<ProfileTask> {
+public class ProfileTaskMapper extends AbstractBanyanDBDeserializer<ProfileTask> {
public static final String ID = "profile_task_query_id";
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
- ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
- ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT);
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.emptyList();
+ public ProfileTaskMapper() {
+ super(ProfileTaskRecord.INDEX_NAME,
+ ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
+ ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
index 99f23117f2..2d966a5955 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordMapper.java
@@ -9,16 +9,12 @@ import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord
import java.util.Collections;
import java.util.List;
-public class ProfileThreadSnapshotRecordMapper implements RowEntityMapper<ProfileThreadSnapshotRecord> {
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
- ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE);
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY);
+public class ProfileThreadSnapshotRecordMapper extends AbstractBanyanDBDeserializer<ProfileThreadSnapshotRecord> {
+ public ProfileThreadSnapshotRecordMapper() {
+ super(ProfileThreadSnapshotRecord.INDEX_NAME,
+ ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
index cc1d48d94f..51f9a5d687 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityMapper.java
@@ -2,12 +2,7 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import java.util.List;
-
+@FunctionalInterface
public interface RowEntityMapper<T> {
- List<String> searchableProjection();
-
- List<String> dataProjection();
-
T map(RowEntity row);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
index 70318136f6..da7be9de35 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordMapper.java
@@ -9,7 +9,13 @@ import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentReco
import java.util.Collections;
import java.util.List;
-public class SegmentRecordMapper implements RowEntityMapper<SegmentRecord> {
+public class SegmentRecordMapper extends AbstractBanyanDBDeserializer<SegmentRecord> {
+ public SegmentRecordMapper() {
+ super(SegmentRecord.INDEX_NAME,
+ ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
+ Collections.singletonList("data_binary"));
+ }
+
@Override
public SegmentRecord map(RowEntity row) {
SegmentRecord record = new SegmentRecord();
@@ -26,14 +32,4 @@ public class SegmentRecordMapper implements RowEntityMapper<SegmentRecord> {
record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
return record;
}
-
- @Override
- public List<String> searchableProjection() {
- return ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time");
- }
-
- @Override
- public List<String> dataProjection() {
- return Collections.singletonList("data_binary");
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java
new file mode 100644
index 0000000000..f6925d3fd3
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceMapper.java
@@ -0,0 +1,70 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import com.google.gson.Gson;
+import com.google.gson.JsonElement;
+import com.google.gson.JsonObject;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.query.enumeration.Language;
+import org.apache.skywalking.oap.server.core.query.type.Attribute;
+import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+public class ServiceInstanceMapper extends AbstractBanyanDBDeserializer<ServiceInstance> {
+ private static final Gson GSON = new Gson();
+
+ public ServiceInstanceMapper() {
+ super(InstanceTraffic.INDEX_NAME,
+ ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
+ Collections.singletonList("data_binary"));
+ }
+
+ @Override
+ public ServiceInstance map(RowEntity row) {
+ ServiceInstance serviceInstance = new ServiceInstance();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ serviceInstance.setId((String) searchable.get(0).getValue());
+ serviceInstance.setInstanceUUID((String) searchable.get(0).getValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ Object o = data.get(0).getValue();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ try {
+ RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
+ serviceInstance.setName(remoteData.getDataStrings(1));
+ final String propString = remoteData.getDataStrings(2);
+ if (StringUtil.isNotEmpty(propString)) {
+ JsonObject properties = GSON.fromJson(propString, JsonObject.class);
+ if (properties != null) {
+ for (Map.Entry<String, JsonElement> property : properties.entrySet()) {
+ String key = property.getKey();
+ String value = property.getValue().getAsString();
+ if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
+ serviceInstance.setLanguage(Language.value(value));
+ } else {
+ serviceInstance.getAttributes().add(new Attribute(key, value));
+ }
+ }
+ } else {
+ serviceInstance.setLanguage(Language.UNKNOWN);
+ }
+ } else {
+ serviceInstance.setLanguage(Language.UNKNOWN);
+ }
+ } catch (InvalidProtocolBufferException ex) {
+ throw new RuntimeException("fail to parse remote data", ex);
+ }
+ } else {
+ serviceInstance.setLanguage(Language.UNKNOWN);
+ }
+ return serviceInstance;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java
new file mode 100644
index 0000000000..e006a1107d
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceMapper.java
@@ -0,0 +1,26 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.query.type.Service;
+
+import java.util.List;
+
+public class ServiceMapper extends AbstractBanyanDBDeserializer<Service> {
+ public ServiceMapper() {
+ super(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP));
+ }
+
+ @Override
+ public Service map(RowEntity row) {
+ Service service = new Service();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ service.setId(row.getId());
+ service.setName((String) searchable.get(0).getValue());
+ service.setGroup((String) searchable.get(2).getValue());
+ return service;
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
new file mode 100644
index 0000000000..b11cca6ed4
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -0,0 +1,42 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.AbstractBanyanDBDeserializer;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BanyanDBDeserializerFactory;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageClient> {
+ protected AbstractBanyanDBDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
+ protected <T> List<T> query(Class<T> clazz, QueryBuilder builder) {
+ return this.query(clazz, builder, 0, 0);
+ }
+
+ protected <T> List<T> query(Class<T> clazz, QueryBuilder builder, long startTimestamp, long endTimestamp) {
+ AbstractBanyanDBDeserializer<T> deserializer = BanyanDBDeserializerFactory.INSTANCE.findDeserializer(clazz);
+
+ final StreamQuery query;
+ if (startTimestamp != 0 && endTimestamp != 0) {
+ query = deserializer.buildStreamQuery();
+ } else {
+ query = deserializer.buildStreamQuery(startTimestamp, endTimestamp);
+ }
+
+ builder.apply(query);
+
+ final StreamQueryResponse resp = getClient().query(query);
+ return resp.getElements().stream().map(deserializer::map).collect(Collectors.toList());
+ }
+
+
+ interface QueryBuilder {
+ void apply(final StreamQuery query);
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index 7e122611bf..57f658a380 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -2,57 +2,48 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
import org.apache.skywalking.oap.server.core.query.type.Alarms;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.AlarmMessageMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.alarm.AlarmRecord} is a stream,
* which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
*/
-public class BanyanDBAlarmQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IAlarmQueryDAO {
- private final RowEntityMapper<AlarmMessage> mapper;
-
+public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarmQueryDAO {
public BanyanDBAlarmQueryDAO(BanyanDBStorageClient client) {
super(client);
- mapper = new AlarmMessageMapper(this);
}
@Override
public Alarms getAlarm(Integer scopeId, String keyword, int limit, int from, long startTB, long endTB, List<Tag> tags) throws IOException {
- final StreamQuery query = new StreamQuery(AlarmRecord.INDEX_NAME, mapper.searchableProjection());
-
- if (Objects.nonNull(scopeId)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AlarmRecord.SCOPE, (long) scopeId));
- }
- if (startTB != 0 && endTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(startTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(endTB)));
- }
-
- // TODO: support keyword search
-
- // TODO: support tag search
-
- query.setLimit(limit);
- query.setOffset(from);
-
- StreamQueryResponse resp = getClient().query(query);
-
- List<AlarmMessage> messages = resp.getElements().stream().map(mapper::map).collect(Collectors.toList());
+ List<AlarmMessage> messages = query(AlarmMessage.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (Objects.nonNull(scopeId)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AlarmRecord.SCOPE, (long) scopeId));
+ }
+ if (startTB != 0 && endTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(startTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(endTB)));
+ }
+
+ // TODO: support keyword search
+
+ // TODO: support tag search
+
+ query.setLimit(limit);
+ query.setOffset(from);
+ }
+ });
Alarms alarms = new Alarms();
alarms.setTotal(messages.size());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index 25f2c27f4b..3bd3af8ad5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -2,61 +2,54 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
import org.apache.skywalking.oap.server.core.browser.source.BrowserErrorCategory;
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLogs;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BrowserErrorLogMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
+import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
*/
-public class BanyanDBBrowserLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IBrowserLogQueryDAO {
- private static final RowEntityMapper<BrowserErrorLog> MAPPER = new BrowserErrorLogMapper();
-
+public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements IBrowserLogQueryDAO {
public BanyanDBBrowserLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public BrowserErrorLogs queryBrowserErrorLogs(String serviceId, String serviceVersionId, String pagePathId, BrowserErrorCategory category, long startSecondTB, long endSecondTB, int limit, int from) throws IOException {
- final StreamQuery query = new StreamQuery(BrowserErrorLogRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.setDataProjections(MAPPER.dataProjection());
-
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_ID, serviceId));
-
- if (startSecondTB != 0 && endSecondTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startSecondTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endSecondTB)));
- }
- if (StringUtil.isNotEmpty(serviceVersionId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
- }
- if (StringUtil.isNotEmpty(pagePathId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.PAGE_PATH_ID, pagePathId));
- }
- if (Objects.nonNull(category)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", BrowserErrorLogRecord.ERROR_CATEGORY, (long) category.getValue()));
- }
-
- query.setOffset(from);
- query.setLimit(limit);
-
- final StreamQueryResponse resp = getClient().query(query);
-
final BrowserErrorLogs logs = new BrowserErrorLogs();
- logs.getLogs().addAll(resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList()));
+ List<BrowserErrorLog> browserErrorLogs = query(BrowserErrorLog.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_ID, serviceId));
+
+ if (startSecondTB != 0 && endSecondTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startSecondTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", BrowserErrorLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endSecondTB)));
+ }
+ if (StringUtil.isNotEmpty(serviceVersionId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
+ }
+ if (StringUtil.isNotEmpty(pagePathId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.PAGE_PATH_ID, pagePathId));
+ }
+ if (Objects.nonNull(category)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", BrowserErrorLogRecord.ERROR_CATEGORY, (long) category.getValue()));
+ }
+
+ query.setOffset(from);
+ query.setLimit(limit);
+ }
+ });
+ logs.getLogs().addAll(browserErrorLogs);
logs.setTotal(logs.getLogs().size());
return logs;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
index 5ab67d95c6..9b2f0d2be5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
@@ -1,23 +1,101 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import com.google.common.base.Strings;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.oap.server.core.query.PaginationUtils;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
import org.apache.skywalking.oap.server.core.query.type.event.EventQueryCondition;
import org.apache.skywalking.oap.server.core.query.type.event.Events;
+import org.apache.skywalking.oap.server.core.query.type.event.Source;
+import org.apache.skywalking.oap.server.core.source.Event;
import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.util.List;
/**
- * ???
* {@link org.apache.skywalking.oap.server.core.source.Event} is a stream
*/
-public class BanyanDBEventQueryDAO implements IEventQueryDAO {
+public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEventQueryDAO {
+ public BanyanDBEventQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public Events queryEvents(EventQueryCondition condition) throws Exception {
- return new Events();
+ List<org.apache.skywalking.oap.server.core.query.type.event.Event> eventList = query(org.apache.skywalking.oap.server.core.query.type.event.Event.class,
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ buildConditions(condition, query);
+
+ PaginationUtils.Page page = PaginationUtils.INSTANCE.exchange(condition.getPaging());
+ query.setLimit(page.getLimit());
+ query.setOffset(page.getFrom());
+ switch (condition.getOrder()) {
+ case ASC:
+ query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.ASC));
+ break;
+ case DES:
+ query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
+ }
+ }
+ });
+
+ Events events = new Events();
+ events.setEvents(eventList);
+ // TODO: how to set total???
+ events.setTotal(eventList.size());
+ return events;
}
@Override
public Events queryEvents(List<EventQueryCondition> conditionList) throws Exception {
- return new Events();
+ Events events = new Events();
+ for (final EventQueryCondition condition : conditionList) {
+ Events subEvents = this.queryEvents(condition);
+ if (subEvents.getEvents().size() == 0) {
+ continue;
+ }
+
+ events.getEvents().addAll(subEvents.getEvents());
+ events.setTotal(events.getTotal() + subEvents.getTotal());
+ }
+
+ return events;
+ }
+
+ private void buildConditions(EventQueryCondition condition, final StreamQuery query) {
+ if (!Strings.isNullOrEmpty(condition.getUuid())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.UUID, condition.getUuid()));
+ }
+ final Source source = condition.getSource();
+ if (source != null) {
+ if (!Strings.isNullOrEmpty(source.getService())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.SERVICE, source.getService()));
+ }
+ if (!Strings.isNullOrEmpty(source.getServiceInstance())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.SERVICE_INSTANCE, source.getServiceInstance()));
+ }
+ if (!Strings.isNullOrEmpty(source.getEndpoint())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.ENDPOINT, source.getEndpoint()));
+ }
+ }
+ if (!Strings.isNullOrEmpty(condition.getName())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.NAME, condition.getName()));
+ }
+ if (condition.getType() != null) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.TYPE, condition.getType().name()));
+ }
+ final Duration time = condition.getTime();
+ if (time != null) {
+ if (time.getStartTimestamp() > 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.gt("searchable", Event.START_TIME, time.getStartTimestamp()));
+ }
+ if (time.getEndTimestamp() > 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.gt("searchable", Event.END_TIME, time.getEndTimestamp()));
+ }
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index 661f65b930..404d60017c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -2,33 +2,25 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Order;
import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
import org.apache.skywalking.oap.server.core.query.type.Log;
import org.apache.skywalking.oap.server.core.query.type.Logs;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.LogMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
*/
-public class BanyanDBLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements ILogQueryDAO {
- private static final RowEntityMapper<Log> MAPPER = new LogMapper();
-
+public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQueryDAO {
public BanyanDBLogQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -38,45 +30,44 @@ public class BanyanDBLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> impl
TraceScopeCondition relatedTrace, Order queryOrder, int from, int limit,
long startTB, long endTB, List<Tag> tags, List<String> keywordsOfContent,
List<String> excludingKeywordsOfContent) throws IOException {
- final StreamQuery query = new StreamQuery(LogRecord.INDEX_NAME, MAPPER.searchableProjection());
- if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_ID, serviceId));
- }
+ List<Log> entities = query(Log.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_ID, serviceId));
+ }
- if (startTB != 0 && endTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endTB)));
- }
+ if (startTB != 0 && endTB != 0) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(startTB)));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AbstractLogRecord.TIMESTAMP, TimeBucket.getTimestamp(endTB)));
+ }
- if (StringUtil.isNotEmpty(serviceInstanceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
- }
- if (StringUtil.isNotEmpty(endpointId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.ENDPOINT_ID, endpointId));
- }
- if (Objects.nonNull(relatedTrace)) {
- if (StringUtil.isNotEmpty(relatedTrace.getTraceId())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_ID, relatedTrace.getTraceId()));
- }
- if (StringUtil.isNotEmpty(relatedTrace.getSegmentId())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_SEGMENT_ID, relatedTrace.getSegmentId()));
- }
- if (Objects.nonNull(relatedTrace.getSpanId())) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AbstractLogRecord.SPAN_ID, (long) relatedTrace.getSpanId()));
- }
- }
+ if (StringUtil.isNotEmpty(serviceInstanceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
+ }
+ if (StringUtil.isNotEmpty(endpointId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.ENDPOINT_ID, endpointId));
+ }
+ if (Objects.nonNull(relatedTrace)) {
+ if (StringUtil.isNotEmpty(relatedTrace.getTraceId())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_ID, relatedTrace.getTraceId()));
+ }
+ if (StringUtil.isNotEmpty(relatedTrace.getSegmentId())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_SEGMENT_ID, relatedTrace.getSegmentId()));
+ }
+ if (Objects.nonNull(relatedTrace.getSpanId())) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AbstractLogRecord.SPAN_ID, (long) relatedTrace.getSpanId()));
+ }
+ }
- // TODO: if we allow to index tags?
+ // TODO: if we allow to index tags?
// if (CollectionUtils.isNotEmpty(tags)) {
// for (final Tag tag : tags) {
// query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
// }
// }
-
- StreamQueryResponse resp = getClient().query(query);
-
- List<Log> entities = resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
-
+ }
+ });
Logs logs = new Logs();
logs.getLogs().addAll(entities);
logs.setTotal(entities.size());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index 678b981f16..dd3d9623a6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -1,15 +1,23 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.oap.server.core.analysis.NodeType;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
import org.apache.skywalking.oap.server.core.query.type.Database;
import org.apache.skywalking.oap.server.core.query.type.Endpoint;
import org.apache.skywalking.oap.server.core.query.type.Service;
import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.io.IOException;
-import java.util.Collections;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic},
@@ -17,39 +25,90 @@ import java.util.List;
* {@link org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic}
* are all streams.
*/
-public class BanyanDBMetadataQueryDAO implements IMetadataQueryDAO {
+public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMetadataQueryDAO {
+ public BanyanDBMetadataQueryDAO(BanyanDBStorageClient client) {
+ super(client);
+ }
+
@Override
public List<Service> getAllServices(String group) throws IOException {
- return Collections.emptyList();
+ return query(Service.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(group)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.GROUP, group));
+ }
+ }
+ });
}
@Override
public List<Service> getAllBrowserServices() throws IOException {
- return Collections.emptyList();
+ return query(Service.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) NodeType.Browser.value()));
+ }
+ });
}
@Override
public List<Database> getAllDatabases() throws IOException {
- return Collections.emptyList();
+ return query(Database.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) NodeType.Database.value()));
+ }
+ });
}
@Override
public List<Service> searchServices(NodeType nodeType, String keyword) throws IOException {
- return null;
+ return query(Service.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
+ }
+ }).stream().filter((s) -> s.getName().contains(keyword)) // TODO: support analyzer in database
+ .collect(Collectors.toList());
}
@Override
public Service searchService(NodeType nodeType, String serviceCode) throws IOException {
- return null;
+ return query(Service.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.NAME, serviceCode));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
+ // only get one
+ query.setLimit(1);
+ }
+ }).stream().findAny().orElse(null);
}
@Override
public List<Endpoint> searchEndpoint(String keyword, String serviceId, int limit) throws IOException {
- return Collections.emptyList();
+ return query(Endpoint.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", EndpointTraffic.SERVICE_ID, serviceId));
+ }
+ }).stream().filter((e) -> e.getName().contains(keyword))
+ .limit(limit).collect(Collectors.toList());
}
@Override
public List<ServiceInstance> getServiceInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
- return Collections.emptyList();
+ return query(ServiceInstance.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
+ final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
+
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", InstanceTraffic.SERVICE_ID, serviceId));
+ }
+ });
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
index 337c288e96..6112593a55 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -2,33 +2,27 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.NetworkAddressAliasMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.util.List;
-import java.util.stream.Collectors;
/**
* {@link NetworkAddressAlias} is a stream
*/
-public class BanyanDBNetworkAddressAliasDAO extends AbstractDAO<BanyanDBStorageClient> implements INetworkAddressAliasDAO {
- private static final RowEntityMapper<NetworkAddressAlias> MAPPER = new NetworkAddressAliasMapper();
-
+public class BanyanDBNetworkAddressAliasDAO extends AbstractBanyanDBDAO implements INetworkAddressAliasDAO {
public BanyanDBNetworkAddressAliasDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
- StreamQuery query = new StreamQuery(NetworkAddressAlias.INDEX_NAME, MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
-
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+ return query(NetworkAddressAlias.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
+ }
+ });
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 6c19e2f701..0095965109 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -1,10 +1,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskLogMapper;
@@ -18,7 +15,7 @@ import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
*/
-public class BanyanDBProfileTaskLogQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileTaskLogQueryDAO {
+public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskLogQueryDAO {
private static final RowEntityMapper<ProfileTaskLog> MAPPER = new ProfileTaskLogMapper();
private final int queryMaxSize;
@@ -30,14 +27,12 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractDAO<BanyanDBStorageC
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
- final StreamQuery query = new StreamQuery(ProfileTaskLogRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.setDataProjections(MAPPER.dataProjection());
- query.setLimit(this.queryMaxSize);
-
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream()
- .map(MAPPER::map)
- .sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
+ return query(ProfileTaskLog.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
+ }
+ }).stream().sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
.collect(Collectors.toList());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 67f28ce62f..23fb8c7d1b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -2,11 +2,9 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
@@ -16,12 +14,11 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.Row
import java.io.IOException;
import java.util.List;
import java.util.Objects;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
*/
-public class BanyanDBProfileTaskQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileTaskQueryDAO {
+public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskQueryDAO {
private static final RowEntityMapper<ProfileTask> MAPPER = new ProfileTaskMapper();
public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
@@ -30,37 +27,36 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractDAO<BanyanDBStorageClie
@Override
public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
- final StreamQuery query = new StreamQuery(ProfileTaskRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.setDataProjections(MAPPER.dataProjection());
+ return query(ProfileTask.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
+ ProfileTaskRecord.SERVICE_ID, serviceId));
+ }
- if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
- ProfileTaskRecord.SERVICE_ID, serviceId));
- }
-
- if (StringUtil.isNotEmpty(endpointName)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
- ProfileTaskRecord.ENDPOINT_NAME, endpointName));
- }
-
- if (Objects.nonNull(startTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable",
- ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
- }
+ if (StringUtil.isNotEmpty(endpointName)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
+ ProfileTaskRecord.ENDPOINT_NAME, endpointName));
+ }
- if (Objects.nonNull(endTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable",
- ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
- }
+ if (Objects.nonNull(startTimeBucket)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable",
+ ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
+ }
- if (Objects.nonNull(limit)) {
- query.setLimit(limit);
- }
+ if (Objects.nonNull(endTimeBucket)) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable",
+ ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
+ }
- query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
+ if (Objects.nonNull(limit)) {
+ query.setLimit(limit);
+ }
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+ query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
+ }
+ });
}
@Override
@@ -69,11 +65,12 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractDAO<BanyanDBStorageClie
return null;
}
- final StreamQuery query = new StreamQuery(ProfileTaskRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskMapper.ID, id));
- query.setLimit(1);
-
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(MAPPER::map).findAny().orElse(null);
+ return query(ProfileTask.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskMapper.ID, id));
+ query.setLimit(1);
+ }
+ }).stream().findAny().orElse(null);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index cb5067e890..e92a0b24ff 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -2,11 +2,9 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
@@ -15,7 +13,6 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.Row
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
import java.io.IOException;
-import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.LinkedList;
@@ -26,7 +23,7 @@ import java.util.stream.Collectors;
/**
* {@link ProfileThreadSnapshotRecord} is a stream
*/
-public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements IProfileThreadSnapshotQueryDAO {
+public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO implements IProfileThreadSnapshotQueryDAO {
private static final RowEntityMapper<ProfileThreadSnapshotRecord> MAPPER = new ProfileThreadSnapshotRecordMapper();
private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
@@ -37,24 +34,30 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractDAO<BanyanDBS
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
- final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.TASK_ID, taskId))
- .appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEQUENCE, 0L));
- StreamQueryResponse resp = getClient().query(query);
-
- final List<String> segmentIDs = new ArrayList<>(resp.size());
- resp.getElements().forEach(elem -> segmentIDs.add(MAPPER.map(elem).getSegmentId()));
- if (segmentIDs.isEmpty()) {
+ List<ProfileThreadSnapshotRecord> resp = query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.TASK_ID, taskId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEQUENCE, 0L));
+ }
+ });
+
+ if (resp.isEmpty()) {
return Collections.emptyList();
}
+ final List<String> segmentIDs = resp.stream().map(ProfileThreadSnapshotRecord::getSegmentId).collect(Collectors.toList());
+
// TODO: support `IN` or `OR` logic operation in BanyanDB
List<BasicTrace> basicTraces = new LinkedList<>();
for (String segmentID : segmentIDs) {
- final StreamQuery traceQuery = new StreamQuery(SegmentRecord.INDEX_NAME, BASIC_TRACE_MAPPER.searchableProjection());
- traceQuery.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.SEGMENT_ID, segmentID));
- StreamQueryResponse traceResponse = getClient().query(traceQuery);
- basicTraces.addAll(traceResponse.getElements().stream().map(BASIC_TRACE_MAPPER::map).collect(Collectors.toList()));
+ List<BasicTrace> subSet = query(BasicTrace.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery traceQuery) {
+ traceQuery.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.SEGMENT_ID, segmentID));
+ }
+ });
+ basicTraces.addAll(subSet);
}
// TODO: Sort in DB with DESC
@@ -79,33 +82,35 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractDAO<BanyanDBS
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
- final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
- .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) maxSequence))
- .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) minSequence));
- query.setDataProjections(MAPPER.dataProjection());
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+ return query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) maxSequence))
+ .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) minSequence));
+ }
+ });
}
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
- final StreamQuery query = new StreamQuery(SegmentRecord.INDEX_NAME, SEGMENT_RECORD_MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.INDEX_NAME, segmentId));
- query.setDataProjections(SEGMENT_RECORD_MAPPER.dataProjection());
- StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(SEGMENT_RECORD_MAPPER::map).findFirst().orElse(null);
+ return query(SegmentRecord.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.INDEX_NAME, segmentId));
+ }
+ }).stream().findFirst().orElse(null);
}
private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) {
- final StreamQuery query = new StreamQuery(ProfileThreadSnapshotRecord.INDEX_NAME, MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
- .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, end))
- .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, start));
- query.setDataProjections(MAPPER.dataProjection());
-
- StreamQueryResponse resp = getClient().query(query);
- List<ProfileThreadSnapshotRecord> records = resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+ List<ProfileThreadSnapshotRecord> records = query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, end))
+ .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, start));
+ }
+ });
switch (aggType) {
case MIN:
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
index f5ee5fb255..be81608a37 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -21,7 +21,11 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-import org.apache.skywalking.oap.server.core.storage.*;
+import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
+import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
+import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
+import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
+import org.apache.skywalking.oap.server.core.storage.StorageDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 838e8a1eb3..bbfd4e3150 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -21,12 +21,14 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.query.type.*;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
+import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.type.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.type.Span;
+import org.apache.skywalking.oap.server.core.query.type.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.type.TraceState;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBSchema;
@@ -34,16 +36,14 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageC
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BasicTraceMapper;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.SegmentRecordMapper;
-import org.joda.time.DateTimeZone;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-import java.util.stream.Collectors;
-public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> implements ITraceQueryDAO {
+public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
private static final RowEntityMapper<SegmentRecord> SEGMENT_RECORD_MAPPER = new SegmentRecordMapper();
private static final RowEntityMapper<BasicTrace> BASIC_TRACE_MAPPER = new BasicTraceMapper();
@@ -55,110 +55,89 @@ public class BanyanDBTraceQueryDAO extends AbstractDAO<BanyanDBStorageClient> im
@Override
public TraceBrief queryBasicTraces(long startSecondTB, long endSecondTB, long minDuration, long maxDuration, String serviceId, String serviceInstanceId, String endpointId, String traceId, int limit, int from, TraceState traceState, QueryOrder queryOrder, List<Tag> tags) throws IOException {
- StreamQuery query;
- if (startSecondTB != 0 && endSecondTB != 0) {
- query = new StreamQuery(BanyanDBSchema.NAME, new TimestampRange(parseMillisFromStartSecondTB(startSecondTB),
- parseMillisFromEndSecondTB(endSecondTB)), BASIC_TRACE_MAPPER.searchableProjection());
- } else {
- query = new StreamQuery(BanyanDBSchema.NAME, BASIC_TRACE_MAPPER.searchableProjection());
- }
- if (minDuration != 0) {
- // duration >= minDuration
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", "duration", minDuration));
- }
- if (maxDuration != 0) {
- // duration <= maxDuration
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", "duration", maxDuration));
- }
+ final QueryBuilder builder = new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (minDuration != 0) {
+ // duration >= minDuration
+ query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", "duration", minDuration));
+ }
+ if (maxDuration != 0) {
+ // duration <= maxDuration
+ query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", "duration", maxDuration));
+ }
- if (!Strings.isNullOrEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_id", serviceId));
- }
+ if (!Strings.isNullOrEmpty(serviceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_id", serviceId));
+ }
- if (!Strings.isNullOrEmpty(serviceInstanceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_instance_id", serviceInstanceId));
- }
+ if (!Strings.isNullOrEmpty(serviceInstanceId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_instance_id", serviceInstanceId));
+ }
- if (!Strings.isNullOrEmpty(endpointId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "endpoint_id", endpointId));
- }
+ if (!Strings.isNullOrEmpty(endpointId)) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "endpoint_id", endpointId));
+ }
- switch (traceState) {
- case ERROR:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ERROR.getState()));
- break;
- case SUCCESS:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.SUCCESS.getState()));
- break;
- default:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ALL.getState()));
- break;
- }
+ switch (traceState) {
+ case ERROR:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ERROR.getState()));
+ break;
+ case SUCCESS:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.SUCCESS.getState()));
+ break;
+ default:
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) BanyanDBSchema.TraceState.ALL.getState()));
+ break;
+ }
- switch (queryOrder) {
- case BY_START_TIME:
- query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
- break;
- case BY_DURATION:
- query.setOrderBy(new StreamQuery.OrderBy("duration", StreamQuery.OrderBy.Type.DESC));
- break;
- }
+ switch (queryOrder) {
+ case BY_START_TIME:
+ query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
+ break;
+ case BY_DURATION:
+ query.setOrderBy(new StreamQuery.OrderBy("duration", StreamQuery.OrderBy.Type.DESC));
+ break;
+ }
- if (CollectionUtils.isNotEmpty(tags)) {
- for (final Tag tag : tags) {
- if (BanyanDBSchema.INDEX_FIELDS.contains(tag.getKey())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ if (CollectionUtils.isNotEmpty(tags)) {
+ for (final Tag tag : tags) {
+ if (BanyanDBSchema.INDEX_FIELDS.contains(tag.getKey())) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ }
+ }
}
+
+ query.setLimit(limit);
+ query.setOffset(from);
}
- }
+ };
- query.setLimit(limit);
- query.setOffset(from);
+ final List<BasicTrace> basicTraces;
+ if (startSecondTB != 0 && endSecondTB != 0) {
+ basicTraces = query(BasicTrace.class, builder, TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
+ } else {
+ basicTraces = query(BasicTrace.class, builder);
+ }
- // build request
- StreamQueryResponse response = this.getClient().query(query);
TraceBrief brief = new TraceBrief();
- brief.setTotal(response.size());
- brief.getTraces().addAll(response.getElements().stream().map(BASIC_TRACE_MAPPER::map).collect(Collectors.toList()));
+ brief.setTotal(basicTraces.size());
+ brief.getTraces().addAll(basicTraces);
return brief;
}
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
- StreamQuery query = new StreamQuery(BanyanDBSchema.NAME, SEGMENT_RECORD_MAPPER.searchableProjection());
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
- query.setDataProjections(SEGMENT_RECORD_MAPPER.dataProjection());
- StreamQueryResponse response = this.getClient().query(query);
- return response.getElements().stream().map(SEGMENT_RECORD_MAPPER::map).collect(Collectors.toList());
+ return query(SegmentRecord.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
+ }
+ });
}
@Override
public List<Span> doFlexibleTraceQuery(String traceId) throws IOException {
return Collections.emptyList();
}
-
- static long parseMillisFromStartSecondTB(long startSecondTB) {
- return YYYYMMDDHHMMSS.withZone(DateTimeZone.UTC).parseMillis(String.valueOf(startSecondTB));
- }
-
- static long parseMillisFromEndSecondTB(long endSecondTB) {
- long t = endSecondTB;
- long second = t % 100;
- if (second > 59) {
- second = 0;
- }
- t = t / 100;
- long minute = t % 100;
- if (minute > 59) {
- minute = 0;
- }
- t = t / 100;
- long hour = t % 100;
- if (hour > 23) {
- hour = 0;
- }
- t = t / 100;
- return YYYYMMDDHHMMSS.withZone(DateTimeZone.UTC)
- .parseMillis(String.valueOf(((t * 100 + hour) * 100 + minute) * 100 + second));
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index 33a69a969c..95731220fd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -2,30 +2,23 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.Tag;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
-import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.DashboardConfigurationMapper;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.RowEntityMapper;
import java.io.IOException;
import java.util.List;
-import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
*/
-public class BanyanDBUITemplateManagementDAO extends AbstractDAO<BanyanDBStorageClient> implements UITemplateManagementDAO {
- private static final RowEntityMapper<DashboardConfiguration> MAPPER = new DashboardConfigurationMapper();
-
+public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO implements UITemplateManagementDAO {
private static final long UI_TEMPLATE_TIMESTAMP = 1L;
public BanyanDBUITemplateManagementDAO(BanyanDBStorageClient client) {
@@ -34,14 +27,15 @@ public class BanyanDBUITemplateManagementDAO extends AbstractDAO<BanyanDBStorage
@Override
public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- StreamQuery query = new StreamQuery(UITemplate.INDEX_NAME, MAPPER.dataProjection());
- query.setLimit(10000);
- if (!includingDisabled) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", UITemplate.DISABLED, (long) BooleanUtils.FALSE));
- }
- query.setDataProjections(MAPPER.dataProjection());
- StreamQueryResponse resp = this.getClient().query(query);
- return resp.getElements().stream().map(MAPPER::map).collect(Collectors.toList());
+ return query(DashboardConfiguration.class, new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setLimit(10000);
+ if (!includingDisabled) {
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", UITemplate.DISABLED, (long) BooleanUtils.FALSE));
+ }
+ }
+ });
}
@Override
[skywalking] 09/24: refactor deserializer
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit a3bba672ec56a8b1bd31a2ff9928d4c1b011d214
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Wed Dec 8 16:45:12 2021 +0800
refactor deserializer
---
.../banyandb/BanyanDBStorageBuilderFactory.java | 70 +++++++
.../plugin/banyandb/BanyanDBStorageProvider.java | 2 +-
.../deserializer/AbstractBanyanDBDeserializer.java | 53 -----
.../deserializer/AlarmMessageDeserializer.java | 69 -------
.../deserializer/BanyanDBDeserializerFactory.java | 72 -------
.../deserializer/BasicTraceDeserializer.java | 49 -----
.../deserializer/BrowserErrorLogDeserializer.java | 72 -------
.../DashboardConfigurationDeserializer.java | 55 ------
.../deserializer/DatabaseDeserializer.java | 43 -----
.../deserializer/EndpointDeserializer.java | 43 -----
.../banyandb/deserializer/EventDeserializer.java | 55 ------
.../banyandb/deserializer/LogDeserializer.java | 69 -------
.../NetworkAddressAliasDeserializer.java | 53 -----
.../deserializer/ProfileTaskDeserializer.java | 54 ------
.../deserializer/ProfileTaskLogDeserializer.java | 53 -----
.../ProfileThreadSnapshotRecordDeserializer.java | 50 -----
.../deserializer/RowEntityDeserializer.java | 26 ---
.../deserializer/SegmentRecordDeserializer.java | 53 -----
.../banyandb/deserializer/ServiceDeserializer.java | 44 -----
.../deserializer/ServiceInstanceDeserializer.java | 79 --------
.../plugin/banyandb/schema/AlarmRecordBuilder.java | 2 +-
.../banyandb/schema/BanyanDBMetricsBuilder.java | 30 ---
.../banyandb/schema/BanyanDBRecordBuilder.java | 61 ------
.../schema/BanyanDBStorageDataBuilder.java | 56 +++++-
.../schema/BrowserErrorLogRecordBuilder.java | 2 +-
.../plugin/banyandb/schema/EventBuilder.java | 9 +-
.../plugin/banyandb/schema/LogRecordBuilder.java | 2 +-
.../storage/plugin/banyandb/schema/Metadata.java | 6 +-
.../schema/NetworkAddressAliasBuilder.java | 2 +-
.../schema/ProfileTaskLogRecordBuilder.java | 2 +-
.../banyandb/schema/ProfileTaskRecordBuilder.java | 2 +-
.../schema/ProfileThreadSnapshotRecordBuilder.java | 2 +-
.../banyandb/schema/SegmentRecordBuilder.java | 2 +-
.../plugin/banyandb/schema/UITemplateBuilder.java | 8 -
.../banyandb/stream/AbstractBanyanDBDAO.java | 50 +++--
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 88 +++++++--
.../stream/BanyanDBBrowserLogQueryDAO.java | 76 ++++++--
.../banyandb/stream/BanyanDBEventQueryDAO.java | 94 ++++++---
.../banyandb/stream/BanyanDBLogQueryDAO.java | 77 ++++++--
.../banyandb/stream/BanyanDBManagementDAO.java | 42 +++-
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 215 ++++++++++++++++-----
.../plugin/banyandb/stream/BanyanDBMetricsDAO.java | 14 +-
.../stream/BanyanDBNetworkAddressAliasDAO.java | 44 ++++-
.../banyandb/stream/BanyanDBNoneStreamDAO.java | 18 +-
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 44 ++++-
.../stream/BanyanDBProfileTaskQueryDAO.java | 108 +++++++----
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 166 ++++++++++++----
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 11 +-
.../plugin/banyandb/stream/BanyanDBStorageDAO.java | 97 +---------
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 96 ++++++---
.../stream/BanyanDBUITemplateManagementDAO.java | 52 +++--
51 files changed, 1036 insertions(+), 1506 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
new file mode 100644
index 0000000000..a07fe1f106
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
@@ -0,0 +1,70 @@
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
+import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.source.Event;
+import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory;
+import org.apache.skywalking.oap.server.core.storage.StorageData;
+import org.apache.skywalking.oap.server.core.storage.StorageHashMapBuilder;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BrowserErrorLogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EventBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.Metadata;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.NetworkAddressAliasBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskLogRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileThreadSnapshotRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
+
+public class BanyanDBStorageBuilderFactory implements StorageBuilderFactory {
+ @Override
+ public BuilderTemplateDefinition builderTemplate() {
+ return new BuilderTemplateDefinition(StorageHashMapBuilder.class.getName(), "metrics-builder");
+ }
+
+ @Override
+ public Class<? extends StorageBuilder> builderOf(Class<? extends StorageData> dataType, Class<? extends StorageBuilder> defaultBuilder) {
+ if (SegmentRecord.class.equals(dataType)) {
+ return SegmentRecordBuilder.class;
+ } else if (AlarmRecord.class.equals(dataType)) {
+ return AlarmRecordBuilder.class;
+ } else if (BrowserErrorLogRecord.class.equals(dataType)) {
+ return BrowserErrorLogRecordBuilder.class;
+ } else if (LogRecord.class.equals(dataType)) {
+ return LogRecordBuilder.class;
+ } else if (ProfileTaskLogRecord.class.equals(dataType)) {
+ return ProfileTaskLogRecordBuilder.class;
+ } else if (ProfileThreadSnapshotRecord.class.equals(dataType)) {
+ return ProfileThreadSnapshotRecordBuilder.class;
+ } else if (ProfileTaskRecord.class.equals(dataType)) {
+ return ProfileTaskRecordBuilder.class;
+ } else if (UITemplate.class.equals(dataType)) {
+ return UITemplateBuilder.class;
+ } else if (Event.class.equals(dataType)) {
+ return EventBuilder.class;
+ } else if (ServiceTraffic.class.equals(dataType)) {
+ return Metadata.ServiceTrafficBuilder.class;
+ } else if (InstanceTraffic.class.equals(dataType)) {
+ return Metadata.InstanceTrafficBuilder.class;
+ } else if (EndpointTraffic.class.equals(dataType)) {
+ return Metadata.EndpointTrafficBuilder.class;
+ } else if (NetworkAddressAlias.class.equals(dataType)) {
+ return NetworkAddressAliasBuilder.class;
+ }
+
+ throw new UnsupportedOperationException("unsupported storage type");
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 1df391d302..aade57b62b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -83,7 +83,7 @@ public class BanyanDBStorageProvider extends ModuleProvider {
@Override
public void prepare() throws ServiceNotProvidedException, ModuleStartException {
- this.registerServiceImplementation(StorageBuilderFactory.class, new StorageBuilderFactory.Default());
+ this.registerServiceImplementation(StorageBuilderFactory.class, new BanyanDBStorageBuilderFactory());
this.client = new BanyanDBStorageClient(config.getHost(), config.getPort(), config.getGroup());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
deleted file mode 100644
index 0b2f0644ad..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AbstractBanyanDBDeserializer.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import org.apache.skywalking.banyandb.v1.client.StreamQuery;
-import org.apache.skywalking.banyandb.v1.client.TimestampRange;
-
-import java.util.Collections;
-import java.util.List;
-
-public abstract class AbstractBanyanDBDeserializer<T> implements RowEntityDeserializer<T> {
- private final String indexName;
- private final List<String> searchableProjection;
- private final List<String> dataProjection;
-
- protected AbstractBanyanDBDeserializer(String indexName, List<String> searchableProjection) {
- this(indexName, searchableProjection, Collections.emptyList());
- }
-
- protected AbstractBanyanDBDeserializer(String indexName, List<String> searchableProjection, List<String> dataProjection) {
- this.indexName = indexName;
- this.searchableProjection = searchableProjection;
- this.dataProjection = dataProjection;
- }
-
- public StreamQuery buildStreamQuery() {
- final StreamQuery query = new StreamQuery(this.indexName, this.searchableProjection);
- query.setDataProjections(this.dataProjection);
- return query;
- }
-
- public StreamQuery buildStreamQuery(long startTimestamp, long endTimestamp) {
- final StreamQuery query = new StreamQuery(this.indexName, new TimestampRange(startTimestamp, endTimestamp), this.searchableProjection);
- query.setDataProjections(this.dataProjection);
- return query;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
deleted file mode 100644
index 2df0ef739b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.base.Charsets;
-import com.google.common.collect.ImmutableList;
-import com.google.gson.Gson;
-import com.google.gson.reflect.TypeToken;
-import com.google.protobuf.ByteString;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
-import org.apache.skywalking.oap.server.core.query.enumeration.Scope;
-import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
-import org.apache.skywalking.oap.server.core.query.type.KeyValue;
-
-import java.util.List;
-
-public class AlarmMessageDeserializer extends AbstractBanyanDBDeserializer<AlarmMessage> {
- private static final Gson GSON = new Gson();
-
- public AlarmMessageDeserializer() {
- super(AlarmRecord.INDEX_NAME,
- ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
- ImmutableList.of(AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA));
- }
-
- @Override
- public AlarmMessage map(RowEntity row) {
- AlarmMessage alarmMessage = new AlarmMessage();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- int scopeID = ((Number) searchable.get(0).getValue()).intValue();
- alarmMessage.setScopeId(scopeID);
- alarmMessage.setScope(Scope.Finder.valueOf(scopeID));
- alarmMessage.setStartTime(((Number) searchable.get(1).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- alarmMessage.setId((String) data.get(0).getValue());
- alarmMessage.setId1((String) data.get(1).getValue());
- alarmMessage.setMessage((String) data.get(2).getValue());
- Object o = data.get(3).getValue();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- this.parseDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
- }
- return alarmMessage;
- }
-
- void parseDataBinary(byte[] dataBinary, List<KeyValue> tags) {
- List<Tag> tagList = GSON.fromJson(new String(dataBinary, Charsets.UTF_8), new TypeToken<List<Tag>>() {
- }.getType());
- tagList.forEach(pair -> tags.add(new KeyValue(pair.getKey(), pair.getValue())));
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
deleted file mode 100644
index 859812530e..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BanyanDBDeserializerFactory.java
+++ /dev/null
@@ -1,72 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
-import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
-import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
-import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
-import org.apache.skywalking.oap.server.core.query.type.Database;
-import org.apache.skywalking.oap.server.core.query.type.Endpoint;
-import org.apache.skywalking.oap.server.core.query.type.Log;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
-import org.apache.skywalking.oap.server.core.query.type.Service;
-import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
-import org.apache.skywalking.oap.server.core.query.type.event.Event;
-
-import java.util.HashMap;
-import java.util.Map;
-
-public enum BanyanDBDeserializerFactory {
- INSTANCE;
-
- private final Map<Class<?>, AbstractBanyanDBDeserializer<?>> registry;
-
- BanyanDBDeserializerFactory() {
- registry = new HashMap<>(10);
- register(AlarmMessage.class, new AlarmMessageDeserializer());
- register(BasicTrace.class, new BasicTraceDeserializer());
- register(BrowserErrorLog.class, new BrowserErrorLogDeserializer());
- register(DashboardConfiguration.class, new DashboardConfigurationDeserializer());
- register(Database.class, new DatabaseDeserializer());
- register(Endpoint.class, new EndpointDeserializer());
- register(Event.class, new EventDeserializer());
- register(Log.class, new LogDeserializer());
- register(NetworkAddressAlias.class, new NetworkAddressAliasDeserializer());
- register(ProfileTaskLog.class, new ProfileTaskLogDeserializer());
- register(ProfileTask.class, new ProfileTaskDeserializer());
- register(ProfileThreadSnapshotRecord.class, new ProfileThreadSnapshotRecordDeserializer());
- register(SegmentRecord.class, new SegmentRecordDeserializer());
- register(ServiceInstance.class, new ServiceInstanceDeserializer());
- register(Service.class, new ServiceDeserializer());
- }
-
- private <T> void register(Class<T> clazz, AbstractBanyanDBDeserializer<T> mapper) {
- this.registry.put(clazz, mapper);
- }
-
- @SuppressWarnings({"unchecked"})
- public <T> AbstractBanyanDBDeserializer<T> findDeserializer(Class<T> clazz) {
- return (AbstractBanyanDBDeserializer<T>) registry.get(clazz);
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java
deleted file mode 100644
index 949ae43e16..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BasicTraceDeserializer.java
+++ /dev/null
@@ -1,49 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.IDManager;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
-
-import java.util.List;
-
-public class BasicTraceDeserializer extends AbstractBanyanDBDeserializer<BasicTrace> {
- public BasicTraceDeserializer() {
- super(SegmentRecord.INDEX_NAME, ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"));
- }
-
- @Override
- public BasicTrace map(RowEntity row) {
- BasicTrace trace = new BasicTrace();
- trace.setSegmentId(row.getId());
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- trace.getTraceIds().add((String) searchable.get(0).getValue());
- trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
- trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- (String) searchable.get(2).getValue()
- ).getEndpointName());
- trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
- trace.setStart(String.valueOf(searchable.get(4).getValue()));
- return trace;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java
deleted file mode 100644
index 92605afa67..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/BrowserErrorLogDeserializer.java
+++ /dev/null
@@ -1,72 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
-import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
-import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
-import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
-
-import java.util.Collections;
-import java.util.List;
-
-public class BrowserErrorLogDeserializer extends AbstractBanyanDBDeserializer<BrowserErrorLog> {
- public BrowserErrorLogDeserializer() {
- super(BrowserErrorLogRecord.INDEX_NAME,
- ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
- BrowserErrorLogRecord.SERVICE_VERSION_ID,
- BrowserErrorLogRecord.PAGE_PATH_ID,
- BrowserErrorLogRecord.ERROR_CATEGORY),
- Collections.singletonList(BrowserErrorLogRecord.DATA_BINARY));
- }
-
- @Override
- public BrowserErrorLog map(RowEntity row) {
- // FIXME: use protobuf directly
- BrowserErrorLog log = new BrowserErrorLog();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- log.setService((String) searchable.get(0).getValue());
- log.setServiceVersion((String) searchable.get(1).getValue());
- log.setPagePath((String) searchable.get(2).getValue());
- log.setCategory(ErrorCategory.valueOf((String) searchable.get(3).getValue()));
- log.setTime(row.getTimestamp());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- Object o = data.get(0).getValue();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- try {
- org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog browserErrorLog = org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog
- .parseFrom((ByteString) o);
- log.setGrade(browserErrorLog.getGrade());
- log.setCol(browserErrorLog.getCol());
- log.setLine(browserErrorLog.getLine());
- log.setMessage(browserErrorLog.getMessage());
- log.setErrorUrl(browserErrorLog.getErrorUrl());
- log.setStack(browserErrorLog.getStack());
- log.setFirstReportedError(browserErrorLog.getFirstReportedError());
- } catch (InvalidProtocolBufferException ex) {
- throw new RuntimeException("fail to parse proto buffer", ex);
- }
- }
- return log;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java
deleted file mode 100644
index c46db9df8d..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DashboardConfigurationDeserializer.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.query.enumeration.TemplateType;
-import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
-import org.apache.skywalking.oap.server.library.util.BooleanUtils;
-
-import java.util.List;
-
-public class DashboardConfigurationDeserializer extends AbstractBanyanDBDeserializer<DashboardConfiguration> {
- public DashboardConfigurationDeserializer() {
- super(UITemplate.INDEX_NAME,
- ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED),
- ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE));
- }
-
- @Override
- public DashboardConfiguration map(RowEntity row) {
- DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // name
- dashboardConfiguration.setName((String) searchable.get(0).getValue());
- // disabled
- dashboardConfiguration.setDisabled(BooleanUtils.valueToBoolean(((Number) searchable.get(1).getValue()).intValue()));
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // activated
- dashboardConfiguration.setActivated(BooleanUtils.valueToBoolean(((Number) data.get(0).getValue()).intValue()));
- // configuration
- dashboardConfiguration.setConfiguration((String) data.get(1).getValue());
- // type
- dashboardConfiguration.setType(TemplateType.forName((String) data.get(2).getValue()));
- return dashboardConfiguration;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java
deleted file mode 100644
index db6d875358..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/DatabaseDeserializer.java
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.query.type.Database;
-
-import java.util.List;
-
-public class DatabaseDeserializer extends AbstractBanyanDBDeserializer<Database> {
- public DatabaseDeserializer() {
- super(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE));
- }
-
- @Override
- public Database map(RowEntity row) {
- Database database = new Database();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- database.setId(row.getId());
- database.setName((String) searchable.get(0).getValue());
- return database;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java
deleted file mode 100644
index e86867c771..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EndpointDeserializer.java
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.query.type.Endpoint;
-
-import java.util.List;
-
-public class EndpointDeserializer extends AbstractBanyanDBDeserializer<Endpoint> {
- public EndpointDeserializer() {
- super(EndpointTraffic.INDEX_NAME,
- ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID));
- }
-
- @Override
- public Endpoint map(RowEntity row) {
- Endpoint endpoint = new Endpoint();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- endpoint.setName((String) searchable.get(0).getValue());
- endpoint.setId((String) searchable.get(1).getValue());
- return endpoint;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java
deleted file mode 100644
index 39783228d2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/EventDeserializer.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.query.type.event.EventType;
-import org.apache.skywalking.oap.server.core.query.type.event.Source;
-import org.apache.skywalking.oap.server.core.source.Event;
-
-import java.util.List;
-
-public class EventDeserializer extends AbstractBanyanDBDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
- public EventDeserializer() {
- super(Event.INDEX_NAME,
- ImmutableList.of(Event.UUID, Event.SERVICE, Event.SERVICE_INSTANCE, Event.ENDPOINT, Event.NAME,
- Event.TYPE, Event.START_TIME, Event.END_TIME),
- ImmutableList.of(Event.MESSAGE, Event.PARAMETERS));
- }
-
- @Override
- public org.apache.skywalking.oap.server.core.query.type.event.Event map(RowEntity row) {
- final org.apache.skywalking.oap.server.core.query.type.event.Event resultEvent = new org.apache.skywalking.oap.server.core.query.type.event.Event();
- // searchable
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- resultEvent.setUuid((String) searchable.get(0).getValue());
- resultEvent.setSource(new Source((String) searchable.get(1).getValue(), (String) searchable.get(2).getValue(), (String) searchable.get(3).getValue()));
- resultEvent.setName((String) searchable.get(4).getValue());
- resultEvent.setType(EventType.parse((String) searchable.get(5).getValue()));
- resultEvent.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- resultEvent.setEndTime(((Number) searchable.get(7).getValue()).longValue());
- // data
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- resultEvent.setMessage((String) data.get(0).getValue());
- resultEvent.setParameters((String) data.get(1).getValue());
- return resultEvent;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java
deleted file mode 100644
index f1bb3bec5d..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/LogDeserializer.java
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
-import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.apm.network.common.v3.KeyStringValuePair;
-import org.apache.skywalking.apm.network.logging.v3.LogTags;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
-import org.apache.skywalking.oap.server.core.query.type.KeyValue;
-import org.apache.skywalking.oap.server.core.query.type.Log;
-
-import java.util.List;
-
-public class LogDeserializer extends AbstractBanyanDBDeserializer<Log> {
- public LogDeserializer() {
- super(LogRecord.INDEX_NAME, ImmutableList.of(
- AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
- AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
- AbstractLogRecord.SPAN_ID),
- ImmutableList.of(AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT, AbstractLogRecord.TAGS_RAW_DATA));
- }
-
- @Override
- public Log map(RowEntity row) {
- Log log = new Log();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- log.setServiceId((String) searchable.get(0).getValue());
- log.setServiceInstanceId((String) searchable.get(1).getValue());
- log.setEndpointId((String) searchable.get(2).getValue());
- log.setTraceId((String) searchable.get(3).getValue());
- log.setTimestamp(row.getTimestamp());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- if (data.get(2).getValue() == null || ((ByteString) data.get(2).getValue()).isEmpty()) {
- log.setContent("");
- } else {
- try {
- // Don't read the tags as they have been in the data binary already.
- LogTags logTags = LogTags.parseFrom((ByteString) data.get(2).getValue());
- for (final KeyStringValuePair pair : logTags.getDataList()) {
- log.getTags().add(new KeyValue(pair.getKey(), pair.getValue()));
- }
- } catch (InvalidProtocolBufferException e) {
- throw new RuntimeException(e);
- }
- }
- return log;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java
deleted file mode 100644
index 6255a0ea40..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/NetworkAddressAliasDeserializer.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-
-import java.util.List;
-
-public class NetworkAddressAliasDeserializer extends AbstractBanyanDBDeserializer<NetworkAddressAlias> {
- public NetworkAddressAliasDeserializer() {
- super(NetworkAddressAlias.INDEX_NAME,
- ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET),
- ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id"));
- }
-
- @Override
- public NetworkAddressAlias map(RowEntity row) {
- NetworkAddressAlias model = new NetworkAddressAlias();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // searchable - last_update_time_bucket
- model.setLastUpdateTimeBucket(((Number) searchable.get(0).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // data - time_bucket
- model.setTimeBucket(((Number) data.get(0).getValue()).longValue());
- // data - address
- model.setAddress((String) data.get(1).getValue());
- // data - represent_service_id
- model.setRepresentServiceId((String) data.get(2).getValue());
- // data - represent_service_instance_id
- model.setRepresentServiceInstanceId((String) data.get(3).getValue());
- return model;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java
deleted file mode 100644
index d3eddedec6..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskDeserializer.java
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-
-import java.util.List;
-
-public class ProfileTaskDeserializer extends AbstractBanyanDBDeserializer<ProfileTask> {
- public static final String ID = "profile_task_query_id";
-
- public ProfileTaskDeserializer() {
- super(ProfileTaskRecord.INDEX_NAME,
- ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
- ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
- ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT));
- }
-
- @Override
- public ProfileTask map(RowEntity row) {
- ProfileTask profileTask = new ProfileTask();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- profileTask.setId((String) searchable.get(0).getValue());
- profileTask.setServiceId((String) searchable.get(1).getValue());
- profileTask.setEndpointName((String) searchable.get(2).getValue());
- profileTask.setStartTime(((Number) searchable.get(3).getValue()).longValue());
- profileTask.setDuration(((Number) searchable.get(4).getValue()).intValue());
- profileTask.setMinDurationThreshold(((Number) searchable.get(5).getValue()).intValue());
- profileTask.setDumpPeriod(((Number) searchable.get(6).getValue()).intValue());
- profileTask.setCreateTime(((Number) searchable.get(7).getValue()).intValue());
- profileTask.setMaxSamplingCount(((Number) searchable.get(8).getValue()).intValue());
- return null;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java
deleted file mode 100644
index 79731bf5fb..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileTaskLogDeserializer.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationType;
-
-import java.util.List;
-
-public class ProfileTaskLogDeserializer extends AbstractBanyanDBDeserializer<ProfileTaskLog> {
- public ProfileTaskLogDeserializer() {
- super(ProfileTaskLogRecord.INDEX_NAME,
- ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME),
- ImmutableList.of(ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.INSTANCE_ID,
- ProfileTaskLogRecord.OPERATION_TYPE));
- }
-
- @Override
- public ProfileTaskLog map(RowEntity row) {
- ProfileTaskLog profileTaskLog = new ProfileTaskLog();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- // searchable - operation_time
- profileTaskLog.setOperationTime(((Number) searchable.get(0).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // searchable - task_id
- profileTaskLog.setTaskId((String) data.get(0).getValue());
- // searchable - instance_id
- profileTaskLog.setInstanceId((String) data.get(1).getValue());
- // searchable - operation_type
- profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(2).getValue()).intValue()));
- return profileTaskLog;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java
deleted file mode 100644
index be815f57c0..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ProfileThreadSnapshotRecordDeserializer.java
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-
-import java.util.Collections;
-import java.util.List;
-
-public class ProfileThreadSnapshotRecordDeserializer extends AbstractBanyanDBDeserializer<ProfileThreadSnapshotRecord> {
- public ProfileThreadSnapshotRecordDeserializer() {
- super(ProfileThreadSnapshotRecord.INDEX_NAME,
- ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
- ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
- Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
- }
-
- @Override
- public ProfileThreadSnapshotRecord map(RowEntity row) {
- ProfileThreadSnapshotRecord record = new ProfileThreadSnapshotRecord();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- record.setTaskId((String) searchable.get(0).getValue());
- record.setSegmentId((String) searchable.get(1).getValue());
- record.setDumpTime(((Number) searchable.get(2).getValue()).longValue());
- record.setSequence(((Number) searchable.get(3).getValue()).intValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- record.setStackBinary(((ByteString) data.get(0).getValue()).toByteArray());
- return record;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java
deleted file mode 100644
index 2e33ed2373..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/RowEntityDeserializer.java
+++ /dev/null
@@ -1,26 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-
-@FunctionalInterface
-public interface RowEntityDeserializer<T> {
- T map(RowEntity row);
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java
deleted file mode 100644
index 0665d614af..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/SegmentRecordDeserializer.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.protobuf.ByteString;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-
-import java.util.Collections;
-import java.util.List;
-
-public class SegmentRecordDeserializer extends AbstractBanyanDBDeserializer<SegmentRecord> {
- public SegmentRecordDeserializer() {
- super(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
- Collections.singletonList("data_binary"));
- }
-
- @Override
- public SegmentRecord map(RowEntity row) {
- SegmentRecord record = new SegmentRecord();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- record.setSegmentId(row.getId());
- record.setTraceId((String) searchable.get(0).getValue());
- record.setIsError(((Number) searchable.get(1).getValue()).intValue());
- record.setServiceId((String) searchable.get(2).getValue());
- record.setServiceInstanceId((String) searchable.get(3).getValue());
- record.setEndpointId((String) searchable.get(4).getValue());
- record.setLatency(((Number) searchable.get(5).getValue()).intValue());
- record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
- return record;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java
deleted file mode 100644
index cf36fd9239..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceDeserializer.java
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.query.type.Service;
-
-import java.util.List;
-
-public class ServiceDeserializer extends AbstractBanyanDBDeserializer<Service> {
- public ServiceDeserializer() {
- super(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP));
- }
-
- @Override
- public Service map(RowEntity row) {
- Service service = new Service();
- final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- service.setId(row.getId());
- service.setName((String) searchable.get(0).getValue());
- service.setGroup((String) searchable.get(2).getValue());
- return service;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java
deleted file mode 100644
index a95867c42e..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/ServiceInstanceDeserializer.java
+++ /dev/null
@@ -1,79 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer;
-
-import com.google.common.collect.ImmutableList;
-import com.google.gson.JsonElement;
-import com.google.protobuf.ByteString;
-import com.google.protobuf.InvalidProtocolBufferException;
-import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.query.enumeration.Language;
-import org.apache.skywalking.oap.server.core.query.type.Attribute;
-import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
-import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
-
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
-
-public class ServiceInstanceDeserializer extends AbstractBanyanDBDeserializer<ServiceInstance> {
- public ServiceInstanceDeserializer() {
- super(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
- Collections.singletonList("data_binary"));
- }
-
- @Override
- public ServiceInstance map(RowEntity row) {
- InstanceTraffic instanceTraffic = new InstanceTraffic();
- final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- Object o = data.get(0).getValue();
- ServiceInstance serviceInstance = new ServiceInstance();
- if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
- try {
- RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
- instanceTraffic.deserialize(remoteData);
- serviceInstance.setName(instanceTraffic.getName());
- serviceInstance.setId(instanceTraffic.getServiceId());
-
- if (instanceTraffic.getProperties() != null) {
- for (Map.Entry<String, JsonElement> property : instanceTraffic.getProperties().entrySet()) {
- String key = property.getKey();
- String value = property.getValue().getAsString();
- if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
- serviceInstance.setLanguage(Language.value(value));
- } else {
- serviceInstance.getAttributes().add(new Attribute(key, value));
- }
- }
- } else {
- serviceInstance.setLanguage(Language.UNKNOWN);
- }
- } catch (InvalidProtocolBufferException ex) {
- throw new RuntimeException("fail to parse remote data", ex);
- }
- } else {
- throw new RuntimeException("unable to parse binary data");
- }
-
- return serviceInstance;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
index b154c9280d..a8a2863306 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/AlarmRecordBuilder.java
@@ -27,7 +27,7 @@ import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
import java.util.ArrayList;
import java.util.List;
-public class AlarmRecordBuilder extends BanyanDBRecordBuilder<AlarmRecord> {
+public class AlarmRecordBuilder extends BanyanDBStorageDataBuilder<AlarmRecord> {
public static final List<String> INDEXED_TAGS = ImmutableList.of(
"level"
);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java
deleted file mode 100644
index 28c3055f47..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBMetricsBuilder.java
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-
-public abstract class BanyanDBMetricsBuilder<T extends Metrics> extends BanyanDBStorageDataBuilder<T> {
- @Override
- protected long timestamp(Model model, T entity) {
- return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java
deleted file mode 100644
index 0310c7173b..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBRecordBuilder.java
+++ /dev/null
@@ -1,61 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
-
-import org.apache.skywalking.banyandb.v1.Banyandb;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
-import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
-import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
-import org.apache.skywalking.oap.server.core.analysis.record.Record;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-public abstract class BanyanDBRecordBuilder<T extends Record> extends BanyanDBStorageDataBuilder<T> {
- @Override
- protected long timestamp(Model model, T entity) {
- return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
- }
-
- protected List<SerializableTag<Banyandb.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
- if (rawTags == null) {
- return Collections.emptyList();
- }
- Map<String, SerializableTag<Banyandb.TagValue>> map = new HashMap<>();
- for (final Tag tag : rawTags) {
- map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
- }
- final List<SerializableTag<Banyandb.TagValue>> tags = new ArrayList<>();
- for (String indexedTag : indexTags) {
- SerializableTag<Banyandb.TagValue> tag = map.get(indexedTag);
- if (tag == null) {
- tags.add(TagAndValue.nullField());
- } else {
- tags.add(tag);
- }
- }
-
- return tags;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
index 6d545f8a9e..03ee0328f3 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BanyanDBStorageDataBuilder.java
@@ -21,29 +21,65 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import org.apache.skywalking.banyandb.v1.Banyandb;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.storage.StorageData;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
+import java.util.ArrayList;
import java.util.Collections;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
-public abstract class BanyanDBStorageDataBuilder<T extends StorageData> {
- public StreamWrite entity2Storage(Model model, T entity) {
- return StreamWrite.builder()
+public abstract class BanyanDBStorageDataBuilder<T extends StorageData> implements StorageBuilder<T, StreamWrite.StreamWriteBuilder> {
+ @Override
+ public T storage2Entity(StreamWrite.StreamWriteBuilder storageData) {
+ return null;
+ }
+
+ @Override
+ public StreamWrite.StreamWriteBuilder entity2Storage(T entity) {
+ StreamWrite.StreamWriteBuilder b = StreamWrite.builder()
.elementId(this.elementID(entity))
- .name(model.getName())
- .timestamp(this.timestamp(model, entity))
.searchableTags(this.searchableTags(entity))
- .dataTags(this.dataTags(entity))
- .build();
+ .dataTags(this.dataTags(entity));
+ Long ts = this.extractTimestamp(entity);
+ if (ts != null) {
+ b.timestamp(ts);
+ }
+ return b;
+ }
+
+ protected Long extractTimestamp(T entity) {
+ return null;
+ }
+
+ protected List<SerializableTag<Banyandb.TagValue>> filterSearchableTags(List<Tag> rawTags, List<String> indexTags) {
+ if (rawTags == null) {
+ return Collections.emptyList();
+ }
+ Map<String, SerializableTag<Banyandb.TagValue>> map = new HashMap<>();
+ for (final Tag tag : rawTags) {
+ map.put(tag.getKey().toLowerCase(), TagAndValue.stringField(tag.getValue()));
+ }
+ final List<SerializableTag<Banyandb.TagValue>> tags = new ArrayList<>();
+ for (String indexedTag : indexTags) {
+ SerializableTag<Banyandb.TagValue> tag = map.get(indexedTag);
+ if (tag == null) {
+ tags.add(TagAndValue.nullField());
+ } else {
+ tags.add(tag);
+ }
+ }
+
+ return tags;
}
protected String elementID(T entity) {
return entity.id();
}
- abstract protected long timestamp(Model model, T entity);
-
abstract protected List<SerializableTag<Banyandb.TagValue>> searchableTags(T entity);
protected List<SerializableTag<Banyandb.TagValue>> dataTags(T entity) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
index 0d1923c6f0..98b9bd0e6b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/BrowserErrorLogRecordBuilder.java
@@ -27,7 +27,7 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class BrowserErrorLogRecordBuilder extends BanyanDBRecordBuilder<BrowserErrorLogRecord> {
+public class BrowserErrorLogRecordBuilder extends BanyanDBStorageDataBuilder<BrowserErrorLogRecord> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(BrowserErrorLogRecord entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>();
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
index cce48e8b71..9a99763c73 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
@@ -4,14 +4,12 @@ import com.google.common.collect.ImmutableList;
import org.apache.skywalking.banyandb.v1.Banyandb;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
-import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.source.Event;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
import java.util.ArrayList;
import java.util.List;
-public class EventBuilder extends BanyanDBMetricsBuilder<Event> {
+public class EventBuilder extends BanyanDBStorageDataBuilder<Event> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(Event entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(8);
@@ -26,11 +24,6 @@ public class EventBuilder extends BanyanDBMetricsBuilder<Event> {
return searchable;
}
- @Override
- protected long timestamp(Model model, Event entity) {
- return TimeBucket.getTimestamp(entity.getTimeBucket(), model.getDownsampling());
- }
-
@Override
protected List<SerializableTag<Banyandb.TagValue>> dataTags(Event entity) {
return ImmutableList.of(
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
index 1d88799fd9..02809d68aa 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/LogRecordBuilder.java
@@ -27,7 +27,7 @@ import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import java.util.ArrayList;
import java.util.List;
-public class LogRecordBuilder extends BanyanDBRecordBuilder<LogRecord> {
+public class LogRecordBuilder extends BanyanDBStorageDataBuilder<LogRecord> {
public static final List<String> INDEXED_TAGS = ImmutableList.of(
"level"
);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
index 1fcd5f1587..66f3218965 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
@@ -30,7 +30,7 @@ import java.util.Collections;
import java.util.List;
public class Metadata {
- public static class ServiceTrafficBuilder extends BanyanDBMetricsBuilder<ServiceTraffic> {
+ public static class ServiceTrafficBuilder extends BanyanDBStorageDataBuilder<ServiceTraffic> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ServiceTraffic entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(3);
@@ -41,7 +41,7 @@ public class Metadata {
}
}
- public static class EndpointTrafficBuilder extends BanyanDBMetricsBuilder<EndpointTraffic> {
+ public static class EndpointTrafficBuilder extends BanyanDBStorageDataBuilder<EndpointTraffic> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(EndpointTraffic entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
@@ -51,7 +51,7 @@ public class Metadata {
}
}
- public static class InstanceTrafficBuilder extends BanyanDBMetricsBuilder<InstanceTraffic> {
+ public static class InstanceTrafficBuilder extends BanyanDBStorageDataBuilder<InstanceTraffic> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(InstanceTraffic entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(2);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
index 1054bd922f..83bf0abfab 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
@@ -27,7 +27,7 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class NetworkAddressAliasBuilder extends BanyanDBMetricsBuilder<NetworkAddressAlias> {
+public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<NetworkAddressAlias> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(NetworkAddressAlias entity) {
return Collections.singletonList(TagAndValue.longField(entity.getLastUpdateTimeBucket()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
index ee0141dc6d..888cac6b10 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
@@ -27,7 +27,7 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class ProfileTaskLogRecordBuilder extends BanyanDBRecordBuilder<ProfileTaskLogRecord> {
+public class ProfileTaskLogRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskLogRecord> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
return Collections.singletonList(TagAndValue.longField(entity.getOperationTime()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
index 0ce548ffc6..b107cadc31 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
@@ -26,7 +26,7 @@ import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
import java.util.ArrayList;
import java.util.List;
-public class ProfileTaskRecordBuilder extends BanyanDBRecordBuilder<ProfileTaskRecord> {
+public class ProfileTaskRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskRecord> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileTaskRecord entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(9);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
index 29f6a8b708..10032246ec 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileThreadSnapshotRecordBuilder.java
@@ -27,7 +27,7 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class ProfileThreadSnapshotRecordBuilder extends BanyanDBRecordBuilder<ProfileThreadSnapshotRecord> {
+public class ProfileThreadSnapshotRecordBuilder extends BanyanDBStorageDataBuilder<ProfileThreadSnapshotRecord> {
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(ProfileThreadSnapshotRecord entity) {
List<SerializableTag<Banyandb.TagValue>> searchable = new ArrayList<>(4);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
index e93335643c..a87dc9aa86 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
@@ -28,7 +28,7 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-public class SegmentRecordBuilder extends BanyanDBRecordBuilder<SegmentRecord> {
+public class SegmentRecordBuilder extends BanyanDBStorageDataBuilder<SegmentRecord> {
public static final List<String> INDEXED_TAGS = ImmutableList.of(
"http.method",
"status_code",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
index 1776e5af8d..41104ee588 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/UITemplateBuilder.java
@@ -23,19 +23,11 @@ import org.apache.skywalking.banyandb.v1.Banyandb;
import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
import java.util.ArrayList;
import java.util.List;
public class UITemplateBuilder extends BanyanDBStorageDataBuilder<UITemplate> {
- public static final long UI_TEMPLATE_TIMESTAMP = 1L;
-
- @Override
- protected long timestamp(Model model, UITemplate entity) {
- return UI_TEMPLATE_TIMESTAMP;
- }
-
@Override
protected List<SerializableTag<Banyandb.TagValue>> searchableTags(UITemplate entity) {
return ImmutableList.of(
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index 71d45dc7b6..0b89545ee4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -18,42 +18,62 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.AbstractBanyanDBDeserializer;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.BanyanDBDeserializerFactory;
import java.util.List;
-import java.util.stream.Collectors;
+import java.util.function.Function;
public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageClient> {
protected AbstractBanyanDBDAO(BanyanDBStorageClient client) {
super(client);
}
- protected <T> List<T> query(Class<T> clazz, QueryBuilder builder) {
- return this.query(clazz, builder, 0, 0);
+ protected StreamQueryResponse query(String indexName, List<String> searchableTags, QueryBuilder builder) {
+ return this.query(indexName, searchableTags, null, builder);
}
- protected <T> List<T> query(Class<T> clazz, QueryBuilder builder, long startTimestamp, long endTimestamp) {
- AbstractBanyanDBDeserializer<T> deserializer = BanyanDBDeserializerFactory.INSTANCE.findDeserializer(clazz);
-
+ protected StreamQueryResponse query(String indexName, List<String> searchableTags, TimestampRange timestampRange,
+ QueryBuilder builder) {
final StreamQuery query;
- if (startTimestamp != 0 && endTimestamp != 0) {
- query = deserializer.buildStreamQuery();
+ if (timestampRange == null) {
+ query = new StreamQuery(indexName, searchableTags);
} else {
- query = deserializer.buildStreamQuery(startTimestamp, endTimestamp);
+ query = new StreamQuery(indexName, timestampRange, searchableTags);
}
builder.apply(query);
- final StreamQueryResponse resp = getClient().query(query);
- return resp.getElements().stream().map(deserializer::map).collect(Collectors.toList());
+ return getClient().query(query);
+ }
+
+ protected abstract static class QueryBuilder {
+ protected static final String SEARCHABLE = "searchable";
+
+ abstract void apply(final StreamQuery query);
+
+ protected PairQueryCondition<Long> eq(String name, long value) {
+ return PairQueryCondition.LongQueryCondition.eq(SEARCHABLE, name, value);
+ }
+
+ protected PairQueryCondition<Long> lte(String name, long value) {
+ return PairQueryCondition.LongQueryCondition.le(SEARCHABLE, name, value);
+ }
+
+ protected PairQueryCondition<Long> gte(String name, long value) {
+ return PairQueryCondition.LongQueryCondition.ge(SEARCHABLE, name, value);
+ }
+
+ protected PairQueryCondition<String> eq(String name, String value) {
+ return PairQueryCondition.StringQueryCondition.eq(SEARCHABLE, name, value);
+ }
}
- interface QueryBuilder {
- void apply(final StreamQuery query);
+ interface RowEntityDeserializer<T> extends Function<RowEntity, T> {
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index 999d89db79..08225e51e1 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -18,13 +18,22 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.base.Charsets;
+import com.google.common.collect.ImmutableList;
+import com.google.gson.reflect.TypeToken;
+import com.google.protobuf.ByteString;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.query.enumeration.Scope;
import org.apache.skywalking.oap.server.core.query.type.AlarmMessage;
import org.apache.skywalking.oap.server.core.query.type.Alarms;
+import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
@@ -33,6 +42,7 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmReco
import java.io.IOException;
import java.util.List;
import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.alarm.AlarmRecord} is a stream,
@@ -45,34 +55,70 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
@Override
public Alarms getAlarm(Integer scopeId, String keyword, int limit, int from, long startTB, long endTB, List<Tag> tags) throws IOException {
- List<AlarmMessage> messages = query(AlarmMessage.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- if (Objects.nonNull(scopeId)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AlarmRecord.SCOPE, (long) scopeId));
- }
- if (startTB != 0 && endTB != 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(startTB)));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", AlarmRecord.START_TIME, TimeBucket.getTimestamp(endTB)));
- }
+ TimestampRange tsRange = null;
+ if (startTB > 0 && endTB > 0) {
+ tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
+ }
- // TODO: support keyword search
+ StreamQueryResponse resp = query(AlarmRecord.INDEX_NAME,
+ ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
+ tsRange,
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA));
- if (CollectionUtils.isNotEmpty(tags)) {
- for (final Tag tag : tags) {
- if (AlarmRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ if (Objects.nonNull(scopeId)) {
+ query.appendCondition(eq(AlarmRecord.SCOPE, (long) scopeId));
}
+
+ // TODO: support keyword search
+
+ if (CollectionUtils.isNotEmpty(tags)) {
+ for (final Tag tag : tags) {
+ if (AlarmRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
+ }
+ }
+ }
+ query.setLimit(limit);
+ query.setOffset(from);
}
- }
- query.setLimit(limit);
- query.setOffset(from);
- }
- });
+ });
+
+ List<AlarmMessage> messages = resp.getElements().stream().map(new AlarmMessageDeserializer())
+ .collect(Collectors.toList());
Alarms alarms = new Alarms();
alarms.setTotal(messages.size());
alarms.getMsgs().addAll(messages);
return alarms;
}
+
+ public static class AlarmMessageDeserializer implements RowEntityDeserializer<AlarmMessage> {
+ @Override
+ public AlarmMessage apply(RowEntity row) {
+ AlarmMessage alarmMessage = new AlarmMessage();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ int scopeID = ((Number) searchable.get(0).getValue()).intValue();
+ alarmMessage.setScopeId(scopeID);
+ alarmMessage.setScope(Scope.Finder.valueOf(scopeID));
+ alarmMessage.setStartTime(((Number) searchable.get(1).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ alarmMessage.setId((String) data.get(0).getValue());
+ alarmMessage.setId1((String) data.get(1).getValue());
+ alarmMessage.setMessage((String) data.get(2).getValue());
+ Object o = data.get(3).getValue();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ this.parseDataBinary(((ByteString) o).toByteArray(), alarmMessage.getTags());
+ }
+ return alarmMessage;
+ }
+
+ void parseDataBinary(byte[] dataBinary, List<KeyValue> tags) {
+ List<Tag> tagList = GSON.fromJson(new String(dataBinary, Charsets.UTF_8), new TypeToken<List<Tag>>() {
+ }.getType());
+ tagList.forEach(pair -> tags.add(new KeyValue(pair.getKey(), pair.getValue())));
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index 803b05e255..c1bbc16941 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -18,20 +18,29 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
import org.apache.skywalking.oap.server.core.browser.source.BrowserErrorCategory;
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLog;
import org.apache.skywalking.oap.server.core.query.type.BrowserErrorLogs;
+import org.apache.skywalking.oap.server.core.query.type.ErrorCategory;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.io.IOException;
+import java.util.Collections;
import java.util.List;
import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
@@ -43,36 +52,71 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
@Override
public BrowserErrorLogs queryBrowserErrorLogs(String serviceId, String serviceVersionId, String pagePathId, BrowserErrorCategory category, long startSecondTB, long endSecondTB, int limit, int from) throws IOException {
+ TimestampRange tsRange = null;
+ if (startSecondTB > 0 && endSecondTB > 0) {
+ tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
+ }
- final QueryBuilder qb = new QueryBuilder() {
+ final BrowserErrorLogs logs = new BrowserErrorLogs();
+ StreamQueryResponse resp = query(BrowserErrorLogRecord.INDEX_NAME, ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
+ BrowserErrorLogRecord.SERVICE_VERSION_ID,
+ BrowserErrorLogRecord.PAGE_PATH_ID,
+ BrowserErrorLogRecord.ERROR_CATEGORY), tsRange, new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_ID, serviceId));
+ query.setDataProjections(Collections.singletonList(BrowserErrorLogRecord.DATA_BINARY));
+ query.appendCondition(eq(BrowserErrorLogRecord.SERVICE_ID, serviceId));
if (StringUtil.isNotEmpty(serviceVersionId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
+ query.appendCondition(eq(BrowserErrorLogRecord.SERVICE_VERSION_ID, serviceVersionId));
}
+
if (StringUtil.isNotEmpty(pagePathId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", BrowserErrorLogRecord.PAGE_PATH_ID, pagePathId));
+ query.appendCondition(eq(BrowserErrorLogRecord.PAGE_PATH_ID, pagePathId));
}
+
if (Objects.nonNull(category)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", BrowserErrorLogRecord.ERROR_CATEGORY, (long) category.getValue()));
+ query.appendCondition(eq(BrowserErrorLogRecord.ERROR_CATEGORY, category.getValue()));
}
query.setOffset(from);
query.setLimit(limit);
}
- };
-
- final BrowserErrorLogs logs = new BrowserErrorLogs();
- final List<BrowserErrorLog> browserErrorLogs;
- if (startSecondTB != 0 && endSecondTB != 0) {
- browserErrorLogs = query(BrowserErrorLog.class, qb, TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
- } else {
- browserErrorLogs = query(BrowserErrorLog.class, qb);
- }
- logs.getLogs().addAll(browserErrorLogs);
+ });
+ logs.getLogs().addAll(resp.getElements().stream().map(new BrowserErrorLogDeserializer()).collect(Collectors.toList()));
logs.setTotal(logs.getLogs().size());
return logs;
}
+
+ public static class BrowserErrorLogDeserializer implements RowEntityDeserializer<BrowserErrorLog> {
+ @Override
+ public BrowserErrorLog apply(RowEntity row) {
+ // FIXME: use protobuf directly
+ BrowserErrorLog log = new BrowserErrorLog();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ log.setService((String) searchable.get(0).getValue());
+ log.setServiceVersion((String) searchable.get(1).getValue());
+ log.setPagePath((String) searchable.get(2).getValue());
+ log.setCategory(ErrorCategory.valueOf((String) searchable.get(3).getValue()));
+ log.setTime(row.getTimestamp());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ Object o = data.get(0).getValue();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ try {
+ org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog browserErrorLog = org.apache.skywalking.apm.network.language.agent.v3.BrowserErrorLog
+ .parseFrom((ByteString) o);
+ log.setGrade(browserErrorLog.getGrade());
+ log.setCol(browserErrorLog.getCol());
+ log.setLine(browserErrorLog.getLine());
+ log.setMessage(browserErrorLog.getMessage());
+ log.setErrorUrl(browserErrorLog.getErrorUrl());
+ log.setStack(browserErrorLog.getStack());
+ log.setFirstReportedError(browserErrorLog.getFirstReportedError());
+ } catch (InvalidProtocolBufferException ex) {
+ throw new RuntimeException("fail to parse proto buffer", ex);
+ }
+ }
+ return log;
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
index b85e07d860..44299c38e9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEventQueryDAO.java
@@ -19,11 +19,15 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.query.PaginationUtils;
import org.apache.skywalking.oap.server.core.query.input.Duration;
import org.apache.skywalking.oap.server.core.query.type.event.EventQueryCondition;
+import org.apache.skywalking.oap.server.core.query.type.event.EventType;
import org.apache.skywalking.oap.server.core.query.type.event.Events;
import org.apache.skywalking.oap.server.core.query.type.event.Source;
import org.apache.skywalking.oap.server.core.source.Event;
@@ -31,6 +35,7 @@ import org.apache.skywalking.oap.server.core.storage.query.IEventQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.source.Event} is a stream
@@ -42,10 +47,13 @@ public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEvent
@Override
public Events queryEvents(EventQueryCondition condition) throws Exception {
- List<org.apache.skywalking.oap.server.core.query.type.event.Event> eventList = query(org.apache.skywalking.oap.server.core.query.type.event.Event.class,
+ StreamQueryResponse resp = query(Event.INDEX_NAME,
+ ImmutableList.of(Event.UUID, Event.SERVICE, Event.SERVICE_INSTANCE, Event.ENDPOINT, Event.NAME, Event.TYPE, Event.START_TIME, Event.END_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(Event.MESSAGE, Event.PARAMETERS));
+
buildConditions(condition, query);
PaginationUtils.Page page = PaginationUtils.INSTANCE.exchange(condition.getPaging());
@@ -59,8 +67,43 @@ public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEvent
query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
}
}
+
+ private void buildConditions(EventQueryCondition condition, final StreamQuery query) {
+ if (!Strings.isNullOrEmpty(condition.getUuid())) {
+ query.appendCondition(eq(Event.UUID, condition.getUuid()));
+ }
+ final Source source = condition.getSource();
+ if (source != null) {
+ if (!Strings.isNullOrEmpty(source.getService())) {
+ query.appendCondition(eq(Event.SERVICE, source.getService()));
+ }
+ if (!Strings.isNullOrEmpty(source.getServiceInstance())) {
+ query.appendCondition(eq(Event.SERVICE_INSTANCE, source.getServiceInstance()));
+ }
+ if (!Strings.isNullOrEmpty(source.getEndpoint())) {
+ query.appendCondition(eq(Event.ENDPOINT, source.getEndpoint()));
+ }
+ }
+ if (!Strings.isNullOrEmpty(condition.getName())) {
+ query.appendCondition(eq(Event.NAME, condition.getName()));
+ }
+ if (condition.getType() != null) {
+ query.appendCondition(eq(Event.TYPE, condition.getType().name()));
+ }
+ final Duration time = condition.getTime();
+ if (time != null) {
+ if (time.getStartTimestamp() > 0) {
+ query.appendCondition(gte(Event.START_TIME, time.getStartTimestamp()));
+ }
+ if (time.getEndTimestamp() > 0) {
+ query.appendCondition(lte(Event.END_TIME, time.getEndTimestamp()));
+ }
+ }
+ }
});
+ List<org.apache.skywalking.oap.server.core.query.type.event.Event> eventList = resp.getElements().stream().map(new EventDeserializer()).collect(Collectors.toList());
+
Events events = new Events();
events.setEvents(eventList);
events.setTotal(eventList.size());
@@ -83,36 +126,23 @@ public class BanyanDBEventQueryDAO extends AbstractBanyanDBDAO implements IEvent
return events;
}
- private void buildConditions(EventQueryCondition condition, final StreamQuery query) {
- if (!Strings.isNullOrEmpty(condition.getUuid())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.UUID, condition.getUuid()));
- }
- final Source source = condition.getSource();
- if (source != null) {
- if (!Strings.isNullOrEmpty(source.getService())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.SERVICE, source.getService()));
- }
- if (!Strings.isNullOrEmpty(source.getServiceInstance())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.SERVICE_INSTANCE, source.getServiceInstance()));
- }
- if (!Strings.isNullOrEmpty(source.getEndpoint())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.ENDPOINT, source.getEndpoint()));
- }
- }
- if (!Strings.isNullOrEmpty(condition.getName())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.NAME, condition.getName()));
- }
- if (condition.getType() != null) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", Event.TYPE, condition.getType().name()));
- }
- final Duration time = condition.getTime();
- if (time != null) {
- if (time.getStartTimestamp() > 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.gt("searchable", Event.START_TIME, time.getStartTimestamp()));
- }
- if (time.getEndTimestamp() > 0) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.gt("searchable", Event.END_TIME, time.getEndTimestamp()));
- }
+ public static class EventDeserializer implements RowEntityDeserializer<org.apache.skywalking.oap.server.core.query.type.event.Event> {
+ @Override
+ public org.apache.skywalking.oap.server.core.query.type.event.Event apply(RowEntity row) {
+ final org.apache.skywalking.oap.server.core.query.type.event.Event resultEvent = new org.apache.skywalking.oap.server.core.query.type.event.Event();
+ // searchable
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ resultEvent.setUuid((String) searchable.get(0).getValue());
+ resultEvent.setSource(new Source((String) searchable.get(1).getValue(), (String) searchable.get(2).getValue(), (String) searchable.get(3).getValue()));
+ resultEvent.setName((String) searchable.get(4).getValue());
+ resultEvent.setType(EventType.parse((String) searchable.get(5).getValue()));
+ resultEvent.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ resultEvent.setEndTime(((Number) searchable.get(7).getValue()).longValue());
+ // data
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ resultEvent.setMessage((String) data.get(0).getValue());
+ resultEvent.setParameters((String) data.get(1).getValue());
+ return resultEvent;
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index 8325618f88..6bce5767e7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -18,13 +18,23 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
+import org.apache.skywalking.apm.network.common.v3.KeyStringValuePair;
+import org.apache.skywalking.apm.network.logging.v3.LogTags;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
+import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.query.enumeration.Order;
import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import org.apache.skywalking.oap.server.core.query.type.Log;
import org.apache.skywalking.oap.server.core.query.type.Logs;
import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
@@ -36,6 +46,7 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecord
import java.io.IOException;
import java.util.List;
import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
@@ -53,48 +64,84 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
final QueryBuilder query = new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT, AbstractLogRecord.TAGS_RAW_DATA));
+
if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_ID, serviceId));
+ query.appendCondition(eq(AbstractLogRecord.SERVICE_ID, serviceId));
}
if (StringUtil.isNotEmpty(serviceInstanceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
+ query.appendCondition(eq(AbstractLogRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
}
if (StringUtil.isNotEmpty(endpointId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.ENDPOINT_ID, endpointId));
+ query.appendCondition(eq(AbstractLogRecord.ENDPOINT_ID, endpointId));
}
if (Objects.nonNull(relatedTrace)) {
if (StringUtil.isNotEmpty(relatedTrace.getTraceId())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_ID, relatedTrace.getTraceId()));
+ query.appendCondition(eq(AbstractLogRecord.TRACE_ID, relatedTrace.getTraceId()));
}
if (StringUtil.isNotEmpty(relatedTrace.getSegmentId())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", AbstractLogRecord.TRACE_SEGMENT_ID, relatedTrace.getSegmentId()));
+ query.appendCondition(eq(AbstractLogRecord.TRACE_SEGMENT_ID, relatedTrace.getSegmentId()));
}
if (Objects.nonNull(relatedTrace.getSpanId())) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", AbstractLogRecord.SPAN_ID, (long) relatedTrace.getSpanId()));
+ query.appendCondition(eq(AbstractLogRecord.SPAN_ID, (long) relatedTrace.getSpanId()));
}
}
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
if (LogRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
}
}
}
}
};
- final List<Log> entities;
- if (startTB != 0 && endTB != 0) {
- entities = query(Log.class, query, TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
- } else {
- entities = query(Log.class, query);
+ TimestampRange tsRange = null;
+ if (startTB > 0 && endTB > 0) {
+ tsRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
}
+
+ StreamQueryResponse resp = query(LogRecord.INDEX_NAME,
+ ImmutableList.of(AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
+ AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
+ AbstractLogRecord.SPAN_ID), tsRange, query);
+
+ List<Log> logEntities = resp.getElements().stream().map(new LogDeserializer()).collect(Collectors.toList());
+
Logs logs = new Logs();
- logs.getLogs().addAll(entities);
- logs.setTotal(entities.size());
+ logs.getLogs().addAll(logEntities);
+ logs.setTotal(logEntities.size());
return logs;
}
+
+ public static class LogDeserializer implements RowEntityDeserializer<Log> {
+ @Override
+ public Log apply(RowEntity row) {
+ Log log = new Log();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ log.setServiceId((String) searchable.get(0).getValue());
+ log.setServiceInstanceId((String) searchable.get(1).getValue());
+ log.setEndpointId((String) searchable.get(2).getValue());
+ log.setTraceId((String) searchable.get(3).getValue());
+ log.setTimestamp(row.getTimestamp());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ if (data.get(2).getValue() == null || ((ByteString) data.get(2).getValue()).isEmpty()) {
+ log.setContent("");
+ } else {
+ try {
+ // Don't read the tags as they have been in the data binary already.
+ LogTags logTags = LogTags.parseFrom((ByteString) data.get(2).getValue());
+ for (final KeyStringValuePair pair : logTags.getDataList()) {
+ log.getTags().add(new KeyValue(pair.getKey(), pair.getValue()));
+ }
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ return log;
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
index aecdd13dfd..7e37288428 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
@@ -18,24 +18,54 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
+import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
import java.io.IOException;
+import java.util.Collections;
-@RequiredArgsConstructor
-public class BanyanDBManagementDAO<T extends ManagementData> implements IManagementDAO {
- private final BanyanDBStorageClient client;
+/**
+ * UITemplate insertion DAO
+ *
+ * @param <T> The only ManagementData we have now is {@link UITemplate}
+ */
+public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBanyanDBDAO implements IManagementDAO {
private final BanyanDBStorageDataBuilder<T> storageBuilder;
+ public BanyanDBManagementDAO(BanyanDBStorageClient client, BanyanDBStorageDataBuilder<T> storageBuilder) {
+ super(client);
+ this.storageBuilder = storageBuilder;
+ }
+
@Override
public void insert(Model model, ManagementData storageData) throws IOException {
- StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) storageData);
- client.write(streamWrite);
+ // ensure only insert once
+ StreamQueryResponse resp = query(UITemplate.INDEX_NAME,
+ Collections.singletonList(UITemplate.NAME),
+ new TimestampRange(0L, 2L),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(UITemplate.NAME, storageData.id()));
+ }
+ });
+
+ if (resp != null && resp.getElements().size() > 0) {
+ return;
+ }
+
+ StreamWrite.StreamWriteBuilder streamWrite = this.storageBuilder
+ .entity2Storage((T) storageData)
+ .name(model.getName())
+ .timestamp(1L);
+ getClient().write(streamWrite.build());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index b3b6fece70..fdc1ccb71f 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -18,23 +18,35 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import com.google.common.collect.ImmutableList;
+import com.google.gson.JsonElement;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.InvalidProtocolBufferException;
import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.NodeType;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
+import org.apache.skywalking.oap.server.core.query.enumeration.Language;
+import org.apache.skywalking.oap.server.core.query.type.Attribute;
import org.apache.skywalking.oap.server.core.query.type.Database;
import org.apache.skywalking.oap.server.core.query.type.Endpoint;
import org.apache.skywalking.oap.server.core.query.type.Service;
import org.apache.skywalking.oap.server.core.query.type.ServiceInstance;
+import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
import org.apache.skywalking.oap.server.core.storage.query.IMetadataQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.io.IOException;
+import java.util.Collections;
import java.util.List;
+import java.util.Map;
import java.util.stream.Collectors;
/**
@@ -50,83 +62,184 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
@Override
public List<Service> getAllServices(String group) throws IOException {
- return query(Service.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- if (StringUtil.isNotEmpty(group)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.GROUP, group));
- }
- }
- });
+ StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(group)) {
+ query.appendCondition(eq(ServiceTraffic.GROUP, group));
+ }
+ }
+ });
+
+ return resp.getElements().stream().map(new ServiceDeserializer()).collect(Collectors.toList());
}
@Override
public List<Service> getAllBrowserServices() throws IOException {
- return query(Service.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) NodeType.Browser.value()));
- }
- });
+ StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(ServiceTraffic.NODE_TYPE, NodeType.Browser.value()));
+ }
+ });
+
+ return resp.getElements().stream().map(new ServiceDeserializer()).collect(Collectors.toList());
}
@Override
public List<Database> getAllDatabases() throws IOException {
- return query(Database.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) NodeType.Database.value()));
- }
- });
+ StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME, ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(ServiceTraffic.NODE_TYPE, (long) NodeType.Database.value()));
+ }
+ });
+
+ return resp.getElements().stream().map(new DatabaseDeserializer()).collect(Collectors.toList());
}
@Override
public List<Service> searchServices(NodeType nodeType, String keyword) throws IOException {
- return query(Service.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
- }
- }).stream().filter(s -> s.getName().contains(keyword)) // TODO: support analyzer in database
+ StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(ServiceTraffic.NODE_TYPE, nodeType.value()));
+ }
+ });
+
+ return resp.getElements().stream().map(new ServiceDeserializer())
+ .filter(s -> s.getName().contains(keyword)) // TODO: support analyzer in database
.collect(Collectors.toList());
}
@Override
public Service searchService(NodeType nodeType, String serviceCode) throws IOException {
- return query(Service.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.NAME, serviceCode));
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
- // only get one
- query.setLimit(1);
- }
- }).stream().findAny().orElse(null);
+ StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.NODE_TYPE, ServiceTraffic.GROUP),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ServiceTraffic.NAME, serviceCode));
+ query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
+ // only get one
+ query.setLimit(1);
+ }
+ });
+
+ return resp.getElements().stream().map(new ServiceDeserializer()).findAny().orElse(null);
}
@Override
public List<Endpoint> searchEndpoint(String keyword, String serviceId, int limit) throws IOException {
- return query(Endpoint.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", EndpointTraffic.SERVICE_ID, serviceId));
- }
- }).stream().filter(e -> e.getName().contains(keyword))
+ StreamQueryResponse resp = query(EndpointTraffic.INDEX_NAME,
+ ImmutableList.of(EndpointTraffic.NAME, EndpointTraffic.SERVICE_ID), new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(EndpointTraffic.SERVICE_ID, serviceId));
+ }
+ });
+
+ return resp.getElements().stream().map(new EndpointDeserializer()).filter(e -> e.getName().contains(keyword))
.limit(limit).collect(Collectors.toList());
}
@Override
public List<ServiceInstance> getServiceInstances(long startTimestamp, long endTimestamp, String serviceId) throws IOException {
- return query(ServiceInstance.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
- final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
-
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", InstanceTraffic.SERVICE_ID, serviceId));
+ StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
+ ImmutableList.of(InstanceTraffic.SERVICE_ID, InstanceTraffic.LAST_PING_TIME_BUCKET),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList("data_binary"));
+
+ final long startMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(startTimestamp);
+ final long endMinuteTimeBucket = TimeBucket.getMinuteTimeBucket(endTimestamp);
+
+ query.appendCondition(gte(InstanceTraffic.LAST_PING_TIME_BUCKET, startMinuteTimeBucket));
+ query.appendCondition(lte(InstanceTraffic.LAST_PING_TIME_BUCKET, endMinuteTimeBucket));
+ query.appendCondition(eq(InstanceTraffic.SERVICE_ID, serviceId));
+ }
+ });
+
+ return resp.getElements().stream().map(new ServiceInstanceDeserializer()).collect(Collectors.toList());
+ }
+
+ public static class DatabaseDeserializer implements RowEntityDeserializer<Database> {
+ @Override
+ public Database apply(RowEntity row) {
+ Database database = new Database();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ database.setId(row.getId());
+ database.setName((String) searchable.get(0).getValue());
+ return database;
+ }
+ }
+
+ public static class EndpointDeserializer implements RowEntityDeserializer<Endpoint> {
+ @Override
+ public Endpoint apply(RowEntity row) {
+ Endpoint endpoint = new Endpoint();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ endpoint.setName((String) searchable.get(0).getValue());
+ endpoint.setId((String) searchable.get(1).getValue());
+ return endpoint;
+ }
+ }
+
+ public static class ServiceDeserializer implements RowEntityDeserializer<Service> {
+ @Override
+ public Service apply(RowEntity row) {
+ Service service = new Service();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ service.setId(row.getId());
+ service.setName((String) searchable.get(0).getValue());
+ service.setGroup((String) searchable.get(2).getValue());
+ return service;
+ }
+ }
+
+ public static class ServiceInstanceDeserializer implements RowEntityDeserializer<ServiceInstance> {
+ @Override
+ public ServiceInstance apply(RowEntity row) {
+ InstanceTraffic instanceTraffic = new InstanceTraffic();
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ Object o = data.get(0).getValue();
+ ServiceInstance serviceInstance = new ServiceInstance();
+ if (o instanceof ByteString && !((ByteString) o).isEmpty()) {
+ try {
+ RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
+ instanceTraffic.deserialize(remoteData);
+ serviceInstance.setName(instanceTraffic.getName());
+ serviceInstance.setId(instanceTraffic.getServiceId());
+
+ if (instanceTraffic.getProperties() != null) {
+ for (Map.Entry<String, JsonElement> property : instanceTraffic.getProperties().entrySet()) {
+ String key = property.getKey();
+ String value = property.getValue().getAsString();
+ if (key.equals(InstanceTraffic.PropertyUtil.LANGUAGE)) {
+ serviceInstance.setLanguage(Language.value(value));
+ } else {
+ serviceInstance.getAttributes().add(new Attribute(key, value));
+ }
+ }
+ } else {
+ serviceInstance.setLanguage(Language.UNKNOWN);
+ }
+ } catch (InvalidProtocolBufferException ex) {
+ throw new RuntimeException("fail to parse remote data", ex);
+ }
+ } else {
+ throw new RuntimeException("unable to parse binary data");
}
- });
+
+ return serviceInstance;
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
index 618ff86af9..5aa1b7657d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
@@ -20,12 +20,13 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBMetricsBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
import java.io.IOException;
import java.util.Collections;
@@ -33,7 +34,7 @@ import java.util.List;
@RequiredArgsConstructor
public class BanyanDBMetricsDAO<T extends Metrics> implements IMetricsDAO {
- private final BanyanDBMetricsBuilder<T> storageBuilder;
+ private final BanyanDBStorageDataBuilder<T> storageBuilder;
@Override
public List<Metrics> multiGet(Model model, List<Metrics> metrics) throws IOException {
@@ -42,12 +43,15 @@ public class BanyanDBMetricsDAO<T extends Metrics> implements IMetricsDAO {
@Override
public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
- StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) metrics);
- return new BanyanDBStreamInsertRequest(streamWrite);
+ StreamWrite.StreamWriteBuilder builder = this.storageBuilder.entity2Storage((T) metrics)
+ .name(model.getName())
+ .timestamp(TimeBucket.getTimeBucket(metrics.getTimeBucket(), model.getDownsampling()));
+ return new BanyanDBStreamInsertRequest(builder.build());
}
@Override
public UpdateRequest prepareBatchUpdate(Model model, Metrics metrics) throws IOException {
- return null;
+ return new UpdateRequest() {
+ };
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
index ce36573617..33aec37bac 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -18,13 +18,18 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link NetworkAddressAlias} is a stream
@@ -36,11 +41,36 @@ public class BanyanDBNetworkAddressAliasDAO extends AbstractBanyanDBDAO implemen
@Override
public List<NetworkAddressAlias> loadLastUpdate(long timeBucket) {
- return query(NetworkAddressAlias.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
- }
- });
+ StreamQueryResponse resp = query(NetworkAddressAlias.INDEX_NAME,
+ ImmutableList.of(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(Metrics.TIME_BUCKET, "address", "represent_service_id", "represent_service_instance_id"));
+ query.appendCondition(gte(NetworkAddressAlias.LAST_UPDATE_TIME_BUCKET, timeBucket));
+ }
+ });
+
+ return resp.getElements().stream().map(new NetworkAddressAliasDeserializer()).collect(Collectors.toList());
+ }
+
+ public static class NetworkAddressAliasDeserializer implements RowEntityDeserializer<NetworkAddressAlias> {
+ @Override
+ public NetworkAddressAlias apply(RowEntity row) {
+ NetworkAddressAlias model = new NetworkAddressAlias();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // searchable - last_update_time_bucket
+ model.setLastUpdateTimeBucket(((Number) searchable.get(0).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // data - time_bucket
+ model.setTimeBucket(((Number) data.get(0).getValue()).longValue());
+ // data - address
+ model.setAddress((String) data.get(1).getValue());
+ // data - represent_service_id
+ model.setRepresentServiceId((String) data.get(2).getValue());
+ // data - represent_service_instance_id
+ model.setRepresentServiceInstanceId((String) data.get(3).getValue());
+ return model;
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
index 30ed279432..e19ff9af98 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
@@ -20,22 +20,32 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
import org.apache.skywalking.oap.server.core.storage.INoneStreamDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
import java.io.IOException;
+/**
+ * DAO for NoneStream, specifically ProfileTaskRecord
+ *
+ * @param <T> For NoneStream, we only have {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord}
+ */
@RequiredArgsConstructor
public class BanyanDBNoneStreamDAO<T extends NoneStream> implements INoneStreamDAO {
private final BanyanDBStorageClient client;
- private final BanyanDBRecordBuilder<T> storageBuilder;
+ private final BanyanDBStorageDataBuilder<T> storageBuilder;
@Override
public void insert(Model model, NoneStream noneStream) throws IOException {
- StreamWrite streamWrite = this.storageBuilder.entity2Storage(model, (T) noneStream);
- this.client.write(streamWrite);
+ final long timestamp = TimeBucket.getTimeBucket(noneStream.getTimeBucket(), model.getDownsampling());
+ StreamWrite.StreamWriteBuilder builder =
+ this.storageBuilder.entity2Storage((T) noneStream)
+ .name(model.getName())
+ .timestamp(timestamp);
+ this.client.write(builder.build());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 3a3c12e119..3a4910f3b5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -18,8 +18,14 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
+import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationType;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
@@ -41,12 +47,38 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
- return query(ProfileTaskLog.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
- }
- }).stream().sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
+ StreamQueryResponse resp = query(ProfileTaskLogRecord.INDEX_NAME,
+ ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(ProfileTaskLogRecord.TASK_ID,
+ ProfileTaskLogRecord.INSTANCE_ID,
+ ProfileTaskLogRecord.OPERATION_TYPE));
+ query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
+ }
+ });
+
+ return resp.getElements().stream().map(new ProfileTaskLogDeserializer())
+ .sorted(Comparator.comparingLong(ProfileTaskLog::getOperationTime))
.collect(Collectors.toList());
}
+
+ public static class ProfileTaskLogDeserializer implements RowEntityDeserializer<ProfileTaskLog> {
+ @Override
+ public ProfileTaskLog apply(RowEntity row) {
+ ProfileTaskLog profileTaskLog = new ProfileTaskLog();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // searchable - operation_time
+ profileTaskLog.setOperationTime(((Number) searchable.get(0).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // searchable - task_id
+ profileTaskLog.setTaskId((String) data.get(0).getValue());
+ // searchable - instance_id
+ profileTaskLog.setInstanceId((String) data.get(1).getValue());
+ // searchable - operation_type
+ profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(2).getValue()).intValue()));
+ return profileTaskLog;
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 47c7c57a00..1cbaea95af 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -18,60 +18,66 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.deserializer.ProfileTaskDeserializer;
import java.io.IOException;
import java.util.List;
import java.util.Objects;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
*/
public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskQueryDAO {
+ public static final String ID = "profile_task_query_id";
+
public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@Override
public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
- return query(ProfileTask.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- if (StringUtil.isNotEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
- ProfileTaskRecord.SERVICE_ID, serviceId));
- }
-
- if (StringUtil.isNotEmpty(endpointName)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable",
- ProfileTaskRecord.ENDPOINT_NAME, endpointName));
- }
-
- if (Objects.nonNull(startTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable",
- ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
- }
-
- if (Objects.nonNull(endTimeBucket)) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable",
- ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
- }
-
- if (Objects.nonNull(limit)) {
- query.setLimit(limit);
- }
-
- query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
- }
- });
+ StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
+ ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
+ ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT), new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ if (StringUtil.isNotEmpty(serviceId)) {
+ query.appendCondition(eq(ProfileTaskRecord.SERVICE_ID, serviceId));
+ }
+
+ if (StringUtil.isNotEmpty(endpointName)) {
+ query.appendCondition(eq(ProfileTaskRecord.ENDPOINT_NAME, endpointName));
+ }
+
+ if (Objects.nonNull(startTimeBucket)) {
+ query.appendCondition(gte(ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(startTimeBucket)));
+ }
+
+ if (Objects.nonNull(endTimeBucket)) {
+ query.appendCondition(lte(ProfileTaskRecord.START_TIME, TimeBucket.getTimestamp(endTimeBucket)));
+ }
+
+ if (Objects.nonNull(limit)) {
+ query.setLimit(limit);
+ }
+
+ query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
+ }
+ });
+
+ return resp.getElements().stream().map(new ProfileTaskDeserializer()).collect(Collectors.toList());
}
@Override
@@ -80,12 +86,36 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements
return null;
}
- return query(ProfileTask.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileTaskDeserializer.ID, id));
- query.setLimit(1);
- }
- }).stream().findAny().orElse(null);
+ StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
+ ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
+ ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(ID, id));
+ query.setLimit(1);
+ }
+ });
+
+ return resp.getElements().stream().map(new ProfileTaskDeserializer()).findAny().orElse(null);
+ }
+
+ public static class ProfileTaskDeserializer implements RowEntityDeserializer<ProfileTask> {
+ @Override
+ public ProfileTask apply(RowEntity row) {
+ ProfileTask profileTask = new ProfileTask();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ profileTask.setId((String) searchable.get(0).getValue());
+ profileTask.setServiceId((String) searchable.get(1).getValue());
+ profileTask.setEndpointName((String) searchable.get(2).getValue());
+ profileTask.setStartTime(((Number) searchable.get(3).getValue()).longValue());
+ profileTask.setDuration(((Number) searchable.get(4).getValue()).intValue());
+ profileTask.setMinDurationThreshold(((Number) searchable.get(5).getValue()).intValue());
+ profileTask.setDumpPeriod(((Number) searchable.get(6).getValue()).intValue());
+ profileTask.setCreateTime(((Number) searchable.get(7).getValue()).intValue());
+ profileTask.setMaxSamplingCount(((Number) searchable.get(8).getValue()).intValue());
+ return profileTask;
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index c6b9cdec9f..4da87ac9e5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -18,8 +18,13 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
@@ -27,9 +32,9 @@ import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnaps
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
-import java.util.LinkedList;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Collectors;
@@ -44,30 +49,37 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
- List<ProfileThreadSnapshotRecord> resp = query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.TASK_ID, taskId))
- .appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEQUENCE, 0L));
- }
- });
-
- if (resp.isEmpty()) {
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.appendCondition(eq(ProfileThreadSnapshotRecord.TASK_ID, taskId))
+ .appendCondition(eq(ProfileThreadSnapshotRecord.SEQUENCE, 0L));
+ }
+ });
+
+ if (resp.getElements().isEmpty()) {
return Collections.emptyList();
}
- final List<String> segmentIDs = resp.stream().map(ProfileThreadSnapshotRecord::getSegmentId).collect(Collectors.toList());
+ List<String> segmentIDs = resp.getElements().stream()
+ .map(new ProfileThreadSnapshotRecordDeserializer())
+ .map(ProfileThreadSnapshotRecord::getSegmentId)
+ .collect(Collectors.toList());
// TODO: support `IN` or `OR` logic operation in BanyanDB
- List<BasicTrace> basicTraces = new LinkedList<>();
+ List<BasicTrace> basicTraces = new ArrayList<>();
for (String segmentID : segmentIDs) {
- List<BasicTrace> subSet = query(BasicTrace.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery traceQuery) {
- traceQuery.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.SEGMENT_ID, segmentID));
- }
- });
- basicTraces.addAll(subSet);
+ final StreamQueryResponse segmentRecordResp = query(SegmentRecord.INDEX_NAME, ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery traceQuery) {
+ traceQuery.appendCondition(eq(SegmentRecord.SEGMENT_ID, segmentID));
+ }
+ });
+ basicTraces.addAll(segmentRecordResp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList()));
}
// TODO: Sort in DB with DESC
@@ -92,35 +104,54 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
- return query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
- .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) maxSequence))
- .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.SEQUENCE, (long) minSequence));
- }
- });
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
+
+ query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(lte(ProfileThreadSnapshotRecord.SEQUENCE, maxSequence))
+ .appendCondition(gte(ProfileThreadSnapshotRecord.SEQUENCE, minSequence));
+ }
+ });
+
+ return resp.getElements().stream().map(new ProfileThreadSnapshotRecordDeserializer()).collect(Collectors.toList());
}
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
- return query(SegmentRecord.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", SegmentRecord.INDEX_NAME, segmentId));
- }
- }).stream().findFirst().orElse(null);
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
+ ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList("data_binary"));
+ query.appendCondition(eq(SegmentRecord.INDEX_NAME, segmentId));
+ }
+ });
+
+ return resp.getElements().stream().map(new SegmentRecordDeserializer()).findFirst().orElse(null);
}
private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) {
- List<ProfileThreadSnapshotRecord> records = query(ProfileThreadSnapshotRecord.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
- .appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, end))
- .appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", ProfileThreadSnapshotRecord.DUMP_TIME, start));
- }
- });
+ StreamQueryResponse resp = query(ProfileThreadSnapshotRecord.INDEX_NAME,
+ ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
+
+ query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
+ .appendCondition(lte(ProfileThreadSnapshotRecord.DUMP_TIME, end))
+ .appendCondition(gte(ProfileThreadSnapshotRecord.DUMP_TIME, start));
+ }
+ });
+
+ List<ProfileThreadSnapshotRecord> records = resp.getElements().stream().map(new ProfileThreadSnapshotRecordDeserializer()).collect(Collectors.toList());
switch (aggType) {
case MIN:
@@ -145,4 +176,55 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
enum AggType {
MIN, MAX
}
+
+ public static class ProfileThreadSnapshotRecordDeserializer implements RowEntityDeserializer<ProfileThreadSnapshotRecord> {
+ @Override
+ public ProfileThreadSnapshotRecord apply(RowEntity row) {
+ ProfileThreadSnapshotRecord record = new ProfileThreadSnapshotRecord();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ record.setTaskId((String) searchable.get(0).getValue());
+ record.setSegmentId((String) searchable.get(1).getValue());
+ record.setDumpTime(((Number) searchable.get(2).getValue()).longValue());
+ record.setSequence(((Number) searchable.get(3).getValue()).intValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ record.setStackBinary(((ByteString) data.get(0).getValue()).toByteArray());
+ return record;
+ }
+ }
+
+ public static class SegmentRecordDeserializer implements RowEntityDeserializer<SegmentRecord> {
+ @Override
+ public SegmentRecord apply(RowEntity row) {
+ SegmentRecord record = new SegmentRecord();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ record.setSegmentId(row.getId());
+ record.setTraceId((String) searchable.get(0).getValue());
+ record.setIsError(((Number) searchable.get(1).getValue()).intValue());
+ record.setServiceId((String) searchable.get(2).getValue());
+ record.setServiceInstanceId((String) searchable.get(3).getValue());
+ record.setEndpointId((String) searchable.get(4).getValue());
+ record.setLatency(((Number) searchable.get(5).getValue()).intValue());
+ record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
+ return record;
+ }
+ }
+
+ public static class BasicTraceDeserializer implements RowEntityDeserializer<BasicTrace> {
+ @Override
+ public BasicTrace apply(RowEntity row) {
+ BasicTrace trace = new BasicTrace();
+ trace.setSegmentId(row.getId());
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ trace.getTraceIds().add((String) searchable.get(0).getValue());
+ trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
+ trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ (String) searchable.get(2).getValue()
+ ).getEndpointName());
+ trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
+ trace.setStart(String.valueOf(searchable.get(4).getValue()));
+ return trace;
+ }
+ }
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index 0bd0e30a82..400731d83e 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -20,22 +20,25 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBRecordBuilder;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
import java.io.IOException;
@RequiredArgsConstructor
public class BanyanDBRecordDAO<T extends Record> implements IRecordDAO {
- private final BanyanDBRecordBuilder<T> storageBuilder;
+ private final BanyanDBStorageDataBuilder<T> storageBuilder;
@Override
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- StreamWrite streamWrite = storageBuilder.entity2Storage(model, (T) record);
+ StreamWrite.StreamWriteBuilder builder = storageBuilder.entity2Storage((T) record)
+ .name(model.getName())
+ .timestamp(TimeBucket.getTimeBucket(record.getTimeBucket(), model.getDownsampling()));
- return new BanyanDBStreamInsertRequest(streamWrite);
+ return new BanyanDBStreamInsertRequest(builder.build());
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
index 76c0359153..98f31ebd7e 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBStorageDAO.java
@@ -19,19 +19,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
-import org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.networkalias.NetworkAddressAlias;
-import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
-import org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord;
-import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
-import org.apache.skywalking.oap.server.core.source.Event;
+import org.apache.skywalking.oap.server.core.analysis.config.NoneStream;
+import org.apache.skywalking.oap.server.core.analysis.management.ManagementData;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.core.storage.IManagementDAO;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
@@ -40,19 +31,7 @@ import org.apache.skywalking.oap.server.core.storage.IRecordDAO;
import org.apache.skywalking.oap.server.core.storage.StorageDAO;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.AlarmRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BrowserErrorLogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.EventBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.LogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.Metadata;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.NetworkAddressAliasBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskLogRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileTaskRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.ProfileThreadSnapshotRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
-
-import java.util.Map;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.BanyanDBStorageDataBuilder;
@Slf4j
public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> implements StorageDAO {
@@ -62,79 +41,21 @@ public class BanyanDBStorageDAO extends AbstractDAO<BanyanDBStorageClient> imple
@Override
public IMetricsDAO newMetricsDao(StorageBuilder storageBuilder) {
- try {
- Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
- if (Event.class.equals(returnType)) {
- return new BanyanDBMetricsDAO<>(new EventBuilder());
- } else if (ServiceTraffic.class.equals(returnType)) {
- return new BanyanDBMetricsDAO<>(new Metadata.ServiceTrafficBuilder());
- } else if (InstanceTraffic.class.equals(returnType)) {
- return new BanyanDBMetricsDAO<>(new Metadata.InstanceTrafficBuilder());
- } else if (EndpointTraffic.class.equals(returnType)) {
- return new BanyanDBMetricsDAO<>(new Metadata.EndpointTrafficBuilder());
- } else if (NetworkAddressAlias.class.equals(returnType)) {
- return new BanyanDBMetricsDAO<>(new NetworkAddressAliasBuilder());
- } else {
- throw new IllegalStateException("record type is not supported");
- }
- } catch (NoSuchMethodException noSuchMethodException) {
- log.error("cannot find method storage2Entity", noSuchMethodException);
- throw new RuntimeException("cannot find method storage2Entity");
- }
+ return new BanyanDBMetricsDAO<>((BanyanDBStorageDataBuilder<Metrics>) storageBuilder);
}
@Override
public IRecordDAO newRecordDao(StorageBuilder storageBuilder) {
- try {
- Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
- if (SegmentRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new SegmentRecordBuilder());
- } else if (AlarmRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new AlarmRecordBuilder());
- } else if (BrowserErrorLogRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new BrowserErrorLogRecordBuilder());
- } else if (LogRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new LogRecordBuilder());
- } else if (ProfileTaskLogRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new ProfileTaskLogRecordBuilder());
- } else if (ProfileThreadSnapshotRecord.class.equals(returnType)) {
- return new BanyanDBRecordDAO<>(new ProfileThreadSnapshotRecordBuilder());
- } else {
- throw new IllegalStateException("record type is not supported");
- }
- } catch (NoSuchMethodException noSuchMethodException) {
- log.error("cannot find method storage2Entity", noSuchMethodException);
- throw new RuntimeException("cannot find method storage2Entity");
- }
+ return new BanyanDBRecordDAO<>((BanyanDBStorageDataBuilder<Record>) storageBuilder);
}
@Override
public INoneStreamDAO newNoneStreamDao(StorageBuilder storageBuilder) {
- try {
- Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
- if (ProfileTaskRecord.class.equals(returnType)) {
- return new BanyanDBNoneStreamDAO<>(getClient(), new ProfileTaskRecordBuilder());
- } else {
- throw new IllegalStateException("record type is not supported");
- }
- } catch (NoSuchMethodException noSuchMethodException) {
- log.error("cannot find method storage2Entity", noSuchMethodException);
- throw new RuntimeException("cannot find method storage2Entity");
- }
+ return new BanyanDBNoneStreamDAO<>(getClient(), (BanyanDBStorageDataBuilder<NoneStream>) storageBuilder);
}
@Override
public IManagementDAO newManagementDao(StorageBuilder storageBuilder) {
- try {
- Class<?> returnType = storageBuilder.getClass().getMethod("storage2Entity", Map.class).getReturnType();
- if (UITemplate.class.equals(returnType)) {
- return new BanyanDBManagementDAO<>(getClient(), new UITemplateBuilder());
- } else {
- throw new IllegalStateException("record type is not supported");
- }
- } catch (NoSuchMethodException noSuchMethodException) {
- log.error("cannot find method storage2Entity", noSuchMethodException);
- throw new RuntimeException("cannot find method storage2Entity");
- }
+ return new BanyanDBManagementDAO<>(getClient(), (BanyanDBStorageDataBuilder<ManagementData>) storageBuilder);
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 73e80c8f94..c5bb950bc6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -19,9 +19,15 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.ByteString;
import lombok.Getter;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
@@ -38,6 +44,7 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRe
import java.io.IOException;
import java.util.Collections;
import java.util.List;
+import java.util.stream.Collectors;
public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
@@ -46,39 +53,39 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public TraceBrief queryBasicTraces(long startSecondTB, long endSecondTB, long minDuration, long maxDuration, String serviceId, String serviceInstanceId, String endpointId, String traceId, int limit, int from, TraceState traceState, QueryOrder queryOrder, List<Tag> tags) throws IOException {
- final QueryBuilder builder = new QueryBuilder() {
+ final QueryBuilder q = new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
if (minDuration != 0) {
// duration >= minDuration
- query.appendCondition(PairQueryCondition.LongQueryCondition.ge("searchable", "duration", minDuration));
+ query.appendCondition(gte("duration", minDuration));
}
if (maxDuration != 0) {
// duration <= maxDuration
- query.appendCondition(PairQueryCondition.LongQueryCondition.le("searchable", "duration", maxDuration));
+ query.appendCondition(lte("duration", maxDuration));
}
if (!Strings.isNullOrEmpty(serviceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_id", serviceId));
+ query.appendCondition(eq("service_id", serviceId));
}
if (!Strings.isNullOrEmpty(serviceInstanceId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "service_instance_id", serviceInstanceId));
+ query.appendCondition(eq("service_instance_id", serviceInstanceId));
}
if (!Strings.isNullOrEmpty(endpointId)) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "endpoint_id", endpointId));
+ query.appendCondition(eq("endpoint_id", endpointId));
}
switch (traceState) {
case ERROR:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.ERROR.getState()));
+ query.appendCondition(eq("state", TraceStateStorage.ERROR.getState()));
break;
case SUCCESS:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.SUCCESS.getState()));
+ query.appendCondition(eq("state", TraceStateStorage.SUCCESS.getState()));
break;
default:
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", "state", (long) TraceStateStorage.ALL.getState()));
+ query.appendCondition(eq("state", TraceStateStorage.ALL.getState()));
break;
}
@@ -94,7 +101,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
if (CollectionUtils.isNotEmpty(tags)) {
for (final Tag tag : tags) {
if (SegmentRecordBuilder.INDEXED_TAGS.contains(tag.getKey())) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", tag.getKey(), tag.getValue()));
+ query.appendCondition(eq(tag.getKey(), tag.getValue()));
}
}
}
@@ -104,13 +111,17 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
}
};
- final List<BasicTrace> basicTraces;
- if (startSecondTB != 0 && endSecondTB != 0) {
- basicTraces = query(BasicTrace.class, builder, TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
- } else {
- basicTraces = query(BasicTrace.class, builder);
+ TimestampRange tsRange = null;
+
+ if (startSecondTB > 0 && endSecondTB > 0) {
+ tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
+ ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"), tsRange, q);
+
+ List<BasicTrace> basicTraces = resp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList());
+
TraceBrief brief = new TraceBrief();
brief.setTotal(basicTraces.size());
brief.getTraces().addAll(basicTraces);
@@ -119,12 +130,17 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
- return query(SegmentRecord.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", "trace_id", traceId));
- }
- });
+ StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
+ ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList("data_binary"));
+ query.appendCondition(eq(SegmentRecord.TRACE_ID, traceId));
+ }
+ });
+
+ return resp.getElements().stream().map(new SegmentRecordDeserializer()).collect(Collectors.toList());
}
@Override
@@ -142,4 +158,40 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
this.state = state;
}
}
+
+ public static class BasicTraceDeserializer implements RowEntityDeserializer<BasicTrace> {
+ @Override
+ public BasicTrace apply(RowEntity row) {
+ BasicTrace trace = new BasicTrace();
+ trace.setSegmentId(row.getId());
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ trace.getTraceIds().add((String) searchable.get(0).getValue());
+ trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
+ trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
+ (String) searchable.get(2).getValue()
+ ).getEndpointName());
+ trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
+ trace.setStart(String.valueOf(searchable.get(4).getValue()));
+ return trace;
+ }
+ }
+
+ public static class SegmentRecordDeserializer implements RowEntityDeserializer<SegmentRecord> {
+ @Override
+ public SegmentRecord apply(RowEntity row) {
+ SegmentRecord record = new SegmentRecord();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ record.setSegmentId(row.getId());
+ record.setTraceId((String) searchable.get(0).getValue());
+ record.setIsError(((Number) searchable.get(1).getValue()).intValue());
+ record.setServiceId((String) searchable.get(2).getValue());
+ record.setServiceInstanceId((String) searchable.get(3).getValue());
+ record.setEndpointId((String) searchable.get(4).getValue());
+ record.setLatency(((Number) searchable.get(5).getValue()).intValue());
+ record.setStartTime(((Number) searchable.get(6).getValue()).longValue());
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ record.setDataBinary(((ByteString) data.get(0).getValue()).toByteArray());
+ return record;
+ }
+ }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
index 9ed0af8447..1d9f977b0a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBUITemplateManagementDAO.java
@@ -18,21 +18,25 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
+import com.google.common.collect.ImmutableList;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.Tag;
+import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplate;
+import org.apache.skywalking.oap.server.core.query.enumeration.TemplateType;
import org.apache.skywalking.oap.server.core.query.input.DashboardSetting;
import org.apache.skywalking.oap.server.core.query.type.DashboardConfiguration;
import org.apache.skywalking.oap.server.core.query.type.TemplateChangeStatus;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.UITemplateBuilder;
import java.io.IOException;
import java.util.List;
+import java.util.stream.Collectors;
/**
* {@link org.apache.skywalking.oap.server.core.management.ui.template.UITemplate} is a stream
@@ -44,15 +48,19 @@ public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO impleme
@Override
public List<DashboardConfiguration> getAllTemplates(Boolean includingDisabled) throws IOException {
- return query(DashboardConfiguration.class, new QueryBuilder() {
- @Override
- public void apply(StreamQuery query) {
- query.setLimit(10000);
- if (!includingDisabled) {
- query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", UITemplate.DISABLED, (long) BooleanUtils.FALSE));
- }
- }
- });
+ StreamQueryResponse resp = query(UITemplate.INDEX_NAME, ImmutableList.of(UITemplate.NAME, UITemplate.DISABLED),
+ new QueryBuilder() {
+ @Override
+ public void apply(StreamQuery query) {
+ query.setDataProjections(ImmutableList.of(UITemplate.ACTIVATED, UITemplate.CONFIGURATION, UITemplate.TYPE));
+ query.setLimit(10000);
+ if (!includingDisabled) {
+ query.appendCondition(eq(UITemplate.DISABLED, BooleanUtils.FALSE));
+ }
+ }
+ });
+
+ return resp.getElements().stream().map(new DashboardConfigurationDeserializer()).collect(Collectors.toList());
}
@Override
@@ -71,7 +79,7 @@ public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO impleme
.dataTag(Tag.stringField(uiTemplate.getConfiguration()))
// data - activated
.dataTag(Tag.longField(uiTemplate.getActivated()))
- .timestamp(UITemplateBuilder.UI_TEMPLATE_TIMESTAMP)
+ .timestamp(1L)
.elementId(uiTemplate.id())
.build();
getClient().write(request);
@@ -87,4 +95,24 @@ public class BanyanDBUITemplateManagementDAO extends AbstractBanyanDBDAO impleme
public TemplateChangeStatus disableTemplate(String name) throws IOException {
return TemplateChangeStatus.builder().status(false).message("Can't disable the template").build();
}
+
+ public static class DashboardConfigurationDeserializer implements RowEntityDeserializer<DashboardConfiguration> {
+ @Override
+ public DashboardConfiguration apply(RowEntity row) {
+ DashboardConfiguration dashboardConfiguration = new DashboardConfiguration();
+ final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
+ // name
+ dashboardConfiguration.setName((String) searchable.get(0).getValue());
+ // disabled
+ dashboardConfiguration.setDisabled(BooleanUtils.valueToBoolean(((Number) searchable.get(1).getValue()).intValue()));
+ final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
+ // activated
+ dashboardConfiguration.setActivated(BooleanUtils.valueToBoolean(((Number) data.get(0).getValue()).intValue()));
+ // configuration
+ dashboardConfiguration.setConfiguration((String) data.get(1).getValue());
+ // type
+ dashboardConfiguration.setType(TemplateType.forName((String) data.get(2).getValue()));
+ return dashboardConfiguration;
+ }
+ }
}
[skywalking] 13/24: use unified tag family name
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit f32fd8d6f4e7e97e3a56a9a88aecb0b0070ed331
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 9 18:31:39 2021 +0800
use unified tag family name
---
.../storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index 0b89545ee4..a3d2f1dc85 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -25,6 +25,7 @@ import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
import java.util.List;
import java.util.function.Function;
@@ -53,24 +54,22 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
}
protected abstract static class QueryBuilder {
- protected static final String SEARCHABLE = "searchable";
-
abstract void apply(final StreamQuery query);
protected PairQueryCondition<Long> eq(String name, long value) {
- return PairQueryCondition.LongQueryCondition.eq(SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.eq(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<Long> lte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.le(SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.le(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<Long> gte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.ge(SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.ge(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
}
protected PairQueryCondition<String> eq(String name, String value) {
- return PairQueryCondition.StringQueryCondition.eq(SEARCHABLE, name, value);
+ return PairQueryCondition.StringQueryCondition.eq(StreamMetaInfo.TAG_FAMILY_SEARCHABLE, name, value);
}
}
[skywalking] 14/24: add banyandb plugin to pom
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 16325317dc7dbb7dd855f83f6e103885715d6d4a
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 9 18:45:19 2021 +0800
add banyandb plugin to pom
---
oap-server/server-starter/pom.xml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/oap-server/server-starter/pom.xml b/oap-server/server-starter/pom.xml
index 38f12172fe..967a9439ba 100644
--- a/oap-server/server-starter/pom.xml
+++ b/oap-server/server-starter/pom.xml
@@ -197,6 +197,11 @@
<artifactId>storage-iotdb-plugin</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>storage-banyandb-plugin</artifactId>
+ <version>${project.version}</version>
+ </dependency>
<!-- storage module -->
<!-- queryBuild module -->
[skywalking] 12/24: fix license headers
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit bddab99719bee04cadc6fccebad94630dd0e337b
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 9 18:28:40 2021 +0800
fix license headers
---
.../plugin/banyandb/BanyanDBStorageBuilderFactory.java | 18 ++++++++++++++++++
.../storage/plugin/banyandb/schema/EventBuilder.java | 18 ++++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
index a07fe1f106..b87d3d1dc5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageBuilderFactory.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
index 9a99763c73..bc604776c9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EventBuilder.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb.schema;
import com.google.common.collect.ImmutableList;
[skywalking] 24/24: add measure id and check sharding keys
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit cd13a7372ac6c0a9b74698c07780a6022c453039
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Sat Apr 30 16:55:24 2022 +0800
add measure id and check sharding keys
---
.../storage/plugin/banyandb/BanyanDBConverter.java | 2 +-
.../storage/plugin/banyandb/MetadataRegistry.java | 42 ++++++++++++++++------
2 files changed, 32 insertions(+), 12 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
index c512b5653c..c3c936ef84 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
@@ -45,7 +45,7 @@ public class BanyanDBConverter {
@Override
public Object get(String fieldName) {
MetadataRegistry.ColumnSpec spec = schema.getSpec(fieldName);
- if (double.class.equals(spec.getModelColumn().getType())) {
+ if (double.class.equals(spec.getColumnClass())) {
return ByteUtil.bytes2Double(rowEntity.getTagValue(fieldName));
} else {
return rowEntity.getTagValue(fieldName);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
index 6c6fbb7658..3ff4cb2833 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
@@ -67,6 +67,7 @@ import java.util.stream.Collectors;
public enum MetadataRegistry {
INSTANCE;
+ private static final String ID = "id";
private final Map<String, Schema> registry = new ConcurrentHashMap<>();
public NamedSchema<?> registerModel(Model model, ConfigService configService) {
@@ -87,25 +88,33 @@ public enum MetadataRegistry {
.filter(Objects::nonNull)
.collect(Collectors.toList());
-
if (partialMetadata.getKind() == Kind.STREAM) {
final Stream.Builder builder = Stream.create(partialMetadata.getGroup(), partialMetadata.getName());
+ if (entities.isEmpty()) {
+ log.warn("sharding keys of model[stream.{}] must not be empty", model.getName());
+// throw new IllegalStateException("sharding keys of model[" + model.getName() + "] must not be empty");
+ }
builder.setEntityRelativeTags(entities);
builder.addTagFamilies(tagFamilySpecs);
builder.addIndexes(indexRules);
-
registry.put(model.getName(), schemaBuilder.build());
return builder.build();
} else {
final Measure.Builder builder = Measure.create(partialMetadata.getGroup(), partialMetadata.getName(),
downSamplingDuration(model.getDownsampling()));
- builder.setEntityRelativeTags(entities);
+ if (entities.isEmpty()) { // if shardingKeys is empty, for measure, we can use ID as a single sharding key.
+ builder.setEntityRelativeTags(ID);
+ } else {
+ builder.setEntityRelativeTags(entities);
+ }
builder.addTagFamilies(tagFamilySpecs);
builder.addIndexes(indexRules);
// parse and set field
Optional<ValueColumnMetadata.ValueColumn> valueColumnOpt = ValueColumnMetadata.INSTANCE
.readValueColumnDefinition(model.getName());
valueColumnOpt.ifPresent(valueColumn -> builder.addField(parseFieldSpec(modelColumnMap.get(valueColumn.getValueCName()), valueColumn)));
+ // register ID
+ schemaBuilder.spec(ID, new ColumnSpec(ColumnType.TAG, String.class));
registry.put(model.getName(), schemaBuilder.build());
return builder.build();
@@ -194,14 +203,14 @@ public enum MetadataRegistry {
.readValueColumnDefinition(model.getName());
for (final ModelColumn col : model.getColumns()) {
if (valueColumnOpt.isPresent() && valueColumnOpt.get().getValueCName().equals(col.getColumnName().getStorageName())) {
- builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.FIELD, col));
+ builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.FIELD, col.getType()));
continue;
}
final TagFamilySpec.TagSpec tagSpec = parseTagSpec(col);
if (tagSpec == null) {
continue;
}
- builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.TAG, col));
+ builder.spec(col.getColumnName().getStorageName(), new ColumnSpec(ColumnType.TAG, col.getType()));
if (col.shouldIndex()) {
// build indexRule
IndexRule indexRule = parseIndexRule(tagSpec.getTagName(), col);
@@ -316,11 +325,22 @@ public enum MetadataRegistry {
}
private List<TagFamilySpec> extractTagFamilySpec(List<TagMetadata> tagMetadataList) {
- return tagMetadataList.stream().collect(Collectors.groupingBy(tagMetadata -> tagMetadata.isIndex() ? PartialMetadata.this.indexFamily() : PartialMetadata.this.nonIndexFamily()))
- .entrySet().stream()
- .map(entry -> TagFamilySpec.create(entry.getKey())
- .addTagSpecs(entry.getValue().stream().map(TagMetadata::getTagSpec).collect(Collectors.toList())).build())
- .collect(Collectors.toList());
+ Map<String, List<TagMetadata>> tagMetadataMap = tagMetadataList.stream()
+ .collect(Collectors.groupingBy(tagMetadata -> tagMetadata.isIndex() ? PartialMetadata.this.indexFamily() : PartialMetadata.this.nonIndexFamily()));
+
+ final List<TagFamilySpec> tagFamilySpecs = new ArrayList<>(tagMetadataMap.size());
+ for (final Map.Entry<String, List<TagMetadata>> entry : tagMetadataMap.entrySet()) {
+ final TagFamilySpec.Builder b = TagFamilySpec.create(entry.getKey())
+ .addTagSpecs(entry.getValue().stream().map(TagMetadata::getTagSpec).collect(Collectors.toList()));
+ if (this.getKind() == Kind.MEASURE && entry.getKey().equals(this.indexFamily())) {
+ // append measure ID, but it should not generate an index in the client side.
+ // BanyanDB will take care of the ID index registration.
+ b.addTagSpec(TagFamilySpec.TagSpec.newIDTag(ID));
+ }
+ tagFamilySpecs.add(b.build());
+ }
+
+ return tagFamilySpecs;
}
public Group getOrCreateGroup(BanyanDBClient client) throws BanyanDBException {
@@ -392,7 +412,7 @@ public enum MetadataRegistry {
@Getter
public static class ColumnSpec {
private final ColumnType columnType;
- private final ModelColumn modelColumn;
+ private final Class<?> columnClass;
}
public enum ColumnType {
[skywalking] 08/24: fix pom and fix style issues
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 4f41af33689ddab6d11de227f30cfc5501bdb1f7
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Sun Dec 5 23:05:56 2021 +0800
fix pom and fix style issues
---
oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml | 2 +-
.../plugin/banyandb/deserializer/AlarmMessageDeserializer.java | 2 +-
.../server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java | 1 -
.../storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java | 4 ++--
4 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
index 76848c1b4b..d5ebbfc969 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
@@ -21,7 +21,7 @@
<parent>
<artifactId>server-storage-plugin</artifactId>
<groupId>org.apache.skywalking</groupId>
- <version>8.9.0-SNAPSHOT</version>
+ <version>8.10.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
index 3165d60afa..2df0ef739b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/deserializer/AlarmMessageDeserializer.java
@@ -34,7 +34,7 @@ import org.apache.skywalking.oap.server.core.query.type.KeyValue;
import java.util.List;
public class AlarmMessageDeserializer extends AbstractBanyanDBDeserializer<AlarmMessage> {
- private final Gson GSON = new Gson();
+ private static final Gson GSON = new Gson();
public AlarmMessageDeserializer() {
super(AlarmRecord.INDEX_NAME,
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index d0daa3375f..71d45dc7b6 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -53,7 +53,6 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
return resp.getElements().stream().map(deserializer::map).collect(Collectors.toList());
}
-
interface QueryBuilder {
void apply(final StreamQuery query);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index bee6508cd2..b3b6fece70 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -87,7 +87,7 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
public void apply(StreamQuery query) {
query.appendCondition(PairQueryCondition.LongQueryCondition.eq("searchable", ServiceTraffic.NODE_TYPE, (long) nodeType.value()));
}
- }).stream().filter((s) -> s.getName().contains(keyword)) // TODO: support analyzer in database
+ }).stream().filter(s -> s.getName().contains(keyword)) // TODO: support analyzer in database
.collect(Collectors.toList());
}
@@ -111,7 +111,7 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
public void apply(StreamQuery query) {
query.appendCondition(PairQueryCondition.StringQueryCondition.eq("searchable", EndpointTraffic.SERVICE_ID, serviceId));
}
- }).stream().filter((e) -> e.getName().contains(keyword))
+ }).stream().filter(e -> e.getName().contains(keyword))
.limit(limit).collect(Collectors.toList());
}
[skywalking] 16/24: support events, profile_task and instance_traffic
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit a4a29bf4dfc2b17bb83feff16f86dcceb15ecb1b
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Fri Jan 21 14:12:07 2022 +0800
support events, profile_task and instance_traffic
---
.../plugin/banyandb/BanyanDBIndexInstaller.java | 2 +-
.../storage/plugin/banyandb/StreamMetaInfo.java | 4 +-
.../schema/NetworkAddressAliasBuilder.java | 7 ++-
.../banyandb/schema/ProfileTaskRecordBuilder.java | 11 +++-
.../plugin/banyandb/stream/BanyanDBBatchDAO.java | 3 +-
.../plugin/banyandb/stream/BanyanDBMetricsDAO.java | 2 +-
.../stream/BanyanDBNetworkAddressAliasDAO.java | 8 +--
.../stream/BanyanDBProfileTaskQueryDAO.java | 9 ++-
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 2 +-
.../src/main/resources/metadata/events.json | 70 ++++++++++++++++++++++
.../{segment/duration.json => create_time.json} | 4 +-
.../index_rules/{segment => }/db.instance.json | 2 +-
.../index_rules/{segment => }/db.type.json | 2 +-
.../index_rules/{ui_template => }/disabled.json | 2 +-
.../{segment/duration.json => dump_period.json} | 4 +-
.../index_rules/{segment => }/duration.json | 2 +-
.../{segment/duration.json => end_time.json} | 4 +-
.../{ui_template/name.json => endpoint.json} | 4 +-
.../index_rules/{segment => }/endpoint_id.json | 2 +-
.../{segment/mq.broker.json => endpoint_name.json} | 4 +-
.../index_rules/{segment => }/http.method.json | 2 +-
.../index_rules/{ui_template/name.json => id.json} | 4 +-
.../{segment/duration.json => last_ping.json} | 6 +-
.../duration.json => last_update_time_bucket.json} | 6 +-
.../duration.json => max_sampling_count.json} | 4 +-
.../duration.json => min_duration_threshold.json} | 4 +-
.../index_rules/{segment => }/mq.broker.json | 2 +-
.../index_rules/{segment => }/mq.queue.json | 2 +-
.../index_rules/{segment => }/mq.topic.json | 2 +-
.../index_rules/{ui_template => }/name.json | 2 +-
.../{ui_template/name.json => service.json} | 4 +-
.../{ui_template/name.json => service_id.json} | 4 +-
.../db.instance.json => service_instance.json} | 4 +-
.../{segment/duration.json => start_time.json} | 4 +-
.../index_rules/{segment => }/status_code.json | 2 +-
.../index_rules/{segment => }/trace_id.json | 2 +-
.../{ui_template/name.json => type.json} | 2 +-
.../{ui_template/name.json => uuid.json} | 4 +-
.../main/resources/metadata/instance_traffic.json | 47 +++++++++++++++
.../resources/metadata/network_address_alias.json | 50 ++++++++++++++++
.../src/main/resources/metadata/profile_task.json | 61 +++++++++++++++++++
41 files changed, 305 insertions(+), 61 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index c9566b1549..6a8f789976 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -41,7 +41,7 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
if (metaInfo != null) {
log.info("install index {}", model.getName());
- ((BanyanDBStorageClient) client).createStream(metaInfo);
+// ((BanyanDBStorageClient) client).createStream(metaInfo);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
index 9dda0ead72..de7af730ee 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
@@ -103,6 +103,7 @@ public class StreamMetaInfo {
try {
InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream("metadata/" + name + ".json");
if (is == null) {
+ log.warn("fail to find definition for {}", name);
return null;
}
String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
@@ -118,8 +119,9 @@ public class StreamMetaInfo {
private static BanyandbDatabase.IndexRule parseIndexRulesFromJSON(String streamName, String name) {
try {
InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream(String.join("/",
- new String[]{"metadata", "index_rules", streamName, name + ".json"}));
+ new String[]{"metadata", "index_rules", name + ".json"}));
if (is == null) {
+ log.warn("fail to find index rules for {}", streamName);
return null;
}
String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
index ee85a50b03..ca2d78d2ef 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
@@ -35,9 +35,14 @@ public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<Netwo
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(3);
+ // 0 - time_bucket
+ data.add(TagAndValue.longField(entity.getTimeBucket()));
+ // 1 - address
data.add(TagAndValue.stringField(entity.getAddress()));
+ // 2 - represent_service_id
data.add(TagAndValue.stringField(entity.getRepresentServiceId()));
+ // 3 - represent_service_instance_id
data.add(TagAndValue.stringField(entity.getRepresentServiceInstanceId()));
return data;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
index bc91290976..23a5bf8fdf 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskRecordBuilder.java
@@ -30,14 +30,23 @@ public class ProfileTaskRecordBuilder extends BanyanDBStorageDataBuilder<Profile
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskRecord entity) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(9);
+ // 0 - id
searchable.add(TagAndValue.stringField(entity.id()));
+ // 1 - service_id
searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ // 2 - endpoint_name
searchable.add(TagAndValue.stringField(entity.getEndpointName()));
+ // 3 - start_time
searchable.add(TagAndValue.longField(entity.getStartTime()));
+ // 4 - duration
searchable.add(TagAndValue.longField(entity.getDuration()));
+ // 5 - min_duration_threshold
searchable.add(TagAndValue.longField(entity.getMinDurationThreshold()));
- searchable.add(TagAndValue.longField(entity.getDuration()));
+ // 6 - dump_period
+ searchable.add(TagAndValue.longField(entity.getDumpPeriod()));
+ // 7 - create_time
searchable.add(TagAndValue.longField(entity.getCreateTime()));
+ // 8 - max_sampling_count
searchable.add(TagAndValue.longField(entity.getMaxSamplingCount()));
return searchable;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
index 9404996887..040b031828 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
@@ -62,9 +62,10 @@ public class BanyanDBBatchDAO extends AbstractDAO<BanyanDBStorageClient> impleme
if (CollectionUtils.isNotEmpty(prepareRequests)) {
return CompletableFuture.allOf(prepareRequests.stream().map(prepareRequest -> {
- if (prepareRequest instanceof InsertRequest) {
+ if (prepareRequest instanceof BanyanDBStreamInsertRequest) {
// TODO: return CompletableFuture<Void>
this.bulkProcessor.add(((BanyanDBStreamInsertRequest) prepareRequest).getStreamWrite());
+ } else {
}
return CompletableFuture.completedFuture(null);
}).toArray(CompletableFuture[]::new));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
index 5aa1b7657d..f1ed5b8d61 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetricsDAO.java
@@ -45,7 +45,7 @@ public class BanyanDBMetricsDAO<T extends Metrics> implements IMetricsDAO {
public InsertRequest prepareBatchInsert(Model model, Metrics metrics) throws IOException {
StreamWrite.StreamWriteBuilder builder = this.storageBuilder.entity2Storage((T) metrics)
.name(model.getName())
- .timestamp(TimeBucket.getTimeBucket(metrics.getTimeBucket(), model.getDownsampling()));
+ .timestamp(TimeBucket.getTimestamp(metrics.getTimeBucket(), model.getDownsampling()));
return new BanyanDBStreamInsertRequest(builder.build());
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
index 33aec37bac..d5006ea689 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNetworkAddressAliasDAO.java
@@ -62,13 +62,13 @@ public class BanyanDBNetworkAddressAliasDAO extends AbstractBanyanDBDAO implemen
// searchable - last_update_time_bucket
model.setLastUpdateTimeBucket(((Number) searchable.get(0).getValue()).longValue());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // data - time_bucket
+ // data 0 - time_bucket
model.setTimeBucket(((Number) data.get(0).getValue()).longValue());
- // data - address
+ // data 1 - address
model.setAddress((String) data.get(1).getValue());
- // data - represent_service_id
+ // data 2 - represent_service_id
model.setRepresentServiceId((String) data.get(2).getValue());
- // data - represent_service_instance_id
+ // data 3 - represent_service_instance_id
model.setRepresentServiceInstanceId((String) data.get(3).getValue());
return model;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 1cbaea95af..93cc99c6c3 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -29,6 +29,7 @@ import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
import java.io.IOException;
import java.util.List;
@@ -39,8 +40,6 @@ import java.util.stream.Collectors;
* {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskRecord} is a stream
*/
public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskQueryDAO {
- public static final String ID = "profile_task_query_id";
-
public BanyanDBProfileTaskQueryDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -48,7 +47,7 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements
@Override
public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
- ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ImmutableList.of(StreamMetaInfo.ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT), new QueryBuilder() {
@Override
@@ -87,13 +86,13 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements
}
StreamQueryResponse resp = query(ProfileTaskRecord.INDEX_NAME,
- ImmutableList.of(ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
+ ImmutableList.of(StreamMetaInfo.ID, ProfileTaskRecord.SERVICE_ID, ProfileTaskRecord.ENDPOINT_NAME,
ProfileTaskRecord.START_TIME, ProfileTaskRecord.DURATION, ProfileTaskRecord.MIN_DURATION_THRESHOLD,
ProfileTaskRecord.DUMP_PERIOD, ProfileTaskRecord.CREATE_TIME, ProfileTaskRecord.MAX_SAMPLING_COUNT),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.appendCondition(eq(ID, id));
+ query.appendCondition(eq(StreamMetaInfo.ID, id));
query.setLimit(1);
}
});
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index 400731d83e..d028fa91c3 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -37,7 +37,7 @@ public class BanyanDBRecordDAO<T extends Record> implements IRecordDAO {
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
StreamWrite.StreamWriteBuilder builder = storageBuilder.entity2Storage((T) record)
.name(model.getName())
- .timestamp(TimeBucket.getTimeBucket(record.getTimeBucket(), model.getDownsampling()));
+ .timestamp(TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling()));
return new BanyanDBStreamInsertRequest(builder.build());
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
new file mode 100644
index 0000000000..9c0f091ac2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
@@ -0,0 +1,70 @@
+{
+ "metadata": {
+ "name": "events",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "message",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "parameters",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "uuid",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "service",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "service_instance",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "endpoint",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "name",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "type",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "start_time",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "end_time",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json
similarity index 77%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json
index 9ae7387089..addab64e17 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/create_time.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "create_time",
"group": "default"
},
"tags": [
- "duration"
+ "create_time"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json
similarity index 84%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json
index d1b85c2be2..5d5c159c39 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.instance.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_db.instance",
+ "name": "db.instance",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json
similarity index 85%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json
index 479751ba13..67914ee464 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.type.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/db.type.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_db.type",
+ "name": "db.type",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
similarity index 83%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
index 9146c34d11..8cfb12816a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "ui_template_disabled",
+ "name": "disabled",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json
similarity index 77%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json
index 9ae7387089..25ae87f411 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_period.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "dump_period",
"group": "default"
},
"tags": [
- "duration"
+ "dump_period"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json
similarity index 84%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json
index 9ae7387089..6f18db1359 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/duration.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "duration",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json
similarity index 77%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json
index 9ae7387089..951a1c88b9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/end_time.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "end_time",
"group": "default"
},
"tags": [
- "duration"
+ "end_time"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json
similarity index 79%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json
index 067ca7f123..f9fefdddf0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "endpoint",
"group": "default"
},
"tags": [
- "name"
+ "endpoint"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json
similarity index 84%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json
index ca368266a8..7da3b0d2f5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/endpoint_id.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_id.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_endpoint_id",
+ "name": "endpoint_id",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json
similarity index 77%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json
index 4dca9074a5..0678ae4691 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/endpoint_name.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_mq.broker",
+ "name": "endpoint_name",
"group": "default"
},
"tags": [
- "mq.broker"
+ "endpoint_name"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json
similarity index 84%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json
index 3b5794ec09..3001154bfc 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/http.method.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/http.method.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_http.method",
+ "name": "http.method",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json
similarity index 79%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json
index 067ca7f123..c90b39ac62 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/id.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "id",
"group": "default"
},
"tags": [
- "name"
+ "id"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json
similarity index 76%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json
index 9ae7387089..7d063b2dd4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_ping.json
@@ -1,12 +1,12 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "last_ping",
"group": "default"
},
"tags": [
- "duration"
+ "last_ping"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json
similarity index 69%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json
index 9ae7387089..bbde5cf37b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/last_update_time_bucket.json
@@ -1,12 +1,12 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "last_update_time_bucket",
"group": "default"
},
"tags": [
- "duration"
+ "last_update_time_bucket"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
-}
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json
similarity index 73%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json
index 9ae7387089..1fb14a84c2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/max_sampling_count.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "max_sampling_count",
"group": "default"
},
"tags": [
- "duration"
+ "max_sampling_count"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json
similarity index 70%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json
index 9ae7387089..f359e2d5ba 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/min_duration_threshold.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "min_duration_threshold",
"group": "default"
},
"tags": [
- "duration"
+ "min_duration_threshold"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json
similarity index 84%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json
index 4dca9074a5..0cfbd278dd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.broker.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.broker.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_mq.broker",
+ "name": "mq.broker",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json
similarity index 85%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json
index 1b3126e056..0c589fc6e4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.queue.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.queue.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_mq.queue",
+ "name": "mq.queue",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json
similarity index 85%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json
index ddae44b1a1..5dc51fcf66 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/mq.topic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/mq.topic.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_mq.topic",
+ "name": "mq.topic",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json
similarity index 84%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json
index 067ca7f123..db25d029f2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/name.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "name",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json
similarity index 79%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json
index 067ca7f123..4764f16eca 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "service",
"group": "default"
},
"tags": [
- "name"
+ "service"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json
similarity index 79%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json
index 067ca7f123..76c3f67d22 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_id.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "service_id",
"group": "default"
},
"tags": [
- "name"
+ "service_id"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json
similarity index 75%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json
index d1b85c2be2..73d84e7151 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_instance.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_db.instance",
+ "name": "service_instance",
"group": "default"
},
"tags": [
- "db.instance"
+ "service_instance"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json
similarity index 77%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json
index 9ae7387089..5a73ad12f4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/duration.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/start_time.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "segment_duration",
+ "name": "start_time",
"group": "default"
},
"tags": [
- "duration"
+ "start_time"
],
"type": "TYPE_TREE",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json
similarity index 84%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json
index 3e472a1669..ad034773f5 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/status_code.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/status_code.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_status_code",
+ "name": "status_code",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json
similarity index 84%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json
index 3ac579999c..2f84bfb489 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/trace_id.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/trace_id.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "segment_trace_id",
+ "name": "trace_id",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json
similarity index 84%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json
index 067ca7f123..db25d029f2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/type.json
@@ -1,6 +1,6 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "name",
"group": "default"
},
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json
similarity index 79%
rename from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
rename to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json
index 067ca7f123..4cd401c3dc 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/ui_template/name.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/uuid.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "ui_template_name",
+ "name": "uuid",
"group": "default"
},
"tags": [
- "name"
+ "uuid"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
new file mode 100644
index 0000000000..b63b29c2c2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
@@ -0,0 +1,47 @@
+{
+ "metadata": {
+ "name": "instance_traffic",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "data_binary",
+ "type": "TAG_TYPE_DATA_BINARY"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "last_ping",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "id",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
new file mode 100644
index 0000000000..6d46d97799
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
@@ -0,0 +1,50 @@
+{
+ "metadata": {
+ "name": "network_address_alias",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "time_bucket",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "address",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "represent_service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "represent_service_instance_id",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "last_update_time_bucket",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 52000,
+ "unit": "DURATION_UNIT_WEEK"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
new file mode 100644
index 0000000000..23f89f19c4
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
@@ -0,0 +1,61 @@
+{
+ "metadata": {
+ "name": "profile_task",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "endpoint_name",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "start_time",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "duration",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "min_duration_threshold",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "dump_period",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "create_time",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "max_sampling_count",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
[skywalking] 21/24: make code compiled
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 05569c0693a871a4d83471e1ab0e508c60b84ffe
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Apr 21 21:23:55 2022 +0800
make code compiled
---
.../storage-banyandb-plugin/pom.xml | 2 +-
.../storage/plugin/banyandb/BanyanDBConverter.java | 55 ++--
.../plugin/banyandb/BanyanDBIndexInstaller.java | 47 +--
.../plugin/banyandb/BanyanDBMetricsDAO.java | 1 -
.../plugin/banyandb/BanyanDBStorageClient.java | 30 +-
.../plugin/banyandb/BanyanDBStorageProvider.java | 21 +-
.../storage/plugin/banyandb/MetadataRegistry.java | 319 +++++++++++++--------
.../storage/plugin/banyandb/StreamMetadata.java | 71 -----
.../banyandb/measure/BanyanDBMetadataQueryDAO.java | 7 +-
.../measure/BanyanDBProfileTaskQueryDAO.java | 2 +-
...kQueryDAO.java => BanyanDBServiceLabelDAO.java} | 12 +-
.../banyandb/stream/AbstractBanyanDBDAO.java | 30 +-
.../banyandb/stream/BanyanDBAlarmQueryDAO.java | 13 +-
.../plugin/banyandb/stream/BanyanDBBatchDAO.java | 1 -
.../stream/BanyanDBBrowserLogQueryDAO.java | 16 +-
.../BanyanDBEBPFProfilingDataDAO.java} | 15 +-
.../BanyanDBEBPFProfilingScheduleQueryDAO.java} | 15 +-
.../BanyanDBEBPFProfilingTaskDAO.java} | 16 +-
.../banyandb/stream/BanyanDBLogQueryDAO.java | 32 +--
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 28 +-
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 70 +++--
.../plugin/banyandb/stream/BanyanDBRecordDAO.java | 9 +-
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 27 +-
23 files changed, 396 insertions(+), 443 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
index e2c9ae7817..7546099e4b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/pom.xml
@@ -21,7 +21,7 @@
<parent>
<artifactId>server-storage-plugin</artifactId>
<groupId>org.apache.skywalking</groupId>
- <version>9.0.0-SNAPSHOT</version>
+ <version>9.1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
index 565245ad55..41237f1f3b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConverter.java
@@ -19,11 +19,13 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.model.v1.BanyandbModel;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
-import org.apache.skywalking.banyandb.v1.client.SerializableTag;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
import org.apache.skywalking.banyandb.v1.client.TagAndValue;
+import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
+import org.apache.skywalking.banyandb.v1.client.metadata.Serializable;
import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.type.Convert2Entity;
import org.apache.skywalking.oap.server.core.storage.type.Convert2Storage;
@@ -35,16 +37,11 @@ import java.util.function.Function;
public class BanyanDBConverter {
@RequiredArgsConstructor
public static class StreamToEntity implements Convert2Entity {
- private final StreamMetadata metadata;
private final RowEntity rowEntity;
@Override
public Object get(String fieldName) {
- final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
- if (metadata == null) {
- return null;
- }
- return rowEntity.getValue(metadata.getTagFamilyName(), metadata.getTagSpec().getTagName());
+ return rowEntity.getTagValue(fieldName);
}
@Override
@@ -53,9 +50,9 @@ public class BanyanDBConverter {
}
}
+ @Slf4j
@RequiredArgsConstructor
public static class StreamToStorage implements Convert2Storage<StreamWrite> {
- private final StreamMetadata metadata;
private final StreamWrite streamWrite;
@Override
@@ -64,41 +61,28 @@ public class BanyanDBConverter {
if (Record.TIME_BUCKET.equals(fieldName)) {
return;
}
- final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
- if (metadata == null) {
- return;
- }
- switch (metadata.getTagFamilyName()) {
- case StreamMetadata.TAG_FAMILY_DATA:
- this.streamWrite.dataTag(metadata.getTagIndex(), buildTag(fieldValue));
- break;
- case StreamMetadata.TAG_FAMILY_SEARCHABLE:
- this.streamWrite.searchableTag(metadata.getTagIndex(), buildTag(fieldValue));
- break;
- default:
- throw new IllegalStateException("tag family is not supported");
+ try {
+ this.streamWrite.tag(fieldName, buildTag(fieldValue));
+ } catch (BanyanDBException ex) {
+ log.error("fail to add tag", ex);
}
}
- private SerializableTag<BanyandbModel.TagValue> buildTag(Object value) {
+ private Serializable<BanyandbModel.TagValue> buildTag(Object value) {
if (Integer.class.equals(value.getClass()) || Long.class.equals(value.getClass())) {
- return TagAndValue.longField((long) value);
+ return TagAndValue.longTagValue((long) value);
} else if (String.class.equals(value.getClass())) {
- return TagAndValue.stringField((String) value);
+ return TagAndValue.stringTagValue((String) value);
}
throw new IllegalStateException(value.getClass() + " is not supported");
}
@Override
public void accept(String fieldName, byte[] fieldValue) {
- final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
- if (metadata == null) {
- return;
- }
- if (StreamMetadata.TAG_FAMILY_SEARCHABLE.equals(metadata.getTagFamilyName())) {
- this.streamWrite.searchableTag(metadata.getTagIndex(), TagAndValue.binaryField((fieldValue)));
- } else {
- throw new IllegalStateException("binary tag should not be store in the `data` family");
+ try {
+ this.streamWrite.tag(fieldName, TagAndValue.binaryTagValue(fieldValue));
+ } catch (BanyanDBException ex) {
+ log.error("fail to add tag", ex);
}
}
@@ -120,19 +104,12 @@ public class BanyanDBConverter {
@Override
public Object get(String fieldName) {
- final StreamMetadata.TagMetadata metadata = this.metadata.getTagDefinition().get(fieldName);
- if (metadata == null) {
- return null;
- }
// TODO: get an unmodifiable view of tag
return null;
}
@Override
public StreamWrite obtain() {
- if (metadata.isUseIdAsEntity()) {
- this.accept(StreamMetadata.ID, this.streamWrite.getElementID());
- }
return this.streamWrite;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index f4483b0a14..d894c9990a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -19,9 +19,10 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
-import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
+import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
+import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.banyandb.v1.client.metadata.Group;
+import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.StorageException;
@@ -43,25 +44,39 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
@Override
protected boolean isExists(Model model) throws StorageException {
- // TODO: get from BanyanDB and make a diff?
- return false;
+ final MetadataRegistry.PartialMetadata metadata = MetadataRegistry.INSTANCE.parseMetadata(model);
+ try {
+ final BanyanDBClient c = ((BanyanDBStorageClient) this.client).client;
+ // first check group
+ Group g = metadata.getOrCreateGroup(c);
+ if (g == null) {
+ throw new StorageException("fail to create group " + metadata.getGroup());
+ }
+ log.info("group {} created", g.name());
+ // then check entity schema
+ return metadata.findRemoteSchema(c).isPresent();
+ } catch (BanyanDBException ex) {
+ throw new StorageException("fail to check existence", ex);
+ }
}
@Override
protected void createTable(Model model) throws StorageException {
- if (model.isTimeSeries() && model.isRecord()) { // stream
- StreamMetadata metaInfo = MetadataRegistry.INSTANCE.registerModel(model, this.configService);
- if (metaInfo != null) {
- log.info("install index {}", model.getName());
- ((BanyanDBStorageClient) client).define(
- new Group(metaInfo.getGroup(), Catalog.STREAM, 2, 10, Duration.ofDays(7))
- );
- ((BanyanDBStorageClient) client).define(metaInfo);
+ try {
+ if (model.isTimeSeries() && model.isRecord()) { // stream
+ Stream stream = (Stream) MetadataRegistry.INSTANCE.registerModel(model, this.configService);
+ if (stream != null) {
+ log.info("install stream schema {}", model.getName());
+ ((BanyanDBStorageClient) client).define(stream);
+ }
+ } else if (model.isTimeSeries() && !model.isRecord()) { // measure
+ // TODO: dynamically register Measure
+ log.info("skip measure index {}", model.getName());
+ } else if (!model.isTimeSeries()) { // UITemplate
+ log.info("skip property index {}", model.getName());
}
- } else if (model.isTimeSeries() && !model.isRecord()) { // measure
- log.info("skip measure index {}", model.getName());
- } else if (!model.isTimeSeries()) { // UITemplate
- log.info("skip property index {}", model.getName());
+ } catch (BanyanDBException ex) {
+ throw new StorageException("fail to install schema", ex);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
index af54dd350f..4e84979498 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBMetricsDAO.java
@@ -2,7 +2,6 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import lombok.RequiredArgsConstructor;
import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
-import org.apache.skywalking.oap.server.core.analysis.record.Record;
import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index b1ca206110..dd92c6fd08 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -23,8 +23,7 @@ import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
-import org.apache.skywalking.banyandb.v1.client.metadata.Group;
-import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.client.healthcheck.DelegatedHealthChecker;
@@ -32,25 +31,17 @@ import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckab
import org.apache.skywalking.oap.server.library.util.HealthChecker;
import java.io.IOException;
-import java.util.Map;
-import java.util.concurrent.ConcurrentHashMap;
/**
* BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
* which implement {@link Client} and {@link HealthCheckable}.
*/
public class BanyanDBStorageClient implements Client, HealthCheckable {
- private final BanyanDBClient client;
- private final Map<String, Group> groupMap;
+ final BanyanDBClient client;
private final DelegatedHealthChecker healthChecker = new DelegatedHealthChecker();
public BanyanDBStorageClient(String host, int port) {
this.client = new BanyanDBClient(host, port);
- this.groupMap = new ConcurrentHashMap<>();
- }
-
- public Group define(Group group) {
- return groupMap.computeIfAbsent(group.getName(), s -> client.define(group));
}
@Override
@@ -63,22 +54,19 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
this.client.close();
}
- public StreamQueryResponse query(StreamQuery streamQuery) {
+ public StreamQueryResponse query(StreamQuery streamQuery) throws BanyanDBException {
try {
- StreamQueryResponse response = this.client.queryStreams(streamQuery);
+ StreamQueryResponse response = this.client.query(streamQuery);
this.healthChecker.health();
return response;
- } catch (Throwable t) {
- healthChecker.unHealth(t);
- throw t;
+ } catch (BanyanDBException ex) {
+ healthChecker.unHealth(ex);
+ throw ex;
}
}
- public void define(StreamMetadata streamMetadata) {
- Stream stream = this.client.define(streamMetadata.getStream());
- if (stream != null) {
- this.client.defineIndexRules(stream, streamMetadata.getIndexRules().toArray(new IndexRule[]{}));
- }
+ public void define(Stream stream) throws BanyanDBException {
+ this.client.define(stream);
}
public void write(StreamWrite streamWrite) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 922bf2296a..88a81d3372 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -28,9 +28,13 @@ import org.apache.skywalking.oap.server.core.storage.StorageModule;
import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
import org.apache.skywalking.oap.server.core.storage.model.ModelCreator;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingDataDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingScheduleDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingTaskDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IServiceLabelDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileTaskLogQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IAggregationQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IAlarmQueryDAO;
import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
@@ -50,11 +54,15 @@ import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDB
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBMetadataQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBNetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBServiceLabelDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBTopologyQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.measure.BanyanDBUITemplateManagementDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBAlarmQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBatchDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBBrowserLogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEBPFProfilingDataDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEBPFProfilingScheduleQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBEBPFProfilingTaskDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBHistoryDeleteDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.stream.BanyanDBProfileTaskLogQueryDAO;
@@ -106,13 +114,16 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.registerServiceImplementation(ILogQueryDAO.class, new BanyanDBLogQueryDAO(client));
this.registerServiceImplementation(IProfileTaskQueryDAO.class, new BanyanDBProfileTaskQueryDAO());
this.registerServiceImplementation(IProfileTaskLogQueryDAO.class, new BanyanDBProfileTaskLogQueryDAO(client, this.config.getFetchTaskLogMaxSize()));
- this.registerServiceImplementation(
- IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
+ this.registerServiceImplementation(IProfileThreadSnapshotQueryDAO.class, new BanyanDBProfileThreadSnapshotQueryDAO(client));
this.registerServiceImplementation(UITemplateManagementDAO.class, new BanyanDBUITemplateManagementDAO());
this.registerServiceImplementation(IEventQueryDAO.class, new BanyanDBEventQueryDAO());
this.registerServiceImplementation(ITopologyQueryDAO.class, new BanyanDBTopologyQueryDAO());
+ this.registerServiceImplementation(IEBPFProfilingTaskDAO.class, new BanyanDBEBPFProfilingTaskDAO());
+ this.registerServiceImplementation(IEBPFProfilingDataDAO.class, new BanyanDBEBPFProfilingDataDAO());
+ this.registerServiceImplementation(IEBPFProfilingScheduleDAO.class, new BanyanDBEBPFProfilingScheduleQueryDAO());
// TODO: metrics
+ this.registerServiceImplementation(IServiceLabelDAO.class, new BanyanDBServiceLabelDAO());
this.registerServiceImplementation(IHistoryDeleteDAO.class, new BanyanDBHistoryDeleteDAO());
this.registerServiceImplementation(IMetricsQueryDAO.class, new BanyanDBMetricsQueryDAO());
this.registerServiceImplementation(IAggregationQueryDAO.class, new BanyanDBAggregationQueryDAO());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
index 83af6da867..b8925c7529 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/MetadataRegistry.java
@@ -18,10 +18,17 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-import com.google.common.collect.ImmutableSet;
-import org.apache.skywalking.banyandb.common.v1.BanyandbCommon;
-import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase;
+import io.grpc.Status;
+import lombok.Data;
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
+import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
+import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
+import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
+import org.apache.skywalking.banyandb.v1.client.metadata.Group;
import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.NamedSchema;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
import org.apache.skywalking.oap.server.core.alarm.AlarmRecord;
@@ -33,6 +40,7 @@ import org.apache.skywalking.oap.server.core.storage.model.Model;
import org.apache.skywalking.oap.server.core.storage.model.ModelColumn;
import org.apache.skywalking.oap.server.library.util.StringUtil;
+import javax.annotation.Nullable;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@@ -40,173 +48,230 @@ import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
-import java.util.Set;
+import java.util.Optional;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Function;
import java.util.stream.Collectors;
+@Slf4j
public enum MetadataRegistry {
INSTANCE;
- private final Map<String, StreamMetadata> streams = new HashMap<>();
+ private final Map<String, PartialMetadata> registry = new ConcurrentHashMap<>();
- public StreamMetadata registerModel(Model model, ConfigService configService) {
- BanyandbDatabase.Stream pbStream = parseStreamFromModel(model, configService);
+ public NamedSchema<?> registerModel(Model model, ConfigService configService) {
+ PartialMetadata partialMetadata = parseMetadata(model);
+ final Stream.Builder builder = Stream.create(partialMetadata.getGroup(), partialMetadata.getName());
+ Map<String, ModelColumn> modelColumnMap = model.getColumns().stream()
+ .collect(Collectors.toMap(modelColumn -> modelColumn.getColumnName().getStorageName(), Function.identity()));
+ // parse and set sharding keys
+ builder.setEntityRelativeTags(parseEntityNames(modelColumnMap));
+ // parse and set tag families, which contains tag specs
+ List<TagFamilySpec> specs = parseTagFamilySpecs(model, partialMetadata, configService);
+ builder.addTagFamilies(specs);
+ // parse and add index definition
+ builder.addIndexes(parseIndexRules(specs, partialMetadata.indexFamily(), modelColumnMap));
- final boolean useIdAsEntity = pbStream.getEntity().getTagNamesCount() == 1 &&
- StreamMetadata.ID.equals(pbStream.getEntity().getTagNames(0));
+ registry.put(model.getName(), partialMetadata);
+ return builder.build();
+ }
- final Stream stream = new Stream(pbStream.getMetadata().getGroup(), pbStream.getMetadata().getName());
+ public PartialMetadata findSchema(final String name) {
+ return this.registry.get(name);
+ }
+ List<IndexRule> parseIndexRules(List<TagFamilySpec> specs, String indexTagFamily, Map<String, ModelColumn> modelColumnMap) {
List<IndexRule> indexRules = new ArrayList<>();
- Set<String> entityNameSet = ImmutableSet.copyOf(pbStream.getEntity().getTagNamesList());
- stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
-
- Map<String, StreamMetadata.TagMetadata> tagDefinition = new HashMap<>();
-
- for (BanyandbDatabase.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
- final TagFamilySpec tagFamilySpec = TagFamilySpec.fromProtobuf(pbTagFamilySpec);
- stream.addTagFamilySpec(tagFamilySpec);
-
- int tagIndex = 0;
- for (final TagFamilySpec.TagSpec tagSpec : tagFamilySpec.getTagSpecs()) {
- // register tag
- tagDefinition.put(tagSpec.getTagName(), new StreamMetadata.TagMetadata(tagFamilySpec.getTagFamilyName(), tagSpec, tagIndex++));
-
- // if the tag family equals to "searchable", build index rules
- if (tagFamilySpec.getTagFamilyName().equals(StreamMetadata.TAG_FAMILY_SEARCHABLE)) {
- // check if this spec exists in the entity names
- if (entityNameSet.contains(tagSpec.getTagName())) {
- continue;
- }
- BanyandbDatabase.IndexRule pbIndexRule = parseIndexRuleFromTagSpec(pbStream.getMetadata(), tagSpec);
- IndexRule indexRule = IndexRule.fromProtobuf(pbIndexRule);
- indexRules.add(indexRule);
+ for (final TagFamilySpec spec : specs) {
+ if (!indexTagFamily.equals(spec.tagFamilyName())) {
+ continue;
+ }
+ for (final TagFamilySpec.TagSpec tagSpec : spec.tagSpecs()) {
+ final String tagName = tagSpec.getTagName();
+ // TODO: we need to add support index type in the OAP core
+ // Currently, we only register INVERTED type
+ final ModelColumn modelColumn = modelColumnMap.get(tagName);
+ // if it is null, it must be a user-defined tag
+ if (modelColumn == null) {
+ indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES));
+ continue;
+ }
+ if (modelColumn.getBanyanDBExtension().isGlobalIndexing()) {
+ indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.GLOBAL));
+ } else {
+ indexRules.add(IndexRule.create(tagName, IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES));
}
}
}
-
- StreamMetadata streamMetadata = StreamMetadata.builder().model(model).stream(stream)
- .tagDefinition(tagDefinition)
- .indexRules(indexRules)
- .group(pbStream.getMetadata().getGroup())
- .useIdAsEntity(useIdAsEntity)
- .build();
- streams.put(model.getName(), streamMetadata);
- return streamMetadata;
- }
-
- public StreamMetadata findStreamMetadata(final String name) {
- return this.streams.get(name);
+ return indexRules;
}
- private BanyandbDatabase.Stream parseStreamFromModel(Model model, ConfigService configService) {
+ /**
+ * Parse sharding keys from the {@link Model}
+ *
+ * @param modelColumnMap the mapping between column storageName and {@link ModelColumn}
+ * @return a list of column names in strict order
+ */
+ List<String> parseEntityNames(Map<String, ModelColumn> modelColumnMap) {
List<ModelColumn> shardingColumns = new ArrayList<>();
+ for (final ModelColumn col : modelColumnMap.values()) {
+ if (col.getBanyanDBExtension().isShardingKey()) {
+ shardingColumns.add(col);
+ }
+ }
+ return shardingColumns.stream()
+ .sorted(Comparator.comparingInt(col -> col.getBanyanDBExtension().getShardingKeyIdx()))
+ .map(col -> col.getColumnName().getName())
+ .collect(Collectors.toList());
+ }
- List<BanyandbDatabase.TagSpec> searchableTagsSpecs = new ArrayList<>();
- List<BanyandbDatabase.TagSpec> dataTagsSpecs = new ArrayList<>();
- for (final ModelColumn modelColumn : model.getColumns()) {
- if (modelColumn.getShardingKeyIdx() > -1) {
- shardingColumns.add(modelColumn);
+ List<TagFamilySpec> parseTagFamilySpecs(Model model, PartialMetadata metadata, ConfigService configService) {
+ Map<String, TagFamilySpec.Builder> builderMap = new HashMap<>();
+ for (final ModelColumn col : model.getColumns()) {
+ final TagFamilySpec.TagSpec tagSpec = parseTagSpec(col);
+ if (tagSpec == null) {
+ continue;
}
- if (modelColumn.isIndexOnly()) {
- // skip
- } else if (modelColumn.isStorageOnly()) {
- dataTagsSpecs.add(parseTagSpecFromModelColumn(modelColumn));
+ if (col.shouldIndex()) {
+ builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpec(tagSpec);
} else {
- searchableTagsSpecs.add(parseTagSpecFromModelColumn(modelColumn));
+ builderMap.computeIfAbsent(metadata.nonIndexFamily(), TagFamilySpec::create).addTagSpec(tagSpec);
}
}
- Set<String> entities = shardingColumns.stream()
- .sorted(Comparator.comparingInt(ModelColumn::getShardingKeyIdx))
- .map(modelColumn -> modelColumn.getColumnName().getStorageName())
- .collect(Collectors.toSet());
-
- if (entities.isEmpty()) {
- // if sharding keys are not defined, we have to use ID
- entities = Collections.singleton(StreamMetadata.ID);
- // append ID
- searchableTagsSpecs.add(BanyandbDatabase.TagSpec.newBuilder()
- .setName(StreamMetadata.ID)
- .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build());
- }
-
// add all user-defined indexed tags to the end of the "searchable" family
if (SegmentRecord.INDEX_NAME.equals(model.getName())) {
- searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableTracesTags()));
+ builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableTracesTags()));
} else if (LogRecord.INDEX_NAME.equals(model.getName())) {
- searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableLogsTags()));
+ builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableLogsTags()));
} else if (AlarmRecord.INDEX_NAME.equals(model.getName())) {
- searchableTagsSpecs.addAll(parseTagSpecsFromConfiguration(configService.getSearchableAlarmTags()));
+ builderMap.computeIfAbsent(metadata.indexFamily(), TagFamilySpec::create).addTagSpecs(parseExtraTagSpecs(configService.getSearchableAlarmTags()));
}
- String group = "default-stream";
- if (model.isSuperDataset()) {
- // for superDataset, we should use separate group
- group = model.getName() + "-stream";
- }
+ return builderMap.values().stream().map(TagFamilySpec.Builder::build).collect(Collectors.toList());
+ }
- return BanyandbDatabase.Stream.newBuilder()
- .addTagFamilies(BanyandbDatabase.TagFamilySpec.newBuilder()
- .setName(StreamMetadata.TAG_FAMILY_DATA)
- .addAllTags(dataTagsSpecs)
- .build())
- .addTagFamilies(BanyandbDatabase.TagFamilySpec.newBuilder()
- .setName(StreamMetadata.TAG_FAMILY_SEARCHABLE)
- .addAllTags(searchableTagsSpecs)
- .build())
- .setEntity(BanyandbDatabase.Entity.newBuilder()
- .addAllTagNames(entities)
- .build())
- .setMetadata(BanyandbCommon.Metadata.newBuilder()
- .setGroup(group)
- .setName(model.getName())
- .build())
- .build();
+ /**
+ * Extract extra tags from Configuration.
+ * They are for tags defined for {@link SegmentRecord}, {@link LogRecord} and {@link AlarmRecord}.
+ *
+ * @param tags a series of tags joint by comma
+ * @return a list of {@link org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec.TagSpec} generated from input
+ */
+ private List<TagFamilySpec.TagSpec> parseExtraTagSpecs(String tags) {
+ if (StringUtil.isEmpty(tags)) {
+ return Collections.emptyList();
+ }
+ String[] tagsArray = tags.split(",");
+ if (tagsArray.length == 0) {
+ return Collections.emptyList();
+ }
+ return Arrays.stream(tagsArray)
+ .map(TagFamilySpec.TagSpec::newStringTag)
+ .collect(Collectors.toList());
}
- private BanyandbDatabase.TagSpec parseTagSpecFromModelColumn(ModelColumn modelColumn) {
+ /**
+ * Parse TagSpec from {@link ModelColumn}
+ *
+ * @param modelColumn the column in the model to be parsed
+ * @return a typed tag spec
+ */
+ @Nullable
+ private TagFamilySpec.TagSpec parseTagSpec(ModelColumn modelColumn) {
final Class<?> clazz = modelColumn.getType();
if (String.class.equals(clazz)) {
- return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
- .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build();
+ return TagFamilySpec.TagSpec.newStringTag(modelColumn.getColumnName().getStorageName());
} else if (int.class.equals(clazz) || long.class.equals(clazz)) {
- return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
- .setType(BanyandbDatabase.TagType.TAG_TYPE_INT).build();
+ return TagFamilySpec.TagSpec.newIntTag(modelColumn.getColumnName().getStorageName());
} else if (byte[].class.equals(clazz) || DataTable.class.equals(clazz)) {
- return BanyandbDatabase.TagSpec.newBuilder().setName(modelColumn.getColumnName().getStorageName())
- .setType(BanyandbDatabase.TagType.TAG_TYPE_DATA_BINARY).build();
+ return TagFamilySpec.TagSpec.newBinaryTag(modelColumn.getColumnName().getStorageName());
} else {
+ // TODO: we skip all tags with type of List<String>
+ if ("tags".equals(modelColumn.getColumnName().getStorageName())) {
+ return null;
+ }
throw new IllegalStateException("type " + modelColumn.getType().toString() + " is not supported");
}
}
- private List<BanyandbDatabase.TagSpec> parseTagSpecsFromConfiguration(String tags) {
- if (StringUtil.isEmpty(tags)) {
- return Collections.emptyList();
+ public PartialMetadata parseMetadata(Model model) {
+ if (model.isRecord()) {
+ String group = "stream-default";
+ if (model.isSuperDataset()) {
+ // for superDataset, we should use separate group
+ group = "stream-" + model.getName();
+ }
+ return new PartialMetadata(group, model.getName(), Kind.STREAM);
}
- String[] tagsArray = tags.split(",");
- if (tagsArray.length == 0) {
- return Collections.emptyList();
+ return new PartialMetadata("measure-default", model.getName(), Kind.MEASURE);
+ }
+
+ @RequiredArgsConstructor
+ @Data
+ public static class PartialMetadata {
+ private final String group;
+ private final String name;
+ private final Kind kind;
+
+ public Optional<NamedSchema<?>> findRemoteSchema(BanyanDBClient client) throws BanyanDBException {
+ try {
+ switch (kind) {
+ case STREAM:
+ return Optional.ofNullable(client.findStream(this.group, this.name));
+ case MEASURE:
+ return Optional.ofNullable(client.findMeasure(this.group, this.name));
+ default:
+ throw new IllegalStateException("should not reach here");
+ }
+ } catch (BanyanDBException ex) {
+ if (ex.getStatus().equals(Status.Code.NOT_FOUND)) {
+ return Optional.empty();
+ }
+
+ throw ex;
+ }
+ }
+
+ public Group getOrCreateGroup(BanyanDBClient client) throws BanyanDBException {
+ Group g = client.findGroup(this.group);
+ if (g != null) {
+ return g;
+ }
+ switch (kind) {
+ case STREAM:
+ return client.define(Group.create(this.group, Catalog.STREAM, 2, 0, Duration.ofDays(7)));
+ case MEASURE:
+ return client.define(Group.create(this.group, Catalog.MEASURE, 2, 12, Duration.ofDays(7)));
+ default:
+ throw new IllegalStateException("should not reach here");
+ }
+ }
+
+ public String indexFamily() {
+ switch (kind) {
+ case MEASURE:
+ return "default";
+ case STREAM:
+ return "searchable";
+ default:
+ throw new IllegalStateException("should not reach here");
+ }
+ }
+
+ public String nonIndexFamily() {
+ switch (kind) {
+ case MEASURE:
+ return null;
+ case STREAM:
+ return "binary";
+ default:
+ throw new IllegalStateException("should not reach here");
+ }
}
- return Arrays.stream(tagsArray)
- .map(tagName -> BanyandbDatabase.TagSpec.newBuilder().setName(tagName)
- .setType(BanyandbDatabase.TagType.TAG_TYPE_STRING).build())
- .collect(Collectors.toList());
}
- private BanyandbDatabase.IndexRule parseIndexRuleFromTagSpec(BanyandbCommon.Metadata metadata, TagFamilySpec.TagSpec tagSpec) {
- // In SkyWalking, only "trace_id" should be stored as a global index
- BanyandbDatabase.IndexRule.Location loc = "trace_id".equals(tagSpec.getTagName()) ?
- BanyandbDatabase.IndexRule.Location.LOCATION_GLOBAL :
- BanyandbDatabase.IndexRule.Location.LOCATION_SERIES;
-
- return BanyandbDatabase.IndexRule.newBuilder()
- .setMetadata(BanyandbCommon.Metadata.newBuilder()
- .setName(tagSpec.getTagName()).setGroup(metadata.getGroup()))
- .setLocation(loc)
- .addTags(tagSpec.getTagName())
- // TODO: support TYPE_TREE
- .setType(BanyandbDatabase.IndexRule.Type.TYPE_INVERTED)
- .build();
+ public enum Kind {
+ MEASURE, STREAM;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java
deleted file mode 100644
index 814f5f62e2..0000000000
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetadata.java
+++ /dev/null
@@ -1,71 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.storage.plugin.banyandb;
-
-import lombok.Builder;
-import lombok.Data;
-import lombok.Getter;
-import lombok.extern.slf4j.Slf4j;
-import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
-import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
-import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
-import org.apache.skywalking.oap.server.core.storage.model.Model;
-
-import java.util.List;
-import java.util.Map;
-
-@Getter
-@Builder
-@Slf4j
-public class StreamMetadata {
- public static final String TAG_FAMILY_SEARCHABLE = "searchable";
- public static final String TAG_FAMILY_DATA = "data";
-
- public static final String ID = "id";
-
- private final Model model;
-
- private final Map<String, TagMetadata> tagDefinition;
-
- /**
- * Group of the stream
- */
- private final String group;
- /**
- * Spec of the stream
- */
- private final Stream stream;
- /**
- * Index rules attached to the stream
- */
- private final List<IndexRule> indexRules;
-
- private final int dataFamilySize;
- private final int searchableFamilySize;
-
- private final boolean useIdAsEntity;
-
- @Getter
- @Data
- public static class TagMetadata {
- private final String tagFamilyName;
- private final TagFamilySpec.TagSpec tagSpec;
- private final int tagIndex;
- }
-}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
index 5833c98c56..bff4fc00be 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBMetadataQueryDAO.java
@@ -55,10 +55,15 @@ public class BanyanDBMetadataQueryDAO implements IMetadataQueryDAO {
}
@Override
- public List<Process> listProcesses(String serviceId, String instanceId) throws IOException {
+ public List<Process> listProcesses(String serviceId, String instanceId, String agentId, long lastPingStartTimeBucket, long lastPingEndTimeBucket) throws IOException {
return Collections.emptyList();
}
+ @Override
+ public long getProcessesCount(String serviceId, String instanceId, String agentId, long lastPingStartTimeBucket, long lastPingEndTimeBucket) throws IOException {
+ return 0;
+ }
+
@Override
public Process getProcess(String processId) throws IOException {
return null;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
index 0c6e1ebfd1..690bc6640d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileTaskQueryDAO;
import java.io.IOException;
import java.util.Collections;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBServiceLabelDAO.java
similarity index 67%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBServiceLabelDAO.java
index 0c6e1ebfd1..295af6f75a 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBServiceLabelDAO.java
@@ -18,21 +18,15 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IServiceLabelDAO;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+public class BanyanDBServiceLabelDAO implements IServiceLabelDAO {
@Override
- public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ public List<String> queryAllLabels(String serviceId) throws IOException {
return Collections.emptyList();
}
-
- @Override
- public ProfileTask getById(String id) throws IOException {
- return null;
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index 0f093e396b..c40ac35924 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -22,12 +22,14 @@ import org.apache.skywalking.banyandb.v1.client.PairQueryCondition;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+import org.apache.skywalking.banyandb.v1.client.grpc.exception.BanyanDBException;
import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
+import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
+import java.io.IOException;
import java.time.Instant;
-import java.util.List;
+import java.util.Set;
public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageClient> {
private static final Instant UPPER_BOUND = Instant.ofEpochSecond(0, Long.MAX_VALUE);
@@ -38,41 +40,45 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
super(client);
}
- protected StreamQueryResponse query(StreamMetadata metadata, List<String> searchableTags, QueryBuilder builder) {
+ protected StreamQueryResponse query(MetadataRegistry.PartialMetadata metadata, Set<String> searchableTags, QueryBuilder builder) throws IOException {
return this.query(metadata, searchableTags, null, builder);
}
- protected StreamQueryResponse query(StreamMetadata metadata, List<String> searchableTags, TimestampRange timestampRange,
- QueryBuilder builder) {
+ protected StreamQueryResponse query(MetadataRegistry.PartialMetadata metadata, Set<String> searchableTags, TimestampRange timestampRange,
+ QueryBuilder builder) throws IOException {
final StreamQuery query;
if (timestampRange == null) {
- query = new StreamQuery(metadata.getGroup(), metadata.getModel().getName(), LARGEST_TIME_RANGE, searchableTags);
+ query = new StreamQuery(metadata.getGroup(), metadata.getName(), LARGEST_TIME_RANGE, searchableTags);
} else {
- query = new StreamQuery(metadata.getGroup(), metadata.getModel().getName(), timestampRange, searchableTags);
+ query = new StreamQuery(metadata.getGroup(), metadata.getName(), timestampRange, searchableTags);
}
builder.apply(query);
- return getClient().query(query);
+ try {
+ return getClient().query(query);
+ } catch (BanyanDBException ex) {
+ throw new IOException(ex);
+ }
}
protected abstract static class QueryBuilder {
abstract void apply(final StreamQuery query);
protected PairQueryCondition<Long> eq(String name, long value) {
- return PairQueryCondition.LongQueryCondition.eq(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.eq(name, value);
}
protected PairQueryCondition<Long> lte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.le(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.le(name, value);
}
protected PairQueryCondition<Long> gte(String name, long value) {
- return PairQueryCondition.LongQueryCondition.ge(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.LongQueryCondition.ge(name, value);
}
protected PairQueryCondition<String> eq(String name, String value) {
- return PairQueryCondition.StringQueryCondition.eq(StreamMetadata.TAG_FAMILY_SEARCHABLE, name, value);
+ return PairQueryCondition.StringQueryCondition.eq(name, value);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
index 0b7693d16c..b2a43abff7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBAlarmQueryDAO.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
@@ -34,7 +34,6 @@ import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.List;
@@ -45,8 +44,8 @@ import java.util.Objects;
* which can be used to build a {@link org.apache.skywalking.oap.server.core.query.type.AlarmMessage}
*/
public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarmQueryDAO {
- private final StreamMetadata alarmRecordMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(AlarmRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata alarmRecordMetadata =
+ MetadataRegistry.INSTANCE.findSchema(AlarmRecord.INDEX_NAME);
public BanyanDBAlarmQueryDAO(BanyanDBStorageClient client) {
super(client);
@@ -60,13 +59,11 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
}
StreamQueryResponse resp = query(alarmRecordMetadata,
- ImmutableList.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME),
+ ImmutableSet.of(AlarmRecord.SCOPE, AlarmRecord.START_TIME, AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA),
tsRange,
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(AlarmRecord.ID0, AlarmRecord.ID1, AlarmRecord.ALARM_MESSAGE, AlarmRecord.TAGS_RAW_DATA));
-
if (Objects.nonNull(scopeId)) {
query.appendCondition(eq(AlarmRecord.SCOPE, (long) scopeId));
}
@@ -90,7 +87,7 @@ public class BanyanDBAlarmQueryDAO extends AbstractBanyanDBDAO implements IAlarm
for (final RowEntity rowEntity : resp.getElements()) {
AlarmRecord.Builder builder = new AlarmRecord.Builder();
AlarmRecord alarmRecord = builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(this.alarmRecordMetadata, rowEntity)
+ new BanyanDBConverter.StreamToEntity(rowEntity)
);
AlarmMessage message = new AlarmMessage();
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
index 040b031828..e17b8b66ac 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBatchDAO.java
@@ -31,7 +31,6 @@ import java.util.concurrent.CompletableFuture;
public class BanyanDBBatchDAO extends AbstractDAO<BanyanDBStorageClient> implements IBatchDAO {
private StreamBulkWriteProcessor bulkProcessor;
-
private final int maxBulkSize;
private final int flushInterval;
private final int concurrency;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
index d384210cd9..2c412cf7dc 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBBrowserLogQueryDAO.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import com.google.protobuf.InvalidProtocolBufferException;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
@@ -34,18 +34,16 @@ import org.apache.skywalking.oap.server.core.storage.query.IBrowserLogQueryDAO;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
-import java.util.Collections;
import java.util.Objects;
/**
* {@link org.apache.skywalking.oap.server.core.browser.manual.errorlog.BrowserErrorLogRecord} is a stream
*/
public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements IBrowserLogQueryDAO {
- private final StreamMetadata browserErrorLogRecordMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(BrowserErrorLogRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata browserErrorLogRecordMetadata =
+ MetadataRegistry.INSTANCE.findSchema(BrowserErrorLogRecord.INDEX_NAME);
public BanyanDBBrowserLogQueryDAO(BanyanDBStorageClient client) {
super(client);
@@ -58,13 +56,12 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
tsRange = new TimestampRange(TimeBucket.getTimestamp(startSecondTB), TimeBucket.getTimestamp(endSecondTB));
}
- StreamQueryResponse resp = query(browserErrorLogRecordMetadata, ImmutableList.of(BrowserErrorLogRecord.SERVICE_ID,
+ StreamQueryResponse resp = query(browserErrorLogRecordMetadata, ImmutableSet.of(BrowserErrorLogRecord.SERVICE_ID,
BrowserErrorLogRecord.SERVICE_VERSION_ID,
BrowserErrorLogRecord.PAGE_PATH_ID,
- BrowserErrorLogRecord.ERROR_CATEGORY), tsRange, new QueryBuilder() {
+ BrowserErrorLogRecord.ERROR_CATEGORY, BrowserErrorLogRecord.DATA_BINARY), tsRange, new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(BrowserErrorLogRecord.DATA_BINARY));
query.appendCondition(eq(BrowserErrorLogRecord.SERVICE_ID, serviceId));
if (StringUtil.isNotEmpty(serviceVersionId)) {
@@ -88,8 +85,7 @@ public class BanyanDBBrowserLogQueryDAO extends AbstractBanyanDBDAO implements I
logs.setTotal(resp.size());
for (final RowEntity rowEntity : resp.getElements()) {
- final byte[] dataBinary =
- rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, BrowserErrorLogRecord.DATA_BINARY);
+ final byte[] dataBinary = rowEntity.getTagValue(BrowserErrorLogRecord.DATA_BINARY);
if (dataBinary != null && dataBinary.length > 0) {
BrowserErrorLog log = parserDataBinary(dataBinary);
logs.getLogs().add(log);
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingDataDAO.java
similarity index 67%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingDataDAO.java
index 0c6e1ebfd1..4036fa771c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingDataDAO.java
@@ -16,23 +16,18 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.profiling.ebpf.storage.EBPFProfilingDataRecord;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingDataDAO;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+public class BanyanDBEBPFProfilingDataDAO implements IEBPFProfilingDataDAO {
@Override
- public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ public List<EBPFProfilingDataRecord> queryData(List<String> scheduleIdList, long beginTime, long endTime) throws IOException {
return Collections.emptyList();
}
-
- @Override
- public ProfileTask getById(String id) throws IOException {
- return null;
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingScheduleQueryDAO.java
similarity index 67%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingScheduleQueryDAO.java
index 0c6e1ebfd1..e6e4d529e4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingScheduleQueryDAO.java
@@ -16,23 +16,18 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.query.type.EBPFProfilingSchedule;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingScheduleDAO;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+public class BanyanDBEBPFProfilingScheduleQueryDAO implements IEBPFProfilingScheduleDAO {
@Override
- public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ public List<EBPFProfilingSchedule> querySchedules(String taskId, long startTimeBucket, long endTimeBucket) throws IOException {
return Collections.emptyList();
}
-
- @Override
- public ProfileTask getById(String id) throws IOException {
- return null;
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingTaskDAO.java
similarity index 66%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingTaskDAO.java
index 0c6e1ebfd1..b0a59f3251 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/measure/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBEBPFProfilingTaskDAO.java
@@ -16,23 +16,19 @@
*
*/
-package org.apache.skywalking.oap.server.storage.plugin.banyandb.measure;
+package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import org.apache.skywalking.oap.server.core.query.type.ProfileTask;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
+import org.apache.skywalking.oap.server.core.profiling.ebpf.storage.EBPFProfilingTargetType;
+import org.apache.skywalking.oap.server.core.query.type.EBPFProfilingTask;
+import org.apache.skywalking.oap.server.core.storage.profiling.ebpf.IEBPFProfilingTaskDAO;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
-public class BanyanDBProfileTaskQueryDAO implements IProfileTaskQueryDAO {
+public class BanyanDBEBPFProfilingTaskDAO implements IEBPFProfilingTaskDAO {
@Override
- public List<ProfileTask> getTaskList(String serviceId, String endpointName, Long startTimeBucket, Long endTimeBucket, Integer limit) throws IOException {
+ public List<EBPFProfilingTask> queryTasks(List<String> serviceIdList, EBPFProfilingTargetType targetType, long taskStartTime, long latestUpdateTime) throws IOException {
return Collections.emptyList();
}
-
- @Override
- public ProfileTask getById(String id) throws IOException {
- return null;
- }
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
index 5cef08a726..c6097539c9 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBLogQueryDAO.java
@@ -18,7 +18,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import com.google.protobuf.InvalidProtocolBufferException;
import org.apache.skywalking.apm.network.logging.v3.LogTags;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
@@ -41,7 +41,6 @@ import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.library.util.StringUtil;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.List;
@@ -51,8 +50,8 @@ import java.util.Objects;
* {@link org.apache.skywalking.oap.server.core.analysis.manual.log.LogRecord} is a stream
*/
public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQueryDAO {
- private final StreamMetadata logRecordMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(LogRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata logRecordMetadata =
+ MetadataRegistry.INSTANCE.findSchema(LogRecord.INDEX_NAME);
public BanyanDBLogQueryDAO(BanyanDBStorageClient client) {
super(client);
@@ -66,8 +65,6 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
final QueryBuilder query = new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT, AbstractLogRecord.TAGS_RAW_DATA));
-
if (StringUtil.isNotEmpty(serviceId)) {
query.appendCondition(eq(AbstractLogRecord.SERVICE_ID, serviceId));
}
@@ -105,32 +102,31 @@ public class BanyanDBLogQueryDAO extends AbstractBanyanDBDAO implements ILogQuer
}
StreamQueryResponse resp = query(logRecordMetadata,
- ImmutableList.of(AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
+ ImmutableSet.of(AbstractLogRecord.SERVICE_ID, AbstractLogRecord.SERVICE_INSTANCE_ID,
AbstractLogRecord.ENDPOINT_ID, AbstractLogRecord.TRACE_ID, AbstractLogRecord.TRACE_SEGMENT_ID,
- AbstractLogRecord.SPAN_ID), tsRange, query);
+ AbstractLogRecord.SPAN_ID, AbstractLogRecord.CONTENT_TYPE, AbstractLogRecord.CONTENT,
+ AbstractLogRecord.TAGS_RAW_DATA), tsRange, query);
Logs logs = new Logs();
logs.setTotal(resp.size());
for (final RowEntity rowEntity : resp.getElements()) {
Log log = new Log();
- log.setServiceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.SERVICE_ID));
+ log.setServiceId(rowEntity.getTagValue(AbstractLogRecord.SERVICE_ID));
log.setServiceInstanceId(
- rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.SERVICE_INSTANCE_ID));
+ rowEntity.getTagValue(AbstractLogRecord.SERVICE_INSTANCE_ID));
log.setEndpointId(
- rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.ENDPOINT_ID));
+ rowEntity.getTagValue(AbstractLogRecord.ENDPOINT_ID));
if (log.getEndpointId() != null) {
log.setEndpointName(
IDManager.EndpointID.analysisId(log.getEndpointId()).getEndpointName());
}
- log.setTraceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.TRACE_ID));
- log.setTimestamp(((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE,
- AbstractLogRecord.TIMESTAMP)).longValue());
+ log.setTraceId(rowEntity.getTagValue(AbstractLogRecord.TRACE_ID));
+ log.setTimestamp(((Number) rowEntity.getTagValue(AbstractLogRecord.TIMESTAMP)).longValue());
log.setContentType(ContentType.instanceOf(
- ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA,
- AbstractLogRecord.CONTENT_TYPE)).intValue()));
- log.setContent(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, AbstractLogRecord.CONTENT));
- byte[] dataBinary = rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, AbstractLogRecord.TAGS_RAW_DATA);
+ ((Number) rowEntity.getTagValue(AbstractLogRecord.CONTENT_TYPE)).intValue()));
+ log.setContent(rowEntity.getTagValue(AbstractLogRecord.CONTENT));
+ byte[] dataBinary = rowEntity.getTagValue(AbstractLogRecord.TAGS_RAW_DATA);
if (dataBinary != null && dataBinary.length > 0) {
parserDataBinary(dataBinary, log.getTags());
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 4411957f69..6749118a25 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -18,28 +18,27 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
-import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
+import org.apache.skywalking.oap.server.core.profiling.trace.ProfileTaskLogRecord;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLog;
import org.apache.skywalking.oap.server.core.query.type.ProfileTaskLogOperationType;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.LinkedList;
import java.util.List;
/**
- * {@link org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord} is a stream
+ * {@link ProfileTaskLogRecord} is a stream
*/
public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implements IProfileTaskLogQueryDAO {
- private final StreamMetadata profileTaskLogRecord =
- MetadataRegistry.INSTANCE.findStreamMetadata(ProfileTaskLogRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata profileTaskLogRecord =
+ MetadataRegistry.INSTANCE.findSchema(ProfileTaskLogRecord.INDEX_NAME);
private final int queryMaxSize;
@@ -51,12 +50,11 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
StreamQueryResponse resp = query(profileTaskLogRecord,
- ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME, ProfileTaskLogRecord.INSTANCE_ID),
+ ImmutableSet.of(ProfileTaskLogRecord.OPERATION_TIME, ProfileTaskLogRecord.INSTANCE_ID,
+ ProfileTaskLogRecord.TASK_ID, ProfileTaskLogRecord.OPERATION_TYPE),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(ImmutableList.of(ProfileTaskLogRecord.TASK_ID,
- ProfileTaskLogRecord.OPERATION_TYPE));
query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
}
});
@@ -72,15 +70,13 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
private ProfileTaskLog parseTaskLog(RowEntity data) {
return ProfileTaskLog.builder()
.id(data.getId())
- .taskId(data.getValue(StreamMetadata.TAG_FAMILY_DATA, ProfileTaskLogRecord.TASK_ID))
+ .taskId(data.getTagValue(ProfileTaskLogRecord.TASK_ID))
.instanceId(
- data.getValue(StreamMetadata.TAG_FAMILY_DATA, ProfileTaskLogRecord.INSTANCE_ID))
+ data.getTagValue(ProfileTaskLogRecord.INSTANCE_ID))
.operationType(ProfileTaskLogOperationType.parse(
- ((Number) data.getValue(StreamMetadata.TAG_FAMILY_DATA,
- ProfileTaskLogRecord.OPERATION_TYPE)).intValue()))
+ ((Number) data.getTagValue(ProfileTaskLogRecord.OPERATION_TYPE)).intValue()))
.operationTime(
- ((Number) data.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE,
- ProfileTaskLogRecord.OPERATION_TIME)).longValue())
+ ((Number) data.getTagValue(ProfileTaskLogRecord.OPERATION_TIME)).longValue())
.build();
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index 4c6ac7ddcd..a0cacd0c39 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -18,20 +18,19 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
-import org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.profiling.trace.ProfileThreadSnapshotRecord;
import org.apache.skywalking.oap.server.core.query.type.BasicTrace;
-import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.core.storage.profiling.trace.IProfileThreadSnapshotQueryDAO;
import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.ArrayList;
@@ -49,11 +48,11 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
protected final ProfileThreadSnapshotRecord.Builder builder =
new ProfileThreadSnapshotRecord.Builder();
- private final StreamMetadata profileThreadSnapshotMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(ProfileThreadSnapshotRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata profileThreadSnapshotMetadata =
+ MetadataRegistry.INSTANCE.findSchema(ProfileThreadSnapshotRecord.INDEX_NAME);
- private final StreamMetadata segmentRecordMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(SegmentRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata segmentRecordMetadata =
+ MetadataRegistry.INSTANCE.findSchema(SegmentRecord.INDEX_NAME);
public BanyanDBProfileThreadSnapshotQueryDAO(BanyanDBStorageClient client) {
super(client);
@@ -62,7 +61,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<BasicTrace> queryProfiledSegments(String taskId) throws IOException {
StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
- ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
new QueryBuilder() {
@Override
@@ -78,14 +77,14 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
final List<String> segmentIds = new LinkedList<>();
for (final RowEntity rowEntity : resp.getElements()) {
- segmentIds.add(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, ProfileThreadSnapshotRecord.SEGMENT_ID));
+ segmentIds.add(rowEntity.getTagValue(ProfileThreadSnapshotRecord.SEGMENT_ID));
}
// TODO: support `IN` or `OR` logic operation in BanyanDB
List<BasicTrace> basicTraces = new ArrayList<>();
for (String segmentID : segmentIds) {
final StreamQueryResponse segmentRecordResp = query(segmentRecordMetadata,
- ImmutableList.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
+ ImmutableSet.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery traceQuery) {
@@ -97,15 +96,15 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
BasicTrace basicTrace = new BasicTrace();
basicTrace.setSegmentId(row.getId());
- basicTrace.setStart(String.valueOf(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)));
+ basicTrace.setStart(String.valueOf(row.getTagValue(SegmentRecord.START_TIME)));
basicTrace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.ENDPOINT_ID)
+ row.getTagValue(SegmentRecord.ENDPOINT_ID)
).getEndpointName());
- basicTrace.setDuration(((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ basicTrace.setDuration(((Number) row.getTagValue(SegmentRecord.LATENCY)).intValue());
basicTrace.setError(BooleanUtils.valueToBoolean(
- ((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue()
+ ((Number) row.getTagValue(SegmentRecord.IS_ERROR)).intValue()
));
- basicTrace.getTraceIds().add(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
+ basicTrace.getTraceIds().add(row.getTagValue(SegmentRecord.TRACE_ID));
basicTraces.add(basicTrace);
}
@@ -134,13 +133,12 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public List<ProfileThreadSnapshotRecord> queryRecords(String segmentId, int minSequence, int maxSequence) throws IOException {
StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
- ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
- ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
+ ProfileThreadSnapshotRecord.STACK_BINARY),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
-
query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
.appendCondition(lte(ProfileThreadSnapshotRecord.SEQUENCE, maxSequence))
.appendCondition(gte(ProfileThreadSnapshotRecord.SEQUENCE, minSequence));
@@ -150,7 +148,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
List<ProfileThreadSnapshotRecord> result = new ArrayList<>(maxSequence - minSequence);
for (final RowEntity rowEntity : resp.getElements()) {
ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(profileThreadSnapshotMetadata, rowEntity));
+ new BanyanDBConverter.StreamToEntity(rowEntity));
result.add(record);
}
@@ -160,11 +158,12 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
StreamQueryResponse resp = query(segmentRecordMetadata,
- ImmutableList.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.SERVICE_ID, SegmentRecord.SERVICE_INSTANCE_ID, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
+ ImmutableSet.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.SERVICE_ID,
+ SegmentRecord.SERVICE_INSTANCE_ID, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY,
+ SegmentRecord.START_TIME, SegmentRecord.DATA_BINARY),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(SegmentRecord.DATA_BINARY));
query.appendCondition(eq(SegmentRecord.INDEX_NAME, segmentId));
}
});
@@ -175,31 +174,30 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
final RowEntity rowEntity = resp.getElements().iterator().next();
final SegmentRecord segmentRecord = new SegmentRecord();
- segmentRecord.setSegmentId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.SEGMENT_ID));
- segmentRecord.setTraceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
- segmentRecord.setServiceId(rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.SERVICE_ID));
+ segmentRecord.setSegmentId(rowEntity.getTagValue(SegmentRecord.SEGMENT_ID));
+ segmentRecord.setTraceId(rowEntity.getTagValue(SegmentRecord.TRACE_ID));
+ segmentRecord.setServiceId(rowEntity.getTagValue(SegmentRecord.SERVICE_ID));
segmentRecord.setStartTime(
- ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)).longValue());
+ ((Number) rowEntity.getTagValue(SegmentRecord.START_TIME)).longValue());
segmentRecord.setLatency(
- ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ ((Number) rowEntity.getTagValue(SegmentRecord.LATENCY)).intValue());
segmentRecord.setIsError(
- ((Number) rowEntity.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue());
- byte[] dataBinary = rowEntity.getValue(StreamMetadata.TAG_FAMILY_DATA, SegmentRecord.DATA_BINARY);
+ ((Number) rowEntity.getTagValue(SegmentRecord.IS_ERROR)).intValue());
+ byte[] dataBinary = rowEntity.getTagValue(SegmentRecord.DATA_BINARY);
if (dataBinary != null && dataBinary.length > 0) {
segmentRecord.setDataBinary(dataBinary);
}
return segmentRecord;
}
- private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) {
+ private int querySequenceWithAgg(AggType aggType, String segmentId, long start, long end) throws IOException {
StreamQueryResponse resp = query(profileThreadSnapshotMetadata,
- ImmutableList.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
- ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE),
+ ImmutableSet.of(ProfileThreadSnapshotRecord.TASK_ID, ProfileThreadSnapshotRecord.SEGMENT_ID,
+ ProfileThreadSnapshotRecord.DUMP_TIME, ProfileThreadSnapshotRecord.SEQUENCE,
+ ProfileThreadSnapshotRecord.STACK_BINARY),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(ProfileThreadSnapshotRecord.STACK_BINARY));
-
query.appendCondition(eq(ProfileThreadSnapshotRecord.SEGMENT_ID, segmentId))
.appendCondition(lte(ProfileThreadSnapshotRecord.DUMP_TIME, end))
.appendCondition(gte(ProfileThreadSnapshotRecord.DUMP_TIME, start));
@@ -209,7 +207,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
List<ProfileThreadSnapshotRecord> records = new ArrayList<>();
for (final RowEntity rowEntity : resp.getElements()) {
ProfileThreadSnapshotRecord record = this.builder.storage2Entity(
- new BanyanDBConverter.StreamToEntity(profileThreadSnapshotMetadata, rowEntity));
+ new BanyanDBConverter.StreamToEntity(rowEntity));
records.add(record);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
index a1cf73edcd..3fe0e03ef7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBRecordDAO.java
@@ -29,7 +29,6 @@ import org.apache.skywalking.oap.server.core.storage.type.StorageBuilder;
import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
@@ -39,17 +38,15 @@ public class BanyanDBRecordDAO implements IRecordDAO {
@Override
public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
- StreamMetadata metadata = MetadataRegistry.INSTANCE.findStreamMetadata(model.getName());
+ MetadataRegistry.PartialMetadata metadata = MetadataRegistry.INSTANCE.findSchema(model.getName());
if (metadata == null) {
throw new IOException(model.getName() + " is not registered");
}
StreamWrite streamWrite = new StreamWrite(metadata.getGroup(), // group name
model.getName(), // index-name
record.id(), // identity
- TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling()), // timestamp
- metadata.getDataFamilySize(), // length of the "data" tag family
- metadata.getSearchableFamilySize()); // length of the "searchable" tag family
- Convert2Storage<StreamWrite> convert2Storage = new BanyanDBConverter.StreamToStorage(metadata, streamWrite);
+ TimeBucket.getTimestamp(record.getTimeBucket(), model.getDownsampling())); // timestamp
+ Convert2Storage<StreamWrite> convert2Storage = new BanyanDBConverter.StreamToStorage(streamWrite);
storageBuilder.entity2Storage(record, convert2Storage);
return new BanyanDBStreamInsertRequest(convert2Storage.obtain());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index fc0d458c29..e149b5ab27 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -19,7 +19,7 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
-import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
@@ -39,7 +39,6 @@ import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBConverter;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.MetadataRegistry;
-import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetadata;
import java.io.IOException;
import java.util.ArrayList;
@@ -47,8 +46,8 @@ import java.util.Collections;
import java.util.List;
public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITraceQueryDAO {
- private final StreamMetadata segmentRecordMetadata =
- MetadataRegistry.INSTANCE.findStreamMetadata(SegmentRecord.INDEX_NAME);
+ private final MetadataRegistry.PartialMetadata segmentRecordMetadata =
+ MetadataRegistry.INSTANCE.findSchema(SegmentRecord.INDEX_NAME);
public BanyanDBTraceQueryDAO(BanyanDBStorageClient client) {
super(client);
@@ -117,7 +116,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
}
StreamQueryResponse resp = query(segmentRecordMetadata,
- ImmutableList.of(SegmentRecord.TRACE_ID, // 0 - trace_id
+ ImmutableSet.of(SegmentRecord.TRACE_ID, // 0 - trace_id
SegmentRecord.IS_ERROR, // 1 - is_error
SegmentRecord.SERVICE_ID, // 2 - service_id
SegmentRecord.SERVICE_INSTANCE_ID, // 3 - service_instance_id
@@ -133,15 +132,15 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
BasicTrace basicTrace = new BasicTrace();
basicTrace.setSegmentId(row.getId());
- basicTrace.setStart(String.valueOf(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.START_TIME)));
+ basicTrace.setStart(String.valueOf(row.getTagValue(SegmentRecord.START_TIME)));
basicTrace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.ENDPOINT_ID)
+ row.getTagValue(SegmentRecord.ENDPOINT_ID)
).getEndpointName());
- basicTrace.setDuration(((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.LATENCY)).intValue());
+ basicTrace.setDuration(((Number) row.getTagValue(SegmentRecord.LATENCY)).intValue());
basicTrace.setError(BooleanUtils.valueToBoolean(
- ((Number) row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.IS_ERROR)).intValue()
+ ((Number) row.getTagValue(SegmentRecord.IS_ERROR)).intValue()
));
- basicTrace.getTraceIds().add(row.getValue(StreamMetadata.TAG_FAMILY_SEARCHABLE, SegmentRecord.TRACE_ID));
+ basicTrace.getTraceIds().add(row.getTagValue(SegmentRecord.TRACE_ID));
traceBrief.getTraces().add(basicTrace);
}
@@ -152,17 +151,17 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
StreamQueryResponse resp = query(segmentRecordMetadata,
- ImmutableList.of(SegmentRecord.TRACE_ID,
+ ImmutableSet.of(SegmentRecord.TRACE_ID,
SegmentRecord.IS_ERROR,
SegmentRecord.SERVICE_ID,
SegmentRecord.SERVICE_INSTANCE_ID,
SegmentRecord.ENDPOINT_ID,
SegmentRecord.LATENCY,
- SegmentRecord.START_TIME),
+ SegmentRecord.START_TIME,
+ SegmentRecord.DATA_BINARY),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList(SegmentRecord.DATA_BINARY));
query.appendCondition(eq(SegmentRecord.TRACE_ID, traceId));
}
});
@@ -171,7 +170,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
for (final RowEntity rowEntity : resp.getElements()) {
SegmentRecord segmentRecord = new SegmentRecord.Builder().storage2Entity(
- new BanyanDBConverter.StreamToEntity(segmentRecordMetadata, rowEntity));
+ new BanyanDBConverter.StreamToEntity(rowEntity));
segmentRecords.add(segmentRecord);
}
[skywalking] 17/24: add metadata json
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 9a8979a879eec86a43c7e79214193e49df53d65a
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Jan 27 11:10:14 2022 +0800
add metadata json
---
.../plugin/banyandb/BanyanDBIndexInstaller.java | 2 +-
.../EndpointRelationServerSideMetricsBuilder.java | 4 ++
.../storage/plugin/banyandb/schema/Metadata.java | 14 +++----
.../banyandb/schema/SegmentRecordBuilder.java | 12 +++++-
...ceInstanceRelationClientSideMetricsBuilder.java | 9 +++--
...ceInstanceRelationServerSideMetricsBuilder.java | 9 +++--
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 8 ++--
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 24 +++++------
.../metadata/endpoint_relation_server_side.json | 46 ++++++++++++++++++++++
.../main/resources/metadata/endpoint_traffic.json | 37 +++++++++++++++++
.../metadata/index_rules/component_id.json | 12 ++++++
.../metadata/index_rules/dest_endpoint.json | 12 ++++++
.../metadata/index_rules/dest_service_id.json | 12 ++++++
.../resources/metadata/index_rules/entity_id.json | 12 ++++++
.../resources/metadata/index_rules/latency.json | 12 ++++++
.../metadata/index_rules/source_endpoint.json | 12 ++++++
.../metadata/index_rules/source_service_id.json | 12 ++++++
.../src/main/resources/metadata/profile_task.json | 4 ++
.../src/main/resources/metadata/segment.json | 6 +--
... => service_instance_relation_client_side.json} | 43 +++++++++-----------
... => service_instance_relation_server_side.json} | 43 +++++++++-----------
.../metadata/service_relation_client_side.json | 46 ++++++++++++++++++++++
.../metadata/service_relation_server_side.json | 46 ++++++++++++++++++++++
.../{profile_task.json => service_traffic.json} | 37 ++++++-----------
24 files changed, 363 insertions(+), 111 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
index 6a8f789976..c9566b1549 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -41,7 +41,7 @@ public class BanyanDBIndexInstaller extends ModelInstaller {
StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
if (metaInfo != null) {
log.info("install index {}", model.getName());
-// ((BanyanDBStorageClient) client).createStream(metaInfo);
+ ((BanyanDBStorageClient) client).createStream(metaInfo);
}
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
index ec25c79e10..c6cef854e4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/EndpointRelationServerSideMetricsBuilder.java
@@ -31,14 +31,18 @@ public class EndpointRelationServerSideMetricsBuilder extends BanyanDBStorageDat
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointRelationServerSideMetrics entity) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ // 0 - source_endpoint
searchable.add(TagAndValue.stringField(entity.getSourceEndpoint()));
+ // 1 - dest_endpoint
searchable.add(TagAndValue.stringField(entity.getDestEndpoint()));
+ // 2 - entity_id
searchable.add(TagAndValue.stringField(entity.getEntityId()));
return searchable;
}
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(EndpointRelationServerSideMetrics entity) {
+ // 0 - component_id
return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
index 545cd507d6..9f0fda0ac4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/Metadata.java
@@ -50,17 +50,17 @@ public class Metadata {
searchable.add(TagAndValue.stringField(serviceName));
// 1 - serviceID
searchable.add(TagAndValue.stringField(entity.getServiceId()));
- // 2 - layer
+ // 2 - group
+ searchable.add(TagAndValue.stringField(entity.getGroup()));
+ // 3 - layer
Layer layer = entity.getLayer();
searchable.add(TagAndValue.longField(layer != null ? layer.value() : Layer.UNDEFINED.value()));
- // 3 - group
- searchable.add(TagAndValue.stringField(entity.getGroup()));
return searchable;
}
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceTraffic entity) {
- // shortName
+ // 0 - shortName
return Collections.singletonList(TagAndValue.stringField(entity.getShortName()));
}
}
@@ -69,10 +69,10 @@ public class Metadata {
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(EndpointTraffic entity) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
- // 0 - name
- searchable.add(TagAndValue.stringField(entity.getName()));
- // 1 - serviceID
+ // 0 - serviceID
searchable.add(TagAndValue.stringField(entity.getServiceId()));
+ // 1 - name
+ searchable.add(TagAndValue.stringField(entity.getName()));
return searchable;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
index 0aa54cd3c1..729ad90e15 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/SegmentRecordBuilder.java
@@ -42,13 +42,21 @@ public class SegmentRecordBuilder extends BanyanDBStorageDataBuilder<SegmentReco
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(SegmentRecord segmentRecord) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(10);
+ // 0 - trace_id
searchable.add(TagAndValue.stringField(segmentRecord.getTraceId()));
+ // 1 - is_error
+ searchable.add(TagAndValue.longField(segmentRecord.getIsError()));
+ // 2 - service_id
searchable.add(TagAndValue.stringField(segmentRecord.getServiceId()));
+ // 3 - service_instance_id
searchable.add(TagAndValue.stringField(segmentRecord.getServiceInstanceId()));
+ // 4 - endpoint_id
searchable.add(TagAndValue.stringField(segmentRecord.getEndpointId()));
- searchable.add(TagAndValue.longField(segmentRecord.getStartTime()));
+ // 5 - latency
searchable.add(TagAndValue.longField(segmentRecord.getLatency()));
- searchable.add(TagAndValue.longField(segmentRecord.getIsError()));
+ // 6 - start_time
+ searchable.add(TagAndValue.longField(segmentRecord.getStartTime()));
+ // 7 ~ 13: indexed tags
searchable.addAll(filterSearchableTags(segmentRecord.getTagsRawData(), INDEXED_TAGS));
return searchable;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
index e940f6d291..a7f0f1b156 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
@@ -24,7 +24,6 @@ import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
import java.util.ArrayList;
-import java.util.Collections;
import java.util.List;
public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationClientSideMetrics> {
@@ -32,15 +31,17 @@ public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBSto
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationClientSideMetrics entity) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
searchable.add(TagAndValue.stringField(entity.getEntityId()));
return searchable;
}
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationClientSideMetrics entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
+ data.add(TagAndValue.longField(entity.getComponentId()));
+ data.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
+ data.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
+ return data;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
index 43495bc148..7796265f8d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationServerSideMetricsBuilder.java
@@ -24,7 +24,6 @@ import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
import java.util.ArrayList;
-import java.util.Collections;
import java.util.List;
public class ServiceInstanceRelationServerSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationServerSideMetrics> {
@@ -32,15 +31,17 @@ public class ServiceInstanceRelationServerSideMetricsBuilder extends BanyanDBSto
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationServerSideMetrics entity) {
List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
- searchable.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
- searchable.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
searchable.add(TagAndValue.stringField(entity.getEntityId()));
return searchable;
}
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationServerSideMetrics entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getComponentId()));
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
+ data.add(TagAndValue.longField(entity.getComponentId()));
+ data.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
+ data.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
+ return data;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index b9fc53258c..447d6316ac 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -145,8 +145,8 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
public Endpoint apply(RowEntity row) {
Endpoint endpoint = new Endpoint();
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- endpoint.setName((String) searchable.get(0).getValue());
- endpoint.setId((String) searchable.get(1).getValue());
+ endpoint.setId((String) searchable.get(0).getValue());
+ endpoint.setName((String) searchable.get(1).getValue());
return endpoint;
}
}
@@ -158,9 +158,9 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
service.setName((String) searchable.get(0).getValue());
service.setId((String) searchable.get(1).getValue());
- String layerName = Layer.valueOf(((Number) searchable.get(2).getValue()).intValue()).name();
+ service.setGroup((String) searchable.get(2).getValue());
+ String layerName = Layer.valueOf(((Number) searchable.get(3).getValue()).intValue()).name();
service.getLayers().add(layerName);
- service.setGroup((String) searchable.get(3).getValue());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
service.setShortName((String) data.get(0).getValue());
return service;
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index c5bb950bc6..66053e268d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -58,43 +58,43 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
public void apply(StreamQuery query) {
if (minDuration != 0) {
// duration >= minDuration
- query.appendCondition(gte("duration", minDuration));
+ query.appendCondition(gte(SegmentRecord.LATENCY, minDuration));
}
if (maxDuration != 0) {
// duration <= maxDuration
- query.appendCondition(lte("duration", maxDuration));
+ query.appendCondition(lte(SegmentRecord.LATENCY, maxDuration));
}
if (!Strings.isNullOrEmpty(serviceId)) {
- query.appendCondition(eq("service_id", serviceId));
+ query.appendCondition(eq(SegmentRecord.SERVICE_ID, serviceId));
}
if (!Strings.isNullOrEmpty(serviceInstanceId)) {
- query.appendCondition(eq("service_instance_id", serviceInstanceId));
+ query.appendCondition(eq(SegmentRecord.SERVICE_INSTANCE_ID, serviceInstanceId));
}
if (!Strings.isNullOrEmpty(endpointId)) {
- query.appendCondition(eq("endpoint_id", endpointId));
+ query.appendCondition(eq(SegmentRecord.ENDPOINT_ID, endpointId));
}
switch (traceState) {
case ERROR:
- query.appendCondition(eq("state", TraceStateStorage.ERROR.getState()));
+ query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.ERROR.getState()));
break;
case SUCCESS:
- query.appendCondition(eq("state", TraceStateStorage.SUCCESS.getState()));
+ query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.SUCCESS.getState()));
break;
default:
- query.appendCondition(eq("state", TraceStateStorage.ALL.getState()));
+ query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.ALL.getState()));
break;
}
switch (queryOrder) {
case BY_START_TIME:
- query.setOrderBy(new StreamQuery.OrderBy("start_time", StreamQuery.OrderBy.Type.DESC));
+ query.setOrderBy(new StreamQuery.OrderBy(SegmentRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
break;
case BY_DURATION:
- query.setOrderBy(new StreamQuery.OrderBy("duration", StreamQuery.OrderBy.Type.DESC));
+ query.setOrderBy(new StreamQuery.OrderBy(SegmentRecord.LATENCY, StreamQuery.OrderBy.Type.DESC));
break;
}
@@ -118,7 +118,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
}
StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "state", "endpoint_id", "duration", "start_time"), tsRange, q);
+ ImmutableList.of("trace_id", "is_error", "endpoint_id", "latency", "start_time"), tsRange, q);
List<BasicTrace> basicTraces = resp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList());
@@ -131,7 +131,7 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
+ ImmutableList.of("trace_id", "is_error", "service_id", "service_instance_id", "endpoint_id", "latency", "start_time"),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
new file mode 100644
index 0000000000..0c59e84357
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
@@ -0,0 +1,46 @@
+{
+ "metadata": {
+ "name": "endpoint_relation_server_side",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "component_id",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "source_endpoint",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "dest_endpoint",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "entity_id",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
new file mode 100644
index 0000000000..9b0d1f40ca
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
@@ -0,0 +1,37 @@
+{
+ "metadata": {
+ "name": "endpoint_traffic",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": []
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "name",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json
new file mode 100644
index 0000000000..9b52f22ba2
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/component_id.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "component_id",
+ "group": "default"
+ },
+ "tags": [
+ "component_id"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json
new file mode 100644
index 0000000000..affb1fdf17
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_endpoint.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "dest_endpoint",
+ "group": "default"
+ },
+ "tags": [
+ "dest_endpoint"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json
new file mode 100644
index 0000000000..3bb400392e
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dest_service_id.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "dest_service_id",
+ "group": "default"
+ },
+ "tags": [
+ "dest_service_id"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json
new file mode 100644
index 0000000000..31ece658a6
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/entity_id.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "entity_id",
+ "group": "default"
+ },
+ "tags": [
+ "entity_id"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json
new file mode 100644
index 0000000000..4eb445055f
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/latency.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "latency",
+ "group": "default"
+ },
+ "tags": [
+ "latency"
+ ],
+ "type": "TYPE_TREE",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json
new file mode 100644
index 0000000000..164962f91c
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_endpoint.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "source_endpoint",
+ "group": "default"
+ },
+ "tags": [
+ "source_endpoint"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json
new file mode 100644
index 0000000000..f54b191164
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/source_service_id.json
@@ -0,0 +1,12 @@
+{
+ "metadata": {
+ "name": "source_service_id",
+ "group": "default"
+ },
+ "tags": [
+ "component_id"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
index 23f89f19c4..10aa497e74 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
@@ -4,6 +4,10 @@
"group": "default"
},
"tag_families": [
+ {
+ "name": "data",
+ "tags": []
+ },
{
"name": "searchable",
"tags": [
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
index 1317d9ecf0..066fa45c38 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
@@ -21,7 +21,7 @@
"type": "TAG_TYPE_STRING"
},
{
- "name": "state",
+ "name": "is_error",
"type": "TAG_TYPE_INT"
},
{
@@ -37,7 +37,7 @@
"type": "TAG_TYPE_STRING"
},
{
- "name": "duration",
+ "name": "latency",
"type": "TAG_TYPE_INT"
},
{
@@ -79,7 +79,7 @@
"tag_names": [
"service_id",
"service_instance_id",
- "state"
+ "is_error"
]
},
"opts": {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
similarity index 50%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
index 23f89f19c4..1a6f380e62 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
@@ -1,47 +1,40 @@
{
"metadata": {
- "name": "profile_task",
+ "name": "service_instance_relation_client_side",
"group": "default"
},
"tag_families": [
{
- "name": "searchable",
+ "name": "data",
"tags": [
{
- "name": "id",
- "type": "TAG_TYPE_STRING"
+ "name": "component_id",
+ "type": "TAG_TYPE_INT"
},
{
- "name": "service_id",
+ "name": "source_service_instance_id",
"type": "TAG_TYPE_STRING"
},
{
- "name": "endpoint_name",
+ "name": "dest_service_instance_id",
"type": "TAG_TYPE_STRING"
- },
- {
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "duration",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "min_duration_threshold",
- "type": "TAG_TYPE_INT"
- },
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
{
- "name": "dump_period",
- "type": "TAG_TYPE_INT"
+ "name": "source_service_id",
+ "type": "TAG_TYPE_STRING"
},
{
- "name": "create_time",
- "type": "TAG_TYPE_INT"
+ "name": "dest_service_id",
+ "type": "TAG_TYPE_STRING"
},
{
- "name": "max_sampling_count",
- "type": "TAG_TYPE_INT"
+ "name": "entity_id",
+ "type": "TAG_TYPE_STRING"
}
]
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
similarity index 50%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
index 23f89f19c4..2fa5c396b4 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
@@ -1,47 +1,40 @@
{
"metadata": {
- "name": "profile_task",
+ "name": "service_instance_relation_server_side",
"group": "default"
},
"tag_families": [
{
- "name": "searchable",
+ "name": "data",
"tags": [
{
- "name": "id",
- "type": "TAG_TYPE_STRING"
+ "name": "component_id",
+ "type": "TAG_TYPE_INT"
},
{
- "name": "service_id",
+ "name": "source_service_instance_id",
"type": "TAG_TYPE_STRING"
},
{
- "name": "endpoint_name",
+ "name": "dest_service_instance_id",
"type": "TAG_TYPE_STRING"
- },
- {
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "duration",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "min_duration_threshold",
- "type": "TAG_TYPE_INT"
- },
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
{
- "name": "dump_period",
- "type": "TAG_TYPE_INT"
+ "name": "source_service_id",
+ "type": "TAG_TYPE_STRING"
},
{
- "name": "create_time",
- "type": "TAG_TYPE_INT"
+ "name": "dest_service_id",
+ "type": "TAG_TYPE_STRING"
},
{
- "name": "max_sampling_count",
- "type": "TAG_TYPE_INT"
+ "name": "entity_id",
+ "type": "TAG_TYPE_STRING"
}
]
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
new file mode 100644
index 0000000000..b12e95adaa
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
@@ -0,0 +1,46 @@
+{
+ "metadata": {
+ "name": "service_relation_client_side",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "component_id",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "source_service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "dest_service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "entity_id",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
new file mode 100644
index 0000000000..8844639714
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
@@ -0,0 +1,46 @@
+{
+ "metadata": {
+ "name": "service_relation_server_side",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "component_id",
+ "type": "TAG_TYPE_INT"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "source_service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "dest_service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "entity_id",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
similarity index 51%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
index 23f89f19c4..ba0f1c07ed 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
@@ -1,14 +1,23 @@
{
"metadata": {
- "name": "profile_task",
+ "name": "endpoint_traffic",
"group": "default"
},
"tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "short_name",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ },
{
"name": "searchable",
"tags": [
{
- "name": "id",
+ "name": "name",
"type": "TAG_TYPE_STRING"
},
{
@@ -16,31 +25,11 @@
"type": "TAG_TYPE_STRING"
},
{
- "name": "endpoint_name",
+ "name": "service_group",
"type": "TAG_TYPE_STRING"
},
{
- "name": "start_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "duration",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "min_duration_threshold",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "dump_period",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "create_time",
- "type": "TAG_TYPE_INT"
- },
- {
- "name": "max_sampling_count",
+ "name": "layer",
"type": "TAG_TYPE_INT"
}
]
[skywalking] 18/24: support metadata, segment and topo
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit cfa04fdb6981176bb0a6dc534ffefc10c5130368
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Tue Feb 15 21:43:51 2022 +0800
support metadata, segment and topo
---
.../plugin/banyandb/BanyanDBStorageClient.java | 21 ++-
.../plugin/banyandb/BanyanDBStorageProvider.java | 7 +-
.../storage/plugin/banyandb/StreamMetaInfo.java | 14 +-
.../schema/NetworkAddressAliasBuilder.java | 2 +-
.../schema/ProfileTaskLogRecordBuilder.java | 9 +-
...ceInstanceRelationClientSideMetricsBuilder.java | 4 +-
.../ServiceRelationClientSideMetricsBuilder.java | 2 +-
.../banyandb/stream/AbstractBanyanDBDAO.java | 7 +-
.../banyandb/stream/BanyanDBManagementDAO.java | 4 -
.../banyandb/stream/BanyanDBMetadataQueryDAO.java | 18 ++-
.../banyandb/stream/BanyanDBNoneStreamDAO.java | 2 +-
.../stream/BanyanDBProfileTaskLogQueryDAO.java | 13 +-
.../stream/BanyanDBProfileTaskQueryDAO.java | 2 +-
.../BanyanDBProfileThreadSnapshotQueryDAO.java | 2 +-
.../banyandb/stream/BanyanDBTopologyQueryDAO.java | 167 ++++++++++++++++++---
.../banyandb/stream/BanyanDBTraceQueryDAO.java | 45 +++---
.../metadata/endpoint_relation_server_side.json | 13 +-
.../main/resources/metadata/endpoint_traffic.json | 12 +-
.../src/main/resources/metadata/events.json | 12 +-
.../resources/metadata/index_rules/disabled.json | 2 +-
.../index_rules/{disabled.json => dump_time.json} | 6 +-
.../index_rules/{disabled.json => layer.json} | 4 +-
.../{disabled.json => operation_time.json} | 6 +-
.../index_rules/{disabled.json => segment_id.json} | 4 +-
.../index_rules/{disabled.json => sequence.json} | 6 +-
.../{disabled.json => service_group.json} | 4 +-
.../main/resources/metadata/instance_traffic.json | 13 +-
.../resources/metadata/network_address_alias.json | 13 +-
.../src/main/resources/metadata/profile_task.json | 12 +-
...instance_traffic.json => profile_task_log.json} | 31 ++--
...fic.json => profile_task_segment_snapshot.json} | 29 ++--
.../src/main/resources/metadata/segment.json | 11 +-
.../service_instance_relation_client_side.json | 6 +-
.../service_instance_relation_server_side.json | 6 +-
.../metadata/service_relation_client_side.json | 13 +-
.../metadata/service_relation_server_side.json | 13 +-
.../main/resources/metadata/service_traffic.json | 14 +-
.../src/main/resources/metadata/ui_template.json | 12 +-
38 files changed, 317 insertions(+), 244 deletions(-)
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 7d61e4e085..6b4780cad7 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -19,10 +19,12 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
+import org.apache.skywalking.banyandb.v1.client.GroupedBanyanDBClient;
import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.metadata.Group;
import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.library.client.Client;
@@ -38,10 +40,15 @@ import java.io.IOException;
*/
public class BanyanDBStorageClient implements Client, HealthCheckable {
private final BanyanDBClient client;
+ private GroupedBanyanDBClient streamClient;
private final DelegatedHealthChecker healthChecker = new DelegatedHealthChecker();
- public BanyanDBStorageClient(String host, int port, String group) {
- this.client = new BanyanDBClient(host, port, group);
+ public BanyanDBStorageClient(String host, int port) {
+ this.client = new BanyanDBClient(host, port);
+ }
+
+ public void defineStreamGroup(Group group) {
+ this.streamClient = this.client.attachGroup(group);
}
@Override
@@ -56,7 +63,7 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
public StreamQueryResponse query(StreamQuery streamQuery) {
try {
- StreamQueryResponse response = this.client.queryStreams(streamQuery);
+ StreamQueryResponse response = this.streamClient.queryStreams(streamQuery);
this.healthChecker.health();
return response;
} catch (Throwable t) {
@@ -66,18 +73,18 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
public void createStream(StreamMetaInfo streamMetaInfo) {
- Stream stm = this.client.define(streamMetaInfo.getStream());
+ Stream stm = this.streamClient.define(streamMetaInfo.getStream());
if (stm != null) {
- this.client.defineIndexRules(stm, streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
+ this.streamClient.defineIndexRules(stm, streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
}
}
public void write(StreamWrite streamWrite) {
- this.client.write(streamWrite);
+ this.streamClient.write(streamWrite);
}
public StreamBulkWriteProcessor createBulkProcessor(int maxBulkSize, int flushInterval, int concurrency) {
- return this.client.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
+ return this.streamClient.buildStreamWriteProcessor(maxBulkSize, flushInterval, concurrency);
}
@Override
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index 6d4340e48d..b938ab1159 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -18,6 +18,8 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+import org.apache.skywalking.banyandb.v1.client.metadata.Catalog;
+import org.apache.skywalking.banyandb.v1.client.metadata.Group;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.ConfigService;
import org.apache.skywalking.oap.server.core.storage.IBatchDAO;
@@ -93,7 +95,7 @@ public class BanyanDBStorageProvider extends ModuleProvider {
public void prepare() throws ServiceNotProvidedException, ModuleStartException {
this.registerServiceImplementation(StorageBuilderFactory.class, new BanyanDBStorageBuilderFactory());
- this.client = new BanyanDBStorageClient(config.getHost(), config.getPort(), config.getGroup());
+ this.client = new BanyanDBStorageClient(config.getHost(), config.getPort());
// Stream
this.registerServiceImplementation(IBatchDAO.class, new BanyanDBBatchDAO(client, config.getMaxBulkSize(), config.getFlushInterval(), config.getConcurrentWriteThreads()));
@@ -133,6 +135,9 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.client.registerChecker(healthChecker);
try {
this.client.connect();
+ // create stream group
+ final Group streamGroup = new Group("default-stream", Catalog.STREAM, 2);
+ this.client.defineStreamGroup(streamGroup);
BanyanDBIndexInstaller installer = new BanyanDBIndexInstaller(client, getManager());
getManager().find(CoreModule.NAME).provider().getService(ModelCreator.class).addModelListener(installer);
} catch (Exception e) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
index de7af730ee..0a2af94339 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
@@ -19,13 +19,13 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import com.google.common.base.Charsets;
+import com.google.common.collect.ImmutableSet;
import com.google.common.io.CharStreams;
import com.google.protobuf.util.JsonFormat;
import lombok.Builder;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase;
-import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
@@ -38,6 +38,7 @@ import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Set;
@Getter
@Builder
@@ -69,14 +70,13 @@ public class StreamMetaInfo {
log.warn("fail to find stream schema {}", model.getName());
return null;
}
- BanyandbDatabase.Duration duration = pbStream.getOpts().getTtl();
- Duration ttl = Duration.fromProtobuf(duration);
- final Stream stream = new Stream(pbStream.getMetadata().getName(), pbStream.getOpts().getShardNum(), ttl);
+ final Stream stream = new Stream(pbStream.getMetadata().getName());
List<IndexRule> indexRules = new ArrayList<>();
- stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
+ Set<String> entityNameSet = ImmutableSet.copyOf(pbStream.getEntity().getTagNamesList());
+ stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
for (BanyandbDatabase.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
final TagFamilySpec tagFamilySpec = TagFamilySpec.fromProtobuf(pbTagFamilySpec);
@@ -85,6 +85,10 @@ public class StreamMetaInfo {
// if the tag family equals to "searchable", build index rules
if (tagFamilySpec.getTagFamilyName().equals(TAG_FAMILY_SEARCHABLE)) {
for (final TagFamilySpec.TagSpec tagSpec : tagFamilySpec.getTagSpecs()) {
+ // check if this spec exists in the entity names
+ if (entityNameSet.contains(tagSpec.getTagName())) {
+ continue;
+ }
BanyandbDatabase.IndexRule pbIndexRule = parseIndexRulesFromJSON(model.getName(), tagSpec.getTagName());
if (pbIndexRule == null) {
log.warn("fail to find the index rule for {}", tagSpec.getTagName());
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
index ca2d78d2ef..e429f80b9c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/NetworkAddressAliasBuilder.java
@@ -35,7 +35,7 @@ public class NetworkAddressAliasBuilder extends BanyanDBStorageDataBuilder<Netwo
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(NetworkAddressAlias entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(3);
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(4);
// 0 - time_bucket
data.add(TagAndValue.longField(entity.getTimeBucket()));
// 1 - address
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
index a0cab9c112..1f861ab4cd 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ProfileTaskLogRecordBuilder.java
@@ -24,20 +24,21 @@ import org.apache.skywalking.banyandb.v1.client.TagAndValue;
import org.apache.skywalking.oap.server.core.profile.ProfileTaskLogRecord;
import java.util.ArrayList;
-import java.util.Collections;
import java.util.List;
public class ProfileTaskLogRecordBuilder extends BanyanDBStorageDataBuilder<ProfileTaskLogRecord> {
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ProfileTaskLogRecord entity) {
- return Collections.singletonList(TagAndValue.longField(entity.getOperationTime()));
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(2);
+ searchable.add(TagAndValue.longField(entity.getOperationTime()));
+ searchable.add(TagAndValue.stringField(entity.getInstanceId()));
+ return searchable;
}
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ProfileTaskLogRecord entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(2);
data.add(TagAndValue.stringField(entity.getTaskId()));
- data.add(TagAndValue.stringField(entity.getInstanceId()));
data.add(TagAndValue.longField(entity.getOperationType()));
return data;
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
index a7f0f1b156..dc4de6ddcb 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceInstanceRelationClientSideMetricsBuilder.java
@@ -29,7 +29,7 @@ import java.util.List;
public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceInstanceRelationClientSideMetrics> {
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceInstanceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
searchable.add(TagAndValue.stringField(entity.getEntityId()));
@@ -38,7 +38,7 @@ public class ServiceInstanceRelationClientSideMetricsBuilder extends BanyanDBSto
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> dataTags(ServiceInstanceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>();
+ List<SerializableTag<BanyandbModel.TagValue>> data = new ArrayList<>(3);
data.add(TagAndValue.longField(entity.getComponentId()));
data.add(TagAndValue.stringField(entity.getSourceServiceInstanceId()));
data.add(TagAndValue.stringField(entity.getDestServiceInstanceId()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
index 3d50dde043..cf8ae73fdf 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/schema/ServiceRelationClientSideMetricsBuilder.java
@@ -30,7 +30,7 @@ import java.util.List;
public class ServiceRelationClientSideMetricsBuilder extends BanyanDBStorageDataBuilder<ServiceRelationClientSideMetrics> {
@Override
protected List<SerializableTag<BanyandbModel.TagValue>> searchableTags(ServiceRelationClientSideMetrics entity) {
- List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>();
+ List<SerializableTag<BanyandbModel.TagValue>> searchable = new ArrayList<>(3);
searchable.add(TagAndValue.stringField(entity.getSourceServiceId()));
searchable.add(TagAndValue.stringField(entity.getDestServiceId()));
searchable.add(TagAndValue.stringField(entity.getEntityId()));
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
index a3d2f1dc85..04690a30cb 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/AbstractBanyanDBDAO.java
@@ -27,10 +27,15 @@ import org.apache.skywalking.oap.server.core.storage.AbstractDAO;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.StreamMetaInfo;
+import java.time.Instant;
import java.util.List;
import java.util.function.Function;
public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageClient> {
+ private static final Instant UPPER_BOUND = Instant.ofEpochSecond(0, Long.MAX_VALUE);
+
+ private static final TimestampRange LARGEST_TIME_RANGE = new TimestampRange(0, UPPER_BOUND.toEpochMilli());
+
protected AbstractBanyanDBDAO(BanyanDBStorageClient client) {
super(client);
}
@@ -43,7 +48,7 @@ public abstract class AbstractBanyanDBDAO extends AbstractDAO<BanyanDBStorageCli
QueryBuilder builder) {
final StreamQuery query;
if (timestampRange == null) {
- query = new StreamQuery(indexName, searchableTags);
+ query = new StreamQuery(indexName, LARGEST_TIME_RANGE, searchableTags);
} else {
query = new StreamQuery(indexName, timestampRange, searchableTags);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
index 1990181804..a87a94434d 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBManagementDAO.java
@@ -41,9 +41,6 @@ import java.util.Collections;
* @param <T> The only ManagementData we have now is {@link UITemplate}
*/
public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBanyanDBDAO implements IManagementDAO {
- private final static long START_TIME_MILLI =
- ZonedDateTime.of(2022, 1, 1, 0, 0, 0, 0,
- ZoneOffset.UTC).toInstant().toEpochMilli();
private final BanyanDBStorageDataBuilder<T> storageBuilder;
public BanyanDBManagementDAO(BanyanDBStorageClient client, BanyanDBStorageDataBuilder<T> storageBuilder) {
@@ -56,7 +53,6 @@ public class BanyanDBManagementDAO<T extends ManagementData> extends AbstractBan
// ensure only insert once
StreamQueryResponse resp = query(UITemplate.INDEX_NAME,
Collections.singletonList(UITemplate.NAME),
- new TimestampRange(START_TIME_MILLI, Instant.now().toEpochMilli()),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
index 447d6316ac..85ea280b72 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBMetadataQueryDAO.java
@@ -62,10 +62,11 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
@Override
public List<Service> listServices(final String layer, final String group) throws IOException {
StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.LAYER, ServiceTraffic.GROUP),
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.SERVICE_ID, ServiceTraffic.GROUP, ServiceTraffic.LAYER),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList(ServiceTraffic.SHORT_NAME));
if (StringUtil.isNotEmpty(layer)) {
query.appendCondition(eq(ServiceTraffic.LAYER, Layer.valueOf(layer).value()));
}
@@ -81,10 +82,12 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
@Override
public List<Service> getServices(String serviceId) throws IOException {
StreamQueryResponse resp = query(ServiceTraffic.INDEX_NAME,
- ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.LAYER, ServiceTraffic.GROUP),
+ ImmutableList.of(ServiceTraffic.NAME, ServiceTraffic.SERVICE_ID, ServiceTraffic.GROUP, ServiceTraffic.LAYER),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList(ServiceTraffic.SHORT_NAME));
+
query.appendCondition(eq(ServiceTraffic.SERVICE_ID, serviceId));
}
});
@@ -116,9 +119,11 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
@Override
public ServiceInstance getInstance(String instanceId) throws IOException {
StreamQueryResponse resp = query(InstanceTraffic.INDEX_NAME,
- ImmutableList.of(InstanceTraffic.NAME, InstanceTraffic.SERVICE_ID), new QueryBuilder() {
+ ImmutableList.of(StreamMetaInfo.ID), new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
+ query.setDataProjections(Collections.singletonList("data_binary"));
+
query.appendCondition(eq(StreamMetaInfo.ID, instanceId));
}
});
@@ -145,8 +150,8 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
public Endpoint apply(RowEntity row) {
Endpoint endpoint = new Endpoint();
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
- endpoint.setId((String) searchable.get(0).getValue());
- endpoint.setName((String) searchable.get(1).getValue());
+ endpoint.setId(row.getId());
+ endpoint.setName((String) searchable.get(0).getValue()); // 0 - name
return endpoint;
}
}
@@ -179,7 +184,8 @@ public class BanyanDBMetadataQueryDAO extends AbstractBanyanDBDAO implements IMe
RemoteData remoteData = RemoteData.parseFrom((ByteString) o);
instanceTraffic.deserialize(remoteData);
serviceInstance.setName(instanceTraffic.getName());
- serviceInstance.setId(instanceTraffic.getServiceId());
+ serviceInstance.setId(row.getId());
+ serviceInstance.setInstanceUUID(serviceInstance.getId());
serviceInstance.setLayer(instanceTraffic.getLayer().name());
if (instanceTraffic.getProperties() != null) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
index e19ff9af98..ae5c766897 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBNoneStreamDAO.java
@@ -41,7 +41,7 @@ public class BanyanDBNoneStreamDAO<T extends NoneStream> implements INoneStreamD
@Override
public void insert(Model model, NoneStream noneStream) throws IOException {
- final long timestamp = TimeBucket.getTimeBucket(noneStream.getTimeBucket(), model.getDownsampling());
+ final long timestamp = TimeBucket.getTimestamp(noneStream.getTimeBucket(), model.getDownsampling());
StreamWrite.StreamWriteBuilder builder =
this.storageBuilder.entity2Storage((T) noneStream)
.name(model.getName())
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
index 3a4910f3b5..1da585bb4b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskLogQueryDAO.java
@@ -48,12 +48,11 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
@Override
public List<ProfileTaskLog> getTaskLogList() throws IOException {
StreamQueryResponse resp = query(ProfileTaskLogRecord.INDEX_NAME,
- ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME),
+ ImmutableList.of(ProfileTaskLogRecord.OPERATION_TIME, ProfileTaskLogRecord.INSTANCE_ID),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
query.setDataProjections(ImmutableList.of(ProfileTaskLogRecord.TASK_ID,
- ProfileTaskLogRecord.INSTANCE_ID,
ProfileTaskLogRecord.OPERATION_TYPE));
query.setLimit(BanyanDBProfileTaskLogQueryDAO.this.queryMaxSize);
}
@@ -71,13 +70,13 @@ public class BanyanDBProfileTaskLogQueryDAO extends AbstractBanyanDBDAO implemen
final List<TagAndValue<?>> searchable = row.getTagFamilies().get(0);
// searchable - operation_time
profileTaskLog.setOperationTime(((Number) searchable.get(0).getValue()).longValue());
+ // searchable - instance_id
+ profileTaskLog.setInstanceId((String) searchable.get(1).getValue());
final List<TagAndValue<?>> data = row.getTagFamilies().get(1);
- // searchable - task_id
+ // data - task_id
profileTaskLog.setTaskId((String) data.get(0).getValue());
- // searchable - instance_id
- profileTaskLog.setInstanceId((String) data.get(1).getValue());
- // searchable - operation_type
- profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(2).getValue()).intValue()));
+ // data - operation_type
+ profileTaskLog.setOperationType(ProfileTaskLogOperationType.parse(((Number) data.get(1).getValue()).intValue()));
return profileTaskLog;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
index 93cc99c6c3..4bf0996eb8 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileTaskQueryDAO.java
@@ -72,7 +72,7 @@ public class BanyanDBProfileTaskQueryDAO extends AbstractBanyanDBDAO implements
query.setLimit(limit);
}
- query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
+// query.setOrderBy(new StreamQuery.OrderBy(ProfileTaskRecord.START_TIME, StreamQuery.OrderBy.Type.DESC));
}
});
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
index 4da87ac9e5..6df30bb465 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBProfileThreadSnapshotQueryDAO.java
@@ -124,7 +124,7 @@ public class BanyanDBProfileThreadSnapshotQueryDAO extends AbstractBanyanDBDAO i
@Override
public SegmentRecord getProfiledSegment(String segmentId) throws IOException {
StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "state", "service_id", "service_instance_id", "endpoint_id", "duration", "start_time"),
+ ImmutableList.of(SegmentRecord.TRACE_ID, SegmentRecord.IS_ERROR, SegmentRecord.SERVICE_ID, SegmentRecord.SERVICE_INSTANCE_ID, SegmentRecord.ENDPOINT_ID, SegmentRecord.LATENCY, SegmentRecord.START_TIME),
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
index f574e9b24a..660190dd10 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTopologyQueryDAO.java
@@ -18,7 +18,18 @@
package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import io.vavr.Tuple2;
+import io.vavr.Tuple4;
+import lombok.RequiredArgsConstructor;
+import org.apache.skywalking.banyandb.v1.client.RowEntity;
+import org.apache.skywalking.banyandb.v1.client.StreamQuery;
+import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
+import org.apache.skywalking.banyandb.v1.client.TimestampRange;
+import org.apache.skywalking.oap.server.core.Const;
import org.apache.skywalking.oap.server.core.UnexpectedException;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.endpoint.EndpointRelationServerSideMetrics;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationClientSideMetrics;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.instance.ServiceInstanceRelationServerSideMetrics;
@@ -34,6 +45,8 @@ import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITopologyQueryDAO {
public BanyanDBTopologyQueryDAO(BanyanDBStorageClient client) {
@@ -71,7 +84,7 @@ public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITo
return loadServiceCalls(
ServiceRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
ServiceRelationServerSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationServerSideMetrics.DEST_SERVICE_ID, new ArrayList<>(0), DetectPoint.SERVER
+ ServiceRelationServerSideMetrics.DEST_SERVICE_ID, Collections.emptyList(), DetectPoint.SERVER
);
}
@@ -80,7 +93,7 @@ public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITo
return loadServiceCalls(
ServiceRelationClientSideMetrics.INDEX_NAME, startTB, endTB,
ServiceRelationClientSideMetrics.SOURCE_SERVICE_ID,
- ServiceRelationClientSideMetrics.DEST_SERVICE_ID, new ArrayList<>(0), DetectPoint.CLIENT
+ ServiceRelationClientSideMetrics.DEST_SERVICE_ID, Collections.emptyList(), DetectPoint.CLIENT
);
}
@@ -106,17 +119,11 @@ public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITo
@Override
public List<Call.CallDetail> loadEndpointRelation(long startTB, long endTB, String destEndpointId) throws IOException {
- List<Call.CallDetail> calls = loadEndpointFromSide(
+ return loadEndpointCalls(
EndpointRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
EndpointRelationServerSideMetrics.SOURCE_ENDPOINT,
- EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId, false
+ EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId
);
- calls.addAll(
- loadEndpointFromSide(EndpointRelationServerSideMetrics.INDEX_NAME, startTB, endTB,
- EndpointRelationServerSideMetrics.SOURCE_ENDPOINT,
- EndpointRelationServerSideMetrics.DEST_ENDPOINT, destEndpointId, true
- ));
- return calls;
}
private List<Call.CallDetail> loadServiceCalls(String tableName,
@@ -126,8 +133,37 @@ public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITo
String destCName,
List<String> serviceIds,
DetectPoint detectPoint) throws IOException {
- // TODO: we will impl this method after we support `OR`
- return Collections.emptyList();
+ TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
+ List<Call.CallDetail> calls = new ArrayList<>();
+ if (serviceIds.isEmpty()) {
+ StreamQueryResponse resp = query(tableName, Collections.emptyList(), timeRange, new QueryBuilder() {
+ @Override
+ void apply(StreamQuery query) {
+ // query component_id
+ query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
+ }
+ });
+
+ calls.addAll(resp.getElements().stream().map(new ServiceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
+ } else {
+ for (String fieldOfInterest : ImmutableList.of(sourceCName, destCName)) {
+ for (String serviceID : serviceIds) {
+ StreamQueryResponse resp = query(tableName, ImmutableList.of(fieldOfInterest), timeRange, new QueryBuilder() {
+ @Override
+ void apply(StreamQuery query) {
+ // query component_id
+ query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
+
+ query.appendCondition(eq(fieldOfInterest, serviceID));
+ }
+ });
+
+ calls.addAll(resp.getElements().stream().map(new ServiceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
+ }
+ }
+ }
+
+ return calls;
}
private List<Call.CallDetail> loadServiceInstanceCalls(String tableName,
@@ -138,18 +174,103 @@ public class BanyanDBTopologyQueryDAO extends AbstractBanyanDBDAO implements ITo
String sourceServiceId,
String destServiceId,
DetectPoint detectPoint) throws IOException {
- // TODO: we will impl this method after we support `OR`
- return Collections.emptyList();
+ TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
+
+ Set<Tuple4<String, String, String, String>> productQuerySet = ImmutableSet.of(
+ new Tuple4<>(sourceCName, sourceServiceId, descCName, destServiceId),
+ new Tuple4<>(sourceCName, destServiceId, descCName, sourceServiceId)
+ );
+
+ List<Call.CallDetail> calls = new ArrayList<>();
+
+ for (Tuple4<String, String, String, String> querySet : productQuerySet) {
+ StreamQueryResponse resp = query(tableName, ImmutableList.of(querySet._1(), querySet._3()), timeRange, new QueryBuilder() {
+ @Override
+ void apply(StreamQuery query) {
+ // query component_id
+ query.setDataProjections(Collections.singletonList(ServiceRelationServerSideMetrics.COMPONENT_ID));
+
+ query.appendCondition(eq(querySet._1(), querySet._2()));
+ query.appendCondition(eq(querySet._3(), querySet._4()));
+ }
+ });
+
+ calls.addAll(resp.getElements().stream().map(new InstanceCallDetailDeserializer(detectPoint)).collect(Collectors.toList()));
+ }
+
+ return calls;
}
- private List<Call.CallDetail> loadEndpointFromSide(String tableName,
- long startTB,
- long endTB,
- String sourceCName,
- String destCName,
- String id,
- boolean isSourceId) throws IOException {
- // TODO: we will impl this method after we support `OR`
- return Collections.emptyList();
+ private List<Call.CallDetail> loadEndpointCalls(String tableName,
+ long startTB,
+ long endTB,
+ String sourceCName,
+ String destCName,
+ String id) throws IOException {
+ TimestampRange timeRange = new TimestampRange(TimeBucket.getTimestamp(startTB), TimeBucket.getTimestamp(endTB));
+
+ Set<Tuple2<String, String>> allPossibleQuerySet = ImmutableSet.of(
+ new Tuple2<>(sourceCName, id),
+ new Tuple2<>(destCName, id)
+ );
+
+ List<Call.CallDetail> calls = new ArrayList<>();
+
+ for (Tuple2<String, String> querySet : allPossibleQuerySet) {
+ StreamQueryResponse resp = query(tableName, ImmutableList.of(querySet._1()), timeRange, new QueryBuilder() {
+ @Override
+ void apply(StreamQuery query) {
+ query.appendCondition(eq(querySet._1(), querySet._2()));
+ }
+ });
+
+ calls.addAll(resp.getElements().stream().map(new EndpointCallDetailDeserializer()).collect(Collectors.toList()));
+ }
+
+ return calls;
+ }
+
+ @RequiredArgsConstructor
+ public static class ServiceCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
+ private final DetectPoint detectPoint;
+
+ @Override
+ public Call.CallDetail apply(RowEntity rowEntity) {
+ Call.CallDetail call = new Call.CallDetail();
+ String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
+ String entityId = idsSlice[1];
+ int componentId = ((Number) rowEntity.getTagFamilies().get(1) // Tag Family: "data"
+ .get(0).getValue()).intValue();
+ call.buildFromServiceRelation(entityId, componentId, this.detectPoint);
+ return call;
+ }
+ }
+
+ @RequiredArgsConstructor
+ public static class InstanceCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
+ private final DetectPoint detectPoint;
+
+ @Override
+ public Call.CallDetail apply(RowEntity rowEntity) {
+ Call.CallDetail call = new Call.CallDetail();
+ String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
+ String entityId = idsSlice[1];
+ int componentId = ((Number) rowEntity.getTagFamilies().get(1) // Tag Family: "data"
+ .get(0).getValue()).intValue();
+ call.buildFromInstanceRelation(entityId, componentId, this.detectPoint);
+ return call;
+ }
+ }
+
+ @RequiredArgsConstructor
+ public static class EndpointCallDetailDeserializer implements RowEntityDeserializer<Call.CallDetail> {
+ @Override
+ public Call.CallDetail apply(RowEntity rowEntity) {
+ Call.CallDetail call = new Call.CallDetail();
+ String[] idsSlice = rowEntity.getId().split(Const.ID_CONNECTOR);
+ String entityId = idsSlice[1];
+ call.buildFromEndpointRelation(entityId, DetectPoint.SERVER);
+ return call;
+ }
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
index 66053e268d..642ed519fc 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/stream/BanyanDBTraceQueryDAO.java
@@ -21,7 +21,6 @@ package org.apache.skywalking.oap.server.storage.plugin.banyandb.stream;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.protobuf.ByteString;
-import lombok.Getter;
import org.apache.skywalking.banyandb.v1.client.RowEntity;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
@@ -37,6 +36,7 @@ import org.apache.skywalking.oap.server.core.query.type.Span;
import org.apache.skywalking.oap.server.core.query.type.TraceBrief;
import org.apache.skywalking.oap.server.core.query.type.TraceState;
import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
import org.apache.skywalking.oap.server.library.util.CollectionUtils;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageClient;
import org.apache.skywalking.oap.server.storage.plugin.banyandb.schema.SegmentRecordBuilder;
@@ -79,13 +79,10 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
switch (traceState) {
case ERROR:
- query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.ERROR.getState()));
+ query.appendCondition(eq(SegmentRecord.IS_ERROR, BooleanUtils.TRUE));
break;
case SUCCESS:
- query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.SUCCESS.getState()));
- break;
- default:
- query.appendCondition(eq(SegmentRecord.IS_ERROR, TraceStateStorage.ALL.getState()));
+ query.appendCondition(eq(SegmentRecord.IS_ERROR, BooleanUtils.FALSE));
break;
}
@@ -118,7 +115,14 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
}
StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "is_error", "endpoint_id", "latency", "start_time"), tsRange, q);
+ ImmutableList.of(SegmentRecord.TRACE_ID, // 0 - trace_id
+ SegmentRecord.IS_ERROR, // 1 - is_error
+ SegmentRecord.SERVICE_ID, // 2 - service_id
+ SegmentRecord.SERVICE_INSTANCE_ID, // 3 - service_instance_id
+ SegmentRecord.ENDPOINT_ID, // 4 - endpoint_id
+ SegmentRecord.LATENCY, // 5 - latency
+ SegmentRecord.START_TIME), // 6 - start_time
+ tsRange, q);
List<BasicTrace> basicTraces = resp.getElements().stream().map(new BasicTraceDeserializer()).collect(Collectors.toList());
@@ -131,11 +135,17 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
@Override
public List<SegmentRecord> queryByTraceId(String traceId) throws IOException {
StreamQueryResponse resp = query(SegmentRecord.INDEX_NAME,
- ImmutableList.of("trace_id", "is_error", "service_id", "service_instance_id", "endpoint_id", "latency", "start_time"),
+ ImmutableList.of(SegmentRecord.TRACE_ID, // 0 - trace_id
+ SegmentRecord.IS_ERROR, // 1 - is_error
+ SegmentRecord.SERVICE_ID, // 2 - service_id
+ SegmentRecord.SERVICE_INSTANCE_ID, // 3 - service_instance_id
+ SegmentRecord.ENDPOINT_ID, // 4 - endpoint_id
+ SegmentRecord.LATENCY, // 5 - latency
+ SegmentRecord.START_TIME), // 6 - start_time
new QueryBuilder() {
@Override
public void apply(StreamQuery query) {
- query.setDataProjections(Collections.singletonList("data_binary"));
+ query.setDataProjections(Collections.singletonList(SegmentRecord.DATA_BINARY));
query.appendCondition(eq(SegmentRecord.TRACE_ID, traceId));
}
});
@@ -148,17 +158,6 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
return Collections.emptyList();
}
- public enum TraceStateStorage {
- ALL(0), SUCCESS(1), ERROR(2);
-
- @Getter
- private final int state;
-
- TraceStateStorage(int state) {
- this.state = state;
- }
- }
-
public static class BasicTraceDeserializer implements RowEntityDeserializer<BasicTrace> {
@Override
public BasicTrace apply(RowEntity row) {
@@ -168,10 +167,10 @@ public class BanyanDBTraceQueryDAO extends AbstractBanyanDBDAO implements ITrace
trace.getTraceIds().add((String) searchable.get(0).getValue());
trace.setError(((Long) searchable.get(1).getValue()).intValue() == 1);
trace.getEndpointNames().add(IDManager.EndpointID.analysisId(
- (String) searchable.get(2).getValue()
+ (String) searchable.get(4).getValue()
).getEndpointName());
- trace.setDuration(((Long) searchable.get(3).getValue()).intValue());
- trace.setStart(String.valueOf(searchable.get(4).getValue()));
+ trace.setDuration(((Long) searchable.get(5).getValue()).intValue());
+ trace.setStart(String.valueOf(searchable.get(6).getValue()));
return trace;
}
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
index 0c59e84357..2a203c1679 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_relation_server_side.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "endpoint_relation_server_side",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -33,14 +33,9 @@
],
"entity": {
"tag_names": [
+ "source_endpoint",
+ "dest_endpoint"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
index 9b0d1f40ca..459b77d808 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/endpoint_traffic.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "endpoint_traffic",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -24,14 +24,8 @@
],
"entity": {
"tag_names": [
+ "service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
index 9c0f091ac2..37a02eca24 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/events.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "events",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -57,14 +57,8 @@
],
"entity": {
"tag_names": [
+ "uuid"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
index 8cfb12816a..19a1c1b204 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
@@ -6,7 +6,7 @@
"tags": [
"disabled"
],
- "type": "TYPE_INVERTED",
+ "type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json
similarity index 67%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json
index 8cfb12816a..6073ee7740 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/dump_time.json
@@ -1,12 +1,12 @@
{
"metadata": {
- "name": "disabled",
+ "name": "dump_time",
"group": "default"
},
"tags": [
- "disabled"
+ "dump_time"
],
- "type": "TYPE_INVERTED",
+ "type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json
similarity index 81%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json
index 8cfb12816a..05455e1d6b 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/layer.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "disabled",
+ "name": "layer",
"group": "default"
},
"tags": [
- "disabled"
+ "layer"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json
similarity index 65%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json
index 8cfb12816a..30222bc484 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/operation_time.json
@@ -1,12 +1,12 @@
{
"metadata": {
- "name": "disabled",
+ "name": "operation_time",
"group": "default"
},
"tags": [
- "disabled"
+ "operation_time"
],
- "type": "TYPE_INVERTED",
+ "type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json
similarity index 79%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json
index 8cfb12816a..a58129eab2 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment_id.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "disabled",
+ "name": "segment_id",
"group": "default"
},
"tags": [
- "disabled"
+ "segment_id"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json
similarity index 67%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json
index 8cfb12816a..2bbd960809 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/sequence.json
@@ -1,12 +1,12 @@
{
"metadata": {
- "name": "disabled",
+ "name": "sequence",
"group": "default"
},
"tags": [
- "disabled"
+ "sequence"
],
- "type": "TYPE_INVERTED",
+ "type": "TYPE_TREE",
"location": "LOCATION_SERIES",
"updated_at": "2021-04-15T01:30:15.01Z"
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json
similarity index 77%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json
index 8cfb12816a..3cecbefa13 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/disabled.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/service_group.json
@@ -1,10 +1,10 @@
{
"metadata": {
- "name": "disabled",
+ "name": "service_group",
"group": "default"
},
"tags": [
- "disabled"
+ "service_group"
],
"type": "TYPE_INVERTED",
"location": "LOCATION_SERIES",
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
index b63b29c2c2..fc728af744 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "instance_traffic",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -33,15 +33,8 @@
],
"entity": {
"tag_names": [
-
+ "service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
index 6d46d97799..0dc6f344c0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/network_address_alias.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "network_address_alias",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -37,14 +37,9 @@
],
"entity": {
"tag_names": [
+ "represent_service_id",
+ "represent_service_instance_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 52000,
- "unit": "DURATION_UNIT_WEEK"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
index 10aa497e74..85c5901c8c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "profile_task",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -52,14 +52,8 @@
],
"entity": {
"tag_names": [
+ "service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json
similarity index 51%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json
index b63b29c2c2..6fa6ccb244 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_log.json
@@ -1,15 +1,19 @@
{
"metadata": {
- "name": "instance_traffic",
- "group": "default"
+ "name": "profile_task_log",
+ "group": "default-stream"
},
"tag_families": [
{
"name": "data",
"tags": [
{
- "name": "data_binary",
- "type": "TAG_TYPE_DATA_BINARY"
+ "name": "task_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "operation_type",
+ "type": "TAG_TYPE_INT"
}
]
},
@@ -17,15 +21,11 @@
"name": "searchable",
"tags": [
{
- "name": "service_id",
- "type": "TAG_TYPE_STRING"
- },
- {
- "name": "last_ping",
+ "name": "operation_time",
"type": "TAG_TYPE_INT"
},
{
- "name": "id",
+ "name": "instance_id",
"type": "TAG_TYPE_STRING"
}
]
@@ -33,15 +33,8 @@
],
"entity": {
"tag_names": [
-
+ "instance_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json
similarity index 56%
copy from oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
copy to oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json
index b63b29c2c2..a5fcbcbf5c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/instance_traffic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/profile_task_segment_snapshot.json
@@ -1,14 +1,14 @@
{
"metadata": {
- "name": "instance_traffic",
- "group": "default"
+ "name": "profile_task_segment_snapshot",
+ "group": "default-stream"
},
"tag_families": [
{
"name": "data",
"tags": [
{
- "name": "data_binary",
+ "name": "stack_binary",
"type": "TAG_TYPE_DATA_BINARY"
}
]
@@ -17,31 +17,28 @@
"name": "searchable",
"tags": [
{
- "name": "service_id",
+ "name": "task_id",
"type": "TAG_TYPE_STRING"
},
{
- "name": "last_ping",
+ "name": "segment_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "dump_time",
"type": "TAG_TYPE_INT"
},
{
- "name": "id",
- "type": "TAG_TYPE_STRING"
+ "name": "sequence",
+ "type": "TAG_TYPE_INT"
}
]
}
],
"entity": {
"tag_names": [
-
+ "task_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
index 066fa45c38..7a8343d45c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "segment",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -82,12 +82,5 @@
"is_error"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
index 1a6f380e62..edd49ef287 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_client_side.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "service_instance_relation_client_side",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -41,6 +41,8 @@
],
"entity": {
"tag_names": [
+ "source_service_id",
+ "dest_service_id"
]
},
"opts": {
@@ -50,5 +52,5 @@
"unit": "DURATION_UNIT_DAY"
}
},
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
index 2fa5c396b4..a8c6089421 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_instance_relation_server_side.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "service_instance_relation_server_side",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -41,6 +41,8 @@
],
"entity": {
"tag_names": [
+ "source_service_id",
+ "dest_service_id"
]
},
"opts": {
@@ -50,5 +52,5 @@
"unit": "DURATION_UNIT_DAY"
}
},
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
index b12e95adaa..1f74420c2c 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_client_side.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "service_relation_client_side",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -33,14 +33,9 @@
],
"entity": {
"tag_names": [
+ "source_service_id",
+ "dest_service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
index 8844639714..13c2cc6177 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_relation_server_side.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "service_relation_server_side",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -33,14 +33,9 @@
],
"entity": {
"tag_names": [
+ "source_service_id",
+ "dest_service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
index ba0f1c07ed..b009a199cf 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/service_traffic.json
@@ -1,7 +1,7 @@
{
"metadata": {
- "name": "endpoint_traffic",
- "group": "default"
+ "name": "service_traffic",
+ "group": "default-stream"
},
"tag_families": [
{
@@ -37,14 +37,8 @@
],
"entity": {
"tag_names": [
+ "service_id"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 7,
- "unit": "DURATION_UNIT_DAY"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
index e2a404ff5f..b01e8baae0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/ui_template.json
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "ui_template",
- "group": "default"
+ "group": "default-stream"
},
"tag_families": [
{
@@ -37,14 +37,8 @@
],
"entity": {
"tag_names": [
+ "name"
]
},
- "opts": {
- "shard_num": 2,
- "ttl": {
- "val": 52000,
- "unit": "DURATION_UNIT_WEEK"
- }
- },
- "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+ "updated_at": "2021-04-15T01:30:15.01Z"
}
\ No newline at end of file
[skywalking] 10/24: add installer
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lujiajing pushed a commit to branch banyandb-integration-stream
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit a8e161f033cdbf1e3c32913265c2852ff849e29b
Author: Megrez Lu <lu...@gmail.com>
AuthorDate: Thu Dec 9 18:21:57 2021 +0800
add installer
---
.../src/main/resources/application.yml | 8 +
.../plugin/banyandb/BanyanDBIndexInstaller.java | 44 +++++
.../plugin/banyandb/BanyanDBStorageClient.java | 33 ++++
.../plugin/banyandb/BanyanDBStorageProvider.java | 3 +
.../storage/plugin/banyandb/StreamMetaInfo.java | 191 +++++++++++++++++++++
.../metadata/index_rules/segment/db.instance.json | 13 ++
.../src/main/resources/metadata/segment.json | 93 ++++++++++
7 files changed, 385 insertions(+)
diff --git a/oap-server/server-starter/src/main/resources/application.yml b/oap-server/server-starter/src/main/resources/application.yml
index d8430650b7..12db39b01b 100755
--- a/oap-server/server-starter/src/main/resources/application.yml
+++ b/oap-server/server-starter/src/main/resources/application.yml
@@ -260,6 +260,14 @@ storage:
storageGroup: ${SW_STORAGE_IOTDB_STORAGE_GROUP:root.skywalking}
sessionPoolSize: ${SW_STORAGE_IOTDB_SESSIONPOOL_SIZE:8} # If it's zero, the SessionPool size will be 2*CPU_Cores
fetchTaskLogMaxSize: ${SW_STORAGE_IOTDB_FETCH_TASK_LOG_MAX_SIZE:1000} # the max number of fetch task log in a request
+ banyandb:
+ host: ${SW_STORAGE_BANYANDB_HOST:127.0.0.1}
+ port: ${SW_STORAGE_BANYANDB_PORT:17912}
+ group: ${SW_STORAGE_BANYANDB_GROUP:default}
+ maxBulkSize: ${SW_STORAGE_BANYANDB_MAX_BULK_SIZE:5000}
+ flushInterval: ${SW_STORAGE_BANYANDB_FLUSH_INTERVAL:15}
+ concurrentWriteThreads: ${SW_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS:15}
+ fetchTaskLogMaxSize: ${SW_STORAGE_BANYANDB_FETCH_TASK_LOG_MAX_SIZE:1000} # the max number of fetch task log in a request
agent-analyzer:
selector: ${SW_AGENT_ANALYZER:default}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
new file mode 100644
index 0000000000..ffc5aaf586
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBIndexInstaller.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import org.apache.skywalking.oap.server.core.storage.StorageException;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.model.ModelInstaller;
+import org.apache.skywalking.oap.server.library.client.Client;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+
+public class BanyanDBIndexInstaller extends ModelInstaller {
+ public BanyanDBIndexInstaller(Client client, ModuleManager moduleManager) {
+ super(client, moduleManager);
+ }
+
+ @Override
+ protected boolean isExists(Model model) throws StorageException {
+ return false;
+ }
+
+ @Override
+ protected void createTable(Model model) throws StorageException {
+ StreamMetaInfo metaInfo = StreamMetaInfo.addModel(model);
+ if (metaInfo != null) {
+ ((BanyanDBStorageClient) client).createStream(metaInfo);
+ }
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
index 3ad4daf918..2ee45b12f0 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageClient.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
package org.apache.skywalking.oap.server.storage.plugin.banyandb;
import org.apache.skywalking.banyandb.v1.client.BanyanDBClient;
@@ -5,12 +23,17 @@ import org.apache.skywalking.banyandb.v1.client.StreamBulkWriteProcessor;
import org.apache.skywalking.banyandb.v1.client.StreamQuery;
import org.apache.skywalking.banyandb.v1.client.StreamQueryResponse;
import org.apache.skywalking.banyandb.v1.client.StreamWrite;
+import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
import org.apache.skywalking.oap.server.library.client.Client;
import org.apache.skywalking.oap.server.library.client.healthcheck.DelegatedHealthChecker;
import org.apache.skywalking.oap.server.library.client.healthcheck.HealthCheckable;
import org.apache.skywalking.oap.server.library.util.HealthChecker;
import java.io.IOException;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
/**
* BanyanDBStorageClient is a simple wrapper for the underlying {@link BanyanDBClient},
@@ -45,6 +68,16 @@ public class BanyanDBStorageClient implements Client, HealthCheckable {
}
}
+ public void createStream(StreamMetaInfo streamMetaInfo) {
+ Stream stm = this.client.define(streamMetaInfo.getStream());
+ if (stm != null) {
+ // TODO: should be fixed in SDK
+ this.client.defineIndexRules(stm, ZonedDateTime.from(Instant.now()),
+ ZonedDateTime.of(2099, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC),
+ streamMetaInfo.getIndexRules().toArray(new IndexRule[]{}));
+ }
+ }
+
public void write(StreamWrite streamWrite) {
this.client.write(streamWrite);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
index aade57b62b..64f8e96cbf 100644
--- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java
@@ -26,6 +26,7 @@ import org.apache.skywalking.oap.server.core.storage.StorageDAO;
import org.apache.skywalking.oap.server.core.storage.StorageModule;
import org.apache.skywalking.oap.server.core.storage.cache.INetworkAddressAliasDAO;
import org.apache.skywalking.oap.server.core.storage.management.UITemplateManagementDAO;
+import org.apache.skywalking.oap.server.core.storage.model.ModelCreator;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskLogQueryDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileTaskQueryDAO;
import org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
@@ -122,6 +123,8 @@ public class BanyanDBStorageProvider extends ModuleProvider {
this.client.registerChecker(healthChecker);
try {
this.client.connect();
+ BanyanDBIndexInstaller installer = new BanyanDBIndexInstaller(client, getManager());
+ getManager().find(CoreModule.NAME).provider().getService(ModelCreator.class).addModelListener(installer);
} catch (Exception e) {
throw new ModuleStartException(e.getMessage(), e);
}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
new file mode 100644
index 0000000000..1fbf2eb636
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/StreamMetaInfo.java
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.banyandb;
+
+import com.google.common.base.Charsets;
+import com.google.common.io.CharStreams;
+import com.google.protobuf.util.JsonFormat;
+import lombok.Builder;
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.metadata.Duration;
+import org.apache.skywalking.banyandb.v1.client.metadata.IndexRule;
+import org.apache.skywalking.banyandb.v1.client.metadata.Stream;
+import org.apache.skywalking.banyandb.v1.client.metadata.TagFamilySpec;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+@Getter
+@Builder
+@Slf4j
+public class StreamMetaInfo {
+ public static final String TAG_FAMILY_SEARCHABLE = "searchable";
+ public static final String TAG_FAMILY_DATA = "data";
+
+ private static final Map<String, StreamMetaInfo> STREAMS = new HashMap<>();
+
+ private final Model model;
+
+ /**
+ * stream is the metadata to be used for schema creation,
+ * 1. Read json from resources/metadata/{model.name}.json and deserialize to protobuf,
+ * 2. Iterate over tag families,
+ * 3. Iterate over tags in each tag family
+ * 4.
+ */
+ private final Stream stream;
+
+ private final List<IndexRule> indexRules;
+
+ public static StreamMetaInfo addModel(Model model) {
+ BanyandbMetadata.Stream pbStream = parseStreamFromJSON(model.getName());
+ if (pbStream == null) {
+ log.warn("fail to find the stream schema {}", model.getName());
+ return null;
+ }
+ BanyandbMetadata.Duration duration = pbStream.getOpts().getTtl();
+ Duration ttl = fromProtobuf(duration);
+ final Stream stream = new Stream(pbStream.getMetadata().getName(), pbStream.getOpts().getShardNum(), ttl);
+
+ List<IndexRule> indexRules = new ArrayList<>();
+
+ stream.setEntityTagNames(pbStream.getEntity().getTagNamesList());
+ for (BanyandbMetadata.TagFamilySpec pbTagFamilySpec : pbStream.getTagFamiliesList()) {
+ final TagFamilySpec tagFamilySpec = new TagFamilySpec(pbTagFamilySpec.getName());
+ final boolean needIndexParse = pbTagFamilySpec.getName().equals(TAG_FAMILY_SEARCHABLE);
+ for (final BanyandbMetadata.TagSpec pbTagSpec : pbTagFamilySpec.getTagsList()) {
+ tagFamilySpec.addTagSpec(parseTagSpec(pbTagSpec));
+
+ // if the tag family equals to "searchable", build index rules
+ if (needIndexParse) {
+ BanyandbMetadata.IndexRule pbIndexRule = parseIndexRulesFromJSON(model.getName(), pbTagSpec.getName());
+ if (pbIndexRule == null) {
+ log.warn("fail to find the index rule for {}", pbTagSpec.getName());
+ continue;
+ }
+ IndexRule.IndexType indexType = fromProtobuf(pbIndexRule.getType());
+ IndexRule.IndexLocation indexLocation = fromProtobuf(pbIndexRule.getLocation());
+ IndexRule indexRule = new IndexRule(pbIndexRule.getMetadata().getName(), indexType, indexLocation);
+ indexRule.setTags(new ArrayList<>(pbIndexRule.getTagsList()));
+ indexRules.add(indexRule);
+ }
+ }
+ }
+
+ return StreamMetaInfo.builder().model(model).stream(stream).indexRules(indexRules).build();
+ }
+
+ private static TagFamilySpec.TagSpec parseTagSpec(BanyandbMetadata.TagSpec pbTagSpec) {
+ switch (pbTagSpec.getType()) {
+ case TAG_TYPE_INT:
+ return TagFamilySpec.TagSpec.newIntTag(pbTagSpec.getName());
+ case TAG_TYPE_INT_ARRAY:
+ return TagFamilySpec.TagSpec.newIntArrayTag(pbTagSpec.getName());
+ case TAG_TYPE_STRING:
+ return TagFamilySpec.TagSpec.newStringTag(pbTagSpec.getName());
+ case TAG_TYPE_STRING_ARRAY:
+ return TagFamilySpec.TagSpec.newStringArrayTag(pbTagSpec.getName());
+ case TAG_TYPE_DATA_BINARY:
+ return TagFamilySpec.TagSpec.newBinaryTag(pbTagSpec.getName());
+ default:
+ throw new IllegalArgumentException("unrecognized tag type");
+ }
+ }
+
+ private static BanyandbMetadata.Stream parseStreamFromJSON(String name) {
+ try {
+ InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream("metadata/" + name + ".json");
+ if (is == null) {
+ return null;
+ }
+ String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
+ BanyandbMetadata.Stream.Builder b = BanyandbMetadata.Stream.newBuilder();
+ JsonFormat.parser().merge(result, b);
+ return b.build();
+ } catch (IOException ioEx) {
+ log.error("fail to read json", ioEx);
+ return null;
+ }
+ }
+
+ private static BanyandbMetadata.IndexRule parseIndexRulesFromJSON(String streamName, String name) {
+ try {
+ InputStream is = StreamMetaInfo.class.getClassLoader().getResourceAsStream(String.join("/",
+ new String[]{"metadata", "index_rules", streamName, name + ".json"}));
+ if (is == null) {
+ return null;
+ }
+ String result = CharStreams.toString(new InputStreamReader(is, Charsets.UTF_8));
+ BanyandbMetadata.IndexRule.Builder b = BanyandbMetadata.IndexRule.newBuilder();
+ JsonFormat.parser().merge(result, b);
+ return b.build();
+ } catch (IOException ioEx) {
+ log.error("fail to read json", ioEx);
+ return null;
+ }
+ }
+
+ // TODO: change modifier to public in SDK
+ static Duration fromProtobuf(BanyandbMetadata.Duration duration) {
+ switch (duration.getUnit()) {
+ case DURATION_UNIT_DAY:
+ return Duration.ofDays(duration.getVal());
+ case DURATION_UNIT_HOUR:
+ return Duration.ofHours(duration.getVal());
+ case DURATION_UNIT_MONTH:
+ return Duration.ofMonths(duration.getVal());
+ case DURATION_UNIT_WEEK:
+ return Duration.ofWeeks(duration.getVal());
+ default:
+ throw new IllegalArgumentException("unrecognized DurationUnit");
+ }
+ }
+
+ // TODO: change modifier to public in SDK
+ private static IndexRule.IndexType fromProtobuf(BanyandbMetadata.IndexRule.Type type) {
+ switch (type) {
+ case TYPE_TREE:
+ return IndexRule.IndexType.TREE;
+ case TYPE_INVERTED:
+ return IndexRule.IndexType.INVERTED;
+ default:
+ throw new IllegalArgumentException("unrecognized index type");
+ }
+ }
+
+ // TODO: change modifier to public in SDK
+ private static IndexRule.IndexLocation fromProtobuf(BanyandbMetadata.IndexRule.Location loc) {
+ switch (loc) {
+ case LOCATION_GLOBAL:
+ return IndexRule.IndexLocation.GLOBAL;
+ case LOCATION_SERIES:
+ return IndexRule.IndexLocation.SERIES;
+ default:
+ throw new IllegalArgumentException("unrecognized index location");
+ }
+ }
+}
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
new file mode 100644
index 0000000000..662cff2eac
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/index_rules/segment/db.instance.json
@@ -0,0 +1,13 @@
+{
+ "metadata": {
+ "id": 1,
+ "name": "db.instance",
+ "group": "default"
+ },
+ "tags": [
+ "db.instance"
+ ],
+ "type": "TYPE_INVERTED",
+ "location": "LOCATION_SERIES",
+ "updated_at": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file
diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
new file mode 100644
index 0000000000..1317d9ecf0
--- /dev/null
+++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/resources/metadata/segment.json
@@ -0,0 +1,93 @@
+{
+ "metadata": {
+ "name": "segment",
+ "group": "default"
+ },
+ "tag_families": [
+ {
+ "name": "data",
+ "tags": [
+ {
+ "name": "data_binary",
+ "type": "TAG_TYPE_DATA_BINARY"
+ }
+ ]
+ },
+ {
+ "name": "searchable",
+ "tags": [
+ {
+ "name": "trace_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "state",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "service_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "service_instance_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "endpoint_id",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "duration",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "start_time",
+ "type": "TAG_TYPE_INT"
+ },
+ {
+ "name": "http.method",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "status_code",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "db.type",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "db.instance",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "mq.queue",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "mq.topic",
+ "type": "TAG_TYPE_STRING"
+ },
+ {
+ "name": "mq.broker",
+ "type": "TAG_TYPE_STRING"
+ }
+ ]
+ }
+ ],
+ "entity": {
+ "tag_names": [
+ "service_id",
+ "service_instance_id",
+ "state"
+ ]
+ },
+ "opts": {
+ "shard_num": 2,
+ "ttl": {
+ "val": 7,
+ "unit": "DURATION_UNIT_DAY"
+ }
+ },
+ "updated_at_nanoseconds": "2021-04-15T01:30:15.01Z"
+}
\ No newline at end of file