You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by ha...@apache.org on 2025/07/16 05:38:48 UTC
(skywalking-banyandb) 01/01: Implement write queue for measure
This is an automated email from the ASF dual-hosted git repository.
hanahmily pushed a commit to branch topn
in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git
commit fc49cac2fd4bca052803c0e62c5818582b179980
Author: Gao Hongtao <ha...@gmail.com>
AuthorDate: Wed Jul 16 12:53:10 2025 +0800
Implement write queue for measure
Signed-off-by: Gao Hongtao <ha...@gmail.com>
---
.gitignore | 9 +
.cursorrules => AI_CODING_GUIDELINES.md | 289 ++--
CHANGES.md | 5 +
CONTRIBUTING.md | 11 +
api/data/data.go | 36 +-
api/data/measure.go | 27 +
banyand/backup/lifecycle/steps.go | 2 +-
banyand/dquery/dquery.go | 39 +-
banyand/dquery/measure.go | 4 +-
banyand/dquery/stream.go | 2 +-
banyand/dquery/topn.go | 2 +-
banyand/liaison/grpc/discovery.go | 9 -
banyand/liaison/grpc/measure.go | 102 --
banyand/liaison/grpc/registry_test.go | 4 +-
banyand/liaison/grpc/server.go | 54 +-
banyand/measure/datapoints.go | 31 +-
banyand/measure/flusher.go | 43 +
banyand/measure/introducer.go | 47 +
banyand/measure/measure.go | 4 +-
banyand/measure/measure_suite_test.go | 2 +-
banyand/measure/metadata.go | 177 ++-
banyand/measure/metrics.go | 2 +-
banyand/measure/part.go | 266 +++-
banyand/measure/snapshot.go | 5 +-
banyand/measure/{service.go => svc_data.go} | 264 +++-
banyand/measure/svc_liaison.go | 165 ++
banyand/measure/{service.go => svc_standalone.go} | 94 +-
banyand/{stream => measure}/syncer.go | 91 +-
banyand/measure/topn.go | 8 +-
banyand/measure/tstable.go | 53 +-
banyand/{stream => measure}/wqueue.go | 7 +-
banyand/measure/wqueue_test.go | 90 ++
.../write_standalone.go => measure/write_data.go} | 153 +-
banyand/measure/write_liaison.go | 216 +++
banyand/measure/{write.go => write_standalone.go} | 174 +-
banyand/stream/metadata.go | 47 +-
banyand/stream/metrics.go | 4 +-
banyand/stream/part.go | 9 +-
banyand/stream/snapshot.go | 3 +
banyand/stream/syncer.go | 21 +-
banyand/stream/tstable.go | 30 +-
banyand/stream/wqueue.go | 2 +-
banyand/stream/write.go | 430 -----
.../stream/{write_standalone.go => write_data.go} | 12 +-
banyand/stream/write_liaison.go | 10 +-
banyand/stream/write_standalone.go | 538 ++++---
dist/LICENSE | 562 -------
dist/LICENSE.tpl | 9 -
dist/NOTICE | 5 -
dist/licenses/license-dario.cat-mergo.txt | 28 -
.../license-github.com-Azure-go-ansiterm.txt | 21 -
.../license-github.com-Microsoft-go-winio.txt | 22 -
dist/licenses/license-github.com-Nvveen-Gotty.txt | 26 -
.../license-github.com-RoaringBitmap-roaring.txt | 235 ---
dist/licenses/license-github.com-SkyAPM-bluge.txt | 202 ---
dist/licenses/license-github.com-SkyAPM-clock.txt | 21 -
dist/licenses/license-github.com-SkyAPM-ice.txt | 202 ---
...icense-github.com-VictoriaMetrics-fastcache.txt | 22 -
.../license-github.com-apache-skywalking-cli.txt | 210 ---
...-aws-aws-sdk-go-v2-aws-protocol-eventstream.txt | 202 ---
...license-github.com-aws-aws-sdk-go-v2-config.txt | 202 ---
...se-github.com-aws-aws-sdk-go-v2-credentials.txt | 202 ---
...thub.com-aws-aws-sdk-go-v2-feature-ec2-imds.txt | 202 ---
...om-aws-aws-sdk-go-v2-internal-configsources.txt | 202 ---
...com-aws-aws-sdk-go-v2-internal-endpoints-v2.txt | 202 ---
...e-github.com-aws-aws-sdk-go-v2-internal-ini.txt | 202 ---
...e-github.com-aws-aws-sdk-go-v2-internal-v4a.txt | 202 ---
...-sdk-go-v2-service-internal-accept-encoding.txt | 202 ---
...aws-aws-sdk-go-v2-service-internal-checksum.txt | 202 ---
...ws-sdk-go-v2-service-internal-presigned-url.txt | 202 ---
...aws-aws-sdk-go-v2-service-internal-s3shared.txt | 202 ---
...nse-github.com-aws-aws-sdk-go-v2-service-s3.txt | 202 ---
...se-github.com-aws-aws-sdk-go-v2-service-sso.txt | 202 ---
...ithub.com-aws-aws-sdk-go-v2-service-ssooidc.txt | 202 ---
...se-github.com-aws-aws-sdk-go-v2-service-sts.txt | 202 ---
.../license-github.com-aws-aws-sdk-go-v2.txt | 202 ---
dist/licenses/license-github.com-aws-smithy-go.txt | 175 ---
.../license-github.com-axiomhq-hyperloglog.txt | 19 -
dist/licenses/license-github.com-beorn7-perks.txt | 20 -
.../license-github.com-bits-and-blooms-bitset.txt | 27 -
...nse-github.com-blevesearch-go-porterstemmer.txt | 19 -
.../license-github.com-blevesearch-mmap-go.txt | 25 -
.../license-github.com-blevesearch-segment.txt | 202 ---
...license-github.com-blevesearch-snowballstem.txt | 29 -
.../license-github.com-blevesearch-vellum.txt | 202 ---
.../license-github.com-caio-go-tdigest.txt | 21 -
.../license-github.com-cenkalti-backoff-v4.txt | 20 -
.../license-github.com-cespare-xxhash-v2.txt | 22 -
.../license-github.com-containerd-continuity.txt | 191 ---
.../license-github.com-coreos-go-semver.txt | 202 ---
.../license-github.com-coreos-go-systemd-v22.txt | 191 ---
.../license-github.com-cpuguy83-go-md2man-v2.txt | 21 -
.../license-github.com-davecgh-go-spew.txt | 15 -
.../license-github.com-dgryski-go-metro.txt | 24 -
dist/licenses/license-github.com-docker-cli.txt | 191 ---
dist/licenses/license-github.com-docker-docker.txt | 191 ---
.../license-github.com-docker-go-connections.txt | 191 ---
.../license-github.com-docker-go-units.txt | 191 ---
.../license-github.com-dustin-go-humanize.txt | 21 -
.../licenses/license-github.com-emirpasic-gods.txt | 41 -
...e-github.com-envoyproxy-protoc-gen-validate.txt | 202 ---
.../license-github.com-fsnotify-fsnotify.txt | 25 -
dist/licenses/license-github.com-go-chi-chi-v5.txt | 20 -
dist/licenses/license-github.com-go-ini-ini.txt | 191 ---
dist/licenses/license-github.com-go-logr-logr.txt | 201 ---
dist/licenses/license-github.com-go-logr-stdr.txt | 201 ---
dist/licenses/license-github.com-go-ole-go-ole.txt | 21 -
.../license-github.com-go-resty-resty-v2.txt | 21 -
.../license-github.com-go-task-slim-sprig-v3.txt | 19 -
...license-github.com-go-viper-mapstructure-v2.txt | 21 -
dist/licenses/license-github.com-goccy-go-json.txt | 21 -
dist/licenses/license-github.com-gogo-protobuf.txt | 35 -
.../license-github.com-golang-jwt-jwt-v4.txt | 9 -
.../license-github.com-golang-protobuf.txt | 28 -
dist/licenses/license-github.com-golang-snappy.txt | 27 -
dist/licenses/license-github.com-google-btree.txt | 202 ---
dist/licenses/license-github.com-google-go-cmp.txt | 27 -
dist/licenses/license-github.com-google-pprof.txt | 202 ---
dist/licenses/license-github.com-google-shlex.txt | 202 ---
dist/licenses/license-github.com-google-uuid.txt | 27 -
.../license-github.com-gorilla-websocket.txt | 22 -
...tem-go-grpc-middleware-providers-prometheus.txt | 201 ---
...ub.com-grpc-ecosystem-go-grpc-middleware-v2.txt | 201 ---
...ithub.com-grpc-ecosystem-go-grpc-middleware.txt | 201 ---
...ithub.com-grpc-ecosystem-go-grpc-prometheus.txt | 201 ---
...e-github.com-grpc-ecosystem-grpc-gateway-v2.txt | 27 -
...ense-github.com-grpc-ecosystem-grpc-gateway.txt | 27 -
.../license-github.com-hashicorp-golang-lru.txt | 364 -----
...icense-github.com-inconshreveable-mousetrap.txt | 201 ---
.../license-github.com-jonboulle-clockwork.txt | 201 ---
.../license-github.com-json-iterator-go.txt | 21 -
.../license-github.com-kamstrup-intmap.txt | 23 -
.../license-github.com-klauspost-compress.txt | 304 ----
.../license-github.com-klauspost-cpuid-v2.txt | 22 -
.../license-github.com-lufia-plan9stats.txt | 29 -
.../license-github.com-machinebox-graphql.txt | 201 ---
.../license-github.com-mattn-go-colorable.txt | 21 -
.../license-github.com-mattn-go-isatty.txt | 9 -
.../license-github.com-minio-crc64nvme.txt | 202 ---
.../licenses/license-github.com-minio-md5-simd.txt | 202 ---
.../license-github.com-minio-minio-go-v7.txt | 202 ---
.../license-github.com-moby-docker-image-spec.txt | 201 ---
dist/licenses/license-github.com-moby-sys-user.txt | 202 ---
dist/licenses/license-github.com-moby-term.txt | 191 ---
.../license-github.com-modern-go-concurrent.txt | 201 ---
.../license-github.com-modern-go-reflect2.txt | 201 ---
.../license-github.com-montanaflynn-stats.txt | 21 -
dist/licenses/license-github.com-mschoch-smat.txt | 202 ---
.../license-github.com-munnerz-goautoneg.txt | 31 -
dist/licenses/license-github.com-oklog-run.txt | 201 ---
.../licenses/license-github.com-onsi-ginkgo-v2.txt | 20 -
dist/licenses/license-github.com-onsi-gomega.txt | 20 -
...license-github.com-opencontainers-go-digest.txt | 192 ---
...icense-github.com-opencontainers-image-spec.txt | 191 ---
.../license-github.com-opencontainers-runc.txt | 191 ---
.../license-github.com-ory-dockertest-v3.txt | 201 ---
.../license-github.com-pelletier-go-toml-v2.txt | 22 -
dist/licenses/license-github.com-pkg-errors.txt | 23 -
.../license-github.com-pmezard-go-difflib.txt | 27 -
.../license-github.com-power-devops-perfstat.txt | 23 -
...license-github.com-prometheus-client-golang.txt | 201 ---
.../license-github.com-prometheus-client-model.txt | 201 ---
.../license-github.com-prometheus-common.txt | 201 ---
.../license-github.com-prometheus-procfs.txt | 201 ---
.../licenses/license-github.com-robfig-cron-v3.txt | 21 -
dist/licenses/license-github.com-rs-xid.txt | 19 -
dist/licenses/license-github.com-rs-zerolog.txt | 21 -
.../license-github.com-russross-blackfriday-v2.txt | 29 -
.../license-github.com-sagikazarmark-locafero.txt | 19 -
.../license-github.com-shirou-gopsutil-v3.txt | 61 -
.../license-github.com-shoenig-go-m1cpu.txt | 363 -----
.../license-github.com-sirupsen-logrus.txt | 21 -
dist/licenses/license-github.com-soheilhy-cmux.txt | 202 ---
.../license-github.com-sourcegraph-conc.txt | 21 -
dist/licenses/license-github.com-spf13-afero.txt | 174 --
dist/licenses/license-github.com-spf13-cast.txt | 21 -
dist/licenses/license-github.com-spf13-cobra.txt | 174 --
dist/licenses/license-github.com-spf13-pflag.txt | 28 -
dist/licenses/license-github.com-spf13-viper.txt | 21 -
.../license-github.com-stretchr-testify.txt | 21 -
.../license-github.com-subosito-gotenv.txt | 21 -
.../license-github.com-tklauser-go-sysconf.txt | 29 -
.../license-github.com-tklauser-numcpus.txt | 202 ---
...license-github.com-tmc-grpc-websocket-proxy.txt | 7 -
dist/licenses/license-github.com-urfave-cli-v2.txt | 21 -
.../license-github.com-xeipuuv-gojsonpointer.txt | 202 ---
.../license-github.com-xeipuuv-gojsonreference.txt | 202 ---
.../license-github.com-xeipuuv-gojsonschema.txt | 202 ---
.../license-github.com-xhit-go-str2duration-v2.txt | 27 -
.../license-github.com-xiang90-probing.txt | 22 -
.../licenses/license-github.com-xrash-smetrics.txt | 21 -
.../license-github.com-yusufpapurcu-wmi.txt | 20 -
.../license-github.com-zenizh-go-capturer.txt | 21 -
...cense-github.com-zinclabs-bluge-segment-api.txt | 202 ---
dist/licenses/license-go.etcd.io-bbolt.txt | 20 -
dist/licenses/license-go.etcd.io-etcd-api-v3.txt | 202 ---
.../license-go.etcd.io-etcd-client-pkg-v3.txt | 202 ---
.../licenses/license-go.etcd.io-etcd-client-v2.txt | 202 ---
.../licenses/license-go.etcd.io-etcd-client-v3.txt | 202 ---
dist/licenses/license-go.etcd.io-etcd-pkg-v3.txt | 202 ---
dist/licenses/license-go.etcd.io-etcd-raft-v3.txt | 202 ---
.../licenses/license-go.etcd.io-etcd-server-v3.txt | 202 ---
.../license-go.opentelemetry.io-auto-sdk.txt | 201 ---
...rumentation-google.golang.org-grpc-otelgrpc.txt | 201 ---
...otel-exporters-otlp-otlptrace-otlptracegrpc.txt | 201 ---
...ntelemetry.io-otel-exporters-otlp-otlptrace.txt | 201 ---
.../license-go.opentelemetry.io-otel-metric.txt | 201 ---
.../license-go.opentelemetry.io-otel-sdk.txt | 201 ---
.../license-go.opentelemetry.io-otel-trace.txt | 201 ---
dist/licenses/license-go.opentelemetry.io-otel.txt | 201 ---
.../license-go.opentelemetry.io-proto-otlp.txt | 201 ---
dist/licenses/license-go.uber.org-automaxprocs.txt | 19 -
dist/licenses/license-go.uber.org-mock.txt | 202 ---
dist/licenses/license-go.uber.org-multierr.txt | 19 -
dist/licenses/license-go.uber.org-zap.txt | 19 -
dist/licenses/license-golang.org-x-crypto.txt | 27 -
dist/licenses/license-golang.org-x-exp.txt | 27 -
dist/licenses/license-golang.org-x-mod.txt | 27 -
dist/licenses/license-golang.org-x-net.txt | 27 -
dist/licenses/license-golang.org-x-sys.txt | 27 -
dist/licenses/license-golang.org-x-text.txt | 27 -
dist/licenses/license-golang.org-x-time.txt | 27 -
dist/licenses/license-golang.org-x-tools.txt | 27 -
...e-google.golang.org-genproto-googleapis-api.txt | 202 ---
...e-google.golang.org-genproto-googleapis-rpc.txt | 202 ---
.../license-google.golang.org-genproto.txt | 202 ---
dist/licenses/license-google.golang.org-grpc.txt | 202 ---
.../license-google.golang.org-protobuf.txt | 27 -
.../license-gopkg.in-natefinch-lumberjack.v2.txt | 21 -
dist/licenses/license-gopkg.in-yaml.v2.txt | 201 ---
dist/licenses/license-gopkg.in-yaml.v3.txt | 50 -
dist/licenses/license-sigs.k8s.io-yaml.txt | 306 ----
.../license-skywalking.apache.org-repo-goapi.txt | 201 ---
.../license--babel-helper-string-parser.txt | 22 -
.../license--babel-helper-validator-identifier.txt | 22 -
.../licenses/ui-licenses/license--babel-parser.txt | 19 -
dist/licenses/ui-licenses/license--babel-types.txt | 22 -
.../ui-licenses/license--ctrl-tinycolor.txt | 7 -
.../license--element-plus-icons-vue.txt | 21 -
.../ui-licenses/license--floating-ui-core.txt | 20 -
.../ui-licenses/license--floating-ui-dom.txt | 20 -
.../ui-licenses/license--floating-ui-utils.txt | 20 -
.../license--jridgewell-sourcemap-codec.txt | 21 -
.../license--parcel-watcher-linux-x64-glibc.txt | 21 -
.../license--parcel-watcher-linux-x64-musl.txt | 21 -
.../ui-licenses/license--parcel-watcher.txt | 21 -
.../ui-licenses/license--popperjs-core.txt | 20 -
.../ui-licenses/license--types-lodash-es.txt | 21 -
.../licenses/ui-licenses/license--types-lodash.txt | 21 -
.../ui-licenses/license--types-web-bluetooth.txt | 21 -
.../ui-licenses/license--vue-compiler-core.txt | 21 -
.../ui-licenses/license--vue-compiler-dom.txt | 21 -
.../ui-licenses/license--vue-compiler-sfc.txt | 21 -
.../ui-licenses/license--vue-compiler-ssr.txt | 21 -
.../ui-licenses/license--vue-devtools-api.txt | 0
.../ui-licenses/license--vue-reactivity.txt | 21 -
.../ui-licenses/license--vue-runtime-core.txt | 21 -
.../ui-licenses/license--vue-runtime-dom.txt | 21 -
.../ui-licenses/license--vue-server-renderer.txt | 21 -
dist/licenses/ui-licenses/license--vue-shared.txt | 21 -
dist/licenses/ui-licenses/license--vueuse-core.txt | 21 -
.../ui-licenses/license--vueuse-metadata.txt | 21 -
.../ui-licenses/license--vueuse-shared.txt | 21 -
dist/licenses/ui-licenses/license-argparse.txt | 254 ---
.../ui-licenses/license-async-validator.txt | 9 -
dist/licenses/ui-licenses/license-asynckit.txt | 21 -
dist/licenses/ui-licenses/license-axios.txt | 7 -
dist/licenses/ui-licenses/license-braces.txt | 21 -
.../license-call-bind-apply-helpers.txt | 21 -
dist/licenses/ui-licenses/license-chokidar.txt | 21 -
dist/licenses/ui-licenses/license-codemirror.txt | 21 -
.../ui-licenses/license-combined-stream.txt | 19 -
dist/licenses/ui-licenses/license-csstype.txt | 19 -
dist/licenses/ui-licenses/license-dayjs.txt | 21 -
.../ui-licenses/license-delayed-stream.txt | 19 -
dist/licenses/ui-licenses/license-detect-libc.txt | 201 ---
dist/licenses/ui-licenses/license-dunder-proto.txt | 21 -
dist/licenses/ui-licenses/license-echarts.txt | 222 ---
dist/licenses/ui-licenses/license-element-plus.txt | 0
dist/licenses/ui-licenses/license-entities.txt | 11 -
.../ui-licenses/license-es-define-property.txt | 21 -
dist/licenses/ui-licenses/license-es-errors.txt | 21 -
.../ui-licenses/license-es-object-atoms.txt | 21 -
.../ui-licenses/license-es-set-tostringtag.txt | 21 -
dist/licenses/ui-licenses/license-escape-html.txt | 24 -
.../licenses/ui-licenses/license-estree-walker.txt | 7 -
dist/licenses/ui-licenses/license-fill-range.txt | 21 -
.../ui-licenses/license-follow-redirects.txt | 18 -
dist/licenses/ui-licenses/license-form-data.txt | 19 -
.../licenses/ui-licenses/license-function-bind.txt | 20 -
.../licenses/ui-licenses/license-get-intrinsic.txt | 21 -
dist/licenses/ui-licenses/license-get-proto.txt | 21 -
dist/licenses/ui-licenses/license-gopd.txt | 21 -
dist/licenses/ui-licenses/license-graceful-fs.txt | 15 -
dist/licenses/ui-licenses/license-has-symbols.txt | 21 -
.../ui-licenses/license-has-tostringtag.txt | 21 -
dist/licenses/ui-licenses/license-hasown.txt | 21 -
dist/licenses/ui-licenses/license-immutable.txt | 21 -
dist/licenses/ui-licenses/license-is-extglob.txt | 21 -
dist/licenses/ui-licenses/license-is-glob.txt | 21 -
dist/licenses/ui-licenses/license-is-number.txt | 21 -
dist/licenses/ui-licenses/license-js-yaml.txt | 21 -
dist/licenses/ui-licenses/license-jsonfile.txt | 15 -
dist/licenses/ui-licenses/license-lodash-es.txt | 47 -
.../ui-licenses/license-lodash-unified.txt | 0
dist/licenses/ui-licenses/license-lodash.txt | 47 -
dist/licenses/ui-licenses/license-magic-string.txt | 7 -
.../ui-licenses/license-math-intrinsics.txt | 21 -
dist/licenses/ui-licenses/license-memoize-one.txt | 21 -
dist/licenses/ui-licenses/license-micromatch.txt | 21 -
dist/licenses/ui-licenses/license-mime-db.txt | 23 -
dist/licenses/ui-licenses/license-mime-types.txt | 23 -
dist/licenses/ui-licenses/license-mitt.txt | 21 -
dist/licenses/ui-licenses/license-nanoid.txt | 20 -
.../ui-licenses/license-node-addon-api.txt | 9 -
.../ui-licenses/license-normalize-wheel-es.txt | 30 -
dist/licenses/ui-licenses/license-picocolors.txt | 15 -
dist/licenses/ui-licenses/license-picomatch.txt | 21 -
dist/licenses/ui-licenses/license-pinia.txt | 21 -
dist/licenses/ui-licenses/license-postcss.txt | 20 -
.../ui-licenses/license-proxy-from-env.txt | 20 -
dist/licenses/ui-licenses/license-readdirp.txt | 21 -
dist/licenses/ui-licenses/license-sass.txt | 1661 --------------------
dist/licenses/ui-licenses/license-semver.txt | 15 -
.../licenses/ui-licenses/license-source-map-js.txt | 28 -
.../ui-licenses/license-to-regex-range.txt | 21 -
dist/licenses/ui-licenses/license-tslib.txt | 12 -
dist/licenses/ui-licenses/license-universalify.txt | 20 -
dist/licenses/ui-licenses/license-vue-demi.txt | 21 -
dist/licenses/ui-licenses/license-vue-router.txt | 21 -
dist/licenses/ui-licenses/license-vue.txt | 21 -
dist/licenses/ui-licenses/license-yaml.txt | 13 -
dist/licenses/ui-licenses/license-zrender.txt | 29 -
pkg/cmdsetup/data.go | 8 +-
pkg/cmdsetup/liaison.go | 14 +-
pkg/cmdsetup/standalone.go | 11 +-
pkg/test/setup/setup.go | 37 +-
test/cases/stream/stream.go | 4 +-
.../distributed/lifecycle/lifecycle_suite_test.go | 1 +
test/integration/standalone/other/disk_test.go | 4 +-
340 files changed, 2142 insertions(+), 28500 deletions(-)
diff --git a/.gitignore b/.gitignore
index 343b30b3..474f67fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,3 +64,12 @@ gomock_reflect*
# profile result
*.prof
+
+# cursor
+.cursorrules
+.cursor/
+
+# Claude
+Claude.md
+.claude/
+
diff --git a/.cursorrules b/AI_CODING_GUIDELINES.md
similarity index 54%
rename from .cursorrules
rename to AI_CODING_GUIDELINES.md
index 6fd656d8..5b1d6b11 100644
--- a/.cursorrules
+++ b/AI_CODING_GUIDELINES.md
@@ -15,23 +15,23 @@
# specific language governing permissions and limitations
# under the License.
-# Cursor Rules for SkyWalking BanyanDB Go Project
-# This file configures Cursor to generate Go code that follows the project's formatting and linting standards
+# AI Assistant Rules for SkyWalking BanyanDB Go Project
+# This file provides coding standards and guidelines for AI assistants
+# Compatible with Claude, Cursor, GitHub Copilot, and other LLMs
-# Language-specific rules for Go
-[go]
-description = """
-When generating Go code for the SkyWalking BanyanDB project, follow these strict rules:
+## PROJECT OVERVIEW
+This is the SkyWalking BanyanDB project - a distributed time-series database written in Go.
+Follow these strict coding standards when generating or modifying Go code.
-## FORMATTING RULES (from .golangci-format.yml):
+## FORMATTING RULES
1. Use gofumpt for code formatting (stricter than gofmt)
2. Use gci for import organization with specific sections:
- standard library imports
- default imports
- github.com/apache/skywalking-banyandb/ prefix imports
-3. Maximum line length: 170 characters (from .golangci.yml lll setting)
+3. Maximum line length: 170 characters
-## LINTING RULES (from .golangci.yml):
+## LINTING RULES
1. Variable shadowing prevention (govet shadow enabled)
2. Import aliases for specific packages (importas settings)
3. Error handling patterns (errcheck, errorlint, errname)
@@ -39,13 +39,14 @@ When generating Go code for the SkyWalking BanyanDB project, follow these strict
5. Security considerations (gosec)
6. Documentation standards (godot scope: toplevel)
-## IMPORT ORGANIZATION:
+## IMPORT ORGANIZATION
Follow the gci sections order:
1. Standard library imports
2. Default imports
3. github.com/apache/skywalking-banyandb/ prefix imports
-## IMPORT ALIASES (from importas settings):
+## IMPORT ALIASES
+Use these specific aliases for protobuf packages:
- github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1 → commonv1
- github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1 → databasev1
- github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1 → modelv1
@@ -55,20 +56,21 @@ Follow the gci sections order:
- github.com/apache/skywalking-banyandb/api/proto/banyandb/cluster/v1 → clusterv1
- github.com/apache/skywalking-banyandb/pkg/pb/v1 → pbv1
-## ERROR HANDLING PATTERNS:
+## ERROR HANDLING PATTERNS
1. Always check errors immediately after function calls
2. Use descriptive error variable names to avoid shadowing
3. Follow the errname convention for error types
4. Use errorlint for consistent error handling
+5. Wrap errors with context using fmt.Errorf and %w verb
-## VARIABLE SHADOWING PREVENTION:
+## VARIABLE SHADOWING PREVENTION
1. NEVER shadow variables in the same scope
2. Use unique variable names in nested scopes
3. Be especially careful with common variable names like 'err', 'ctx', 'v', 'i', 'n', etc.
4. When using range loops, use descriptive variable names instead of single letters
5. In if statements with variable declarations, ensure the variable name doesn't conflict with outer scope
-## CODE STYLE:
+## CODE STYLE
1. Use gocritic enabled checks for code quality
2. Follow exhaustive switch/map patterns
3. Use proper field alignment (govet fieldalignment)
@@ -76,101 +78,131 @@ Follow the gci sections order:
5. Use standard library variables when available (usestdlibvars)
6. Follow whitespace rules (whitespace linter)
-## SECURITY:
+## SECURITY
1. Follow gosec security guidelines
2. Be aware of integer overflow conversions (G115 excluded in config)
-## DOCUMENTATION:
+## DOCUMENTATION
1. Add top-level comments for exported functions and types (godot scope: toplevel)
2. Follow docStub patterns for documentation
+3. Use proper sentence structure and punctuation
+4. Describe what the function does, not how it does it
-## CONSTANTS AND MAGIC NUMBERS:
+## CONSTANTS AND MAGIC NUMBERS
1. Use goconst for repeated string literals (min-occurrences: 4)
2. Avoid magic numbers, use named constants
-## COMPLEXITY:
+## COMPLEXITY
1. Keep cyclomatic complexity low (gocyclo)
2. Avoid deeply nested structures
3. Use proper abstraction levels
-## NAMING CONVENTIONS:
+## NAMING CONVENTIONS
1. Use descriptive names for variables, especially in nested scopes
2. Follow Go naming conventions
3. Use proper package naming
4. Follow the project's specific naming patterns
-## TESTING:
+## TESTING
1. Follow proper test naming conventions
2. Use appropriate test helpers
3. Follow the project's testing patterns
-## EXCLUDED PATTERNS:
+## EXCLUDED PATTERNS
The following file patterns are excluded from linting:
- *.pb.go (protobuf generated files)
- *.pb.gw.go (protobuf gateway files)
- *.gen.go (generated files)
- *_mock.go (mock files)
- *_test.go files have relaxed errcheck rules
-"""
-
-# Specific patterns to avoid
-avoid_patterns = [
- # Variable shadowing patterns
- "if err := someFunc(); err != nil {",
- "if v := someValue(); v != nil {",
- "for _, v := range items {",
- "for i, v := range items {",
- "switch v := someValue(); v {",
- "defer func() { if err := cleanup(); err != nil {",
-
- # Import organization violations
- "import (\n\t\"fmt\"\n\t\"github.com/apache/skywalking-banyandb/pkg/something\"\n\t\"os\"\n)",
-
- # Error handling violations
- "someFunc() // ignoring error",
- "if err != nil {\n\treturn\n}",
-
- # Style violations
- "if x == true {",
- "if x == false {",
- "if x != nil && x.y == true {",
-
- # Documentation violations
- "// Function does something",
- "// This function does something",
-]
-
-# Preferred patterns to use
-preferred_patterns = [
- # Proper variable naming to avoid shadowing
- "if resultErr := someFunc(); resultErr != nil {",
- "if item := someValue(); item != nil {",
- "for _, item := range items {",
- "for idx, item := range items {",
- "switch value := someValue(); value {",
- "defer func() { if cleanupErr := cleanup(); cleanupErr != nil {",
-
- # Proper import organization
- "import (\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/apache/skywalking-banyandb/pkg/something\"\n)",
-
- # Proper error handling
- "if err := someFunc(); err != nil {\n\treturn fmt.Errorf(\"failed to do something: %w\", err)\n}",
-
- # Proper style
- "if x {",
- "if !x {",
- "if x != nil && x.y {",
-
- # Proper documentation
- "// DoSomething performs a specific action.",
- "// DoSomething performs a specific action and returns an error if it fails.",
-]
-
-# Code generation guidelines
-guidelines = """
-When generating Go code for this project:
-
-## IMPORTANT: Always follow these steps:
+
+## PATTERNS TO AVOID
+
+### Variable Shadowing Patterns (BAD):
+```go
+if err := someFunc(); err != nil {
+if v := someValue(); v != nil {
+for _, v := range items {
+for i, v := range items {
+switch v := someValue(); v {
+defer func() { if err := cleanup(); err != nil {
+```
+
+### Import Organization Violations (BAD):
+```go
+import (
+ "fmt"
+ "github.com/apache/skywalking-banyandb/pkg/something"
+ "os"
+)
+```
+
+### Error Handling Violations (BAD):
+```go
+someFunc() // ignoring error
+if err != nil {
+ return
+}
+```
+
+### Style Violations (BAD):
+```go
+if x == true {
+if x == false {
+if x != nil && x.y == true {
+```
+
+### Documentation Violations (BAD):
+```go
+// Function does something
+// This function does something
+```
+
+## PREFERRED PATTERNS
+
+### Proper Variable Naming to Avoid Shadowing (GOOD):
+```go
+if resultErr := someFunc(); resultErr != nil {
+if item := someValue(); item != nil {
+for _, item := range items {
+for idx, item := range items {
+switch value := someValue(); value {
+defer func() { if cleanupErr := cleanup(); cleanupErr != nil {
+```
+
+### Proper Import Organization (GOOD):
+```go
+import (
+ "fmt"
+ "os"
+
+ "github.com/apache/skywalking-banyandb/pkg/something"
+)
+```
+
+### Proper Error Handling (GOOD):
+```go
+if err := someFunc(); err != nil {
+ return fmt.Errorf("failed to do something: %w", err)
+}
+```
+
+### Proper Style (GOOD):
+```go
+if x {
+if !x {
+if x != nil && x.y {
+```
+
+### Proper Documentation (GOOD):
+```go
+// DoSomething performs a specific action.
+// DoSomething performs a specific action and returns an error if it fails.
+```
+
+## CODE GENERATION GUIDELINES
+
+### IMPORTANT: Always follow these steps:
1. Check for existing variable names in the current scope before declaring new ones
2. Use descriptive variable names to avoid shadowing
3. Organize imports according to gci sections
@@ -180,7 +212,7 @@ When generating Go code for this project:
7. Keep line length under 170 characters
8. Use gofumpt formatting style
-## Example of good code structure:
+### Example of Good Code Structure:
```go
package example
@@ -215,7 +247,7 @@ func ProcessData(ctx context.Context, data []byte) ([]byte, error) {
}
```
-## Example of bad code (violates multiple rules):
+### Example of Bad Code (violates multiple rules):
```go
package example
@@ -240,74 +272,37 @@ func ProcessData(ctx context.Context, data []byte) ([]byte, error) {
return nil
}
```
-"""
-
-# Linting integration
-linting = """
-This project uses golangci-lint with strict configuration.
-Always run 'golangci-lint run' to verify generated code.
-Key linters enabled:
-- govet (including shadow and fieldalignment)
-- gci (import organization)
-- gofumpt (formatting)
-- errcheck, errorlint, errname (error handling)
-- gosimple, staticcheck, stylecheck (code quality)
-- gosec (security)
-- godot (documentation)
-"""
-
-# Common variable names to be extra careful with
-sensitive_variables = [
- "err", "error", "ctx", "context", "v", "value", "i", "idx", "index",
- "n", "num", "count", "size", "len", "length", "k", "key", "val",
- "result", "res", "data", "item", "obj", "object", "msg", "message",
- "resp", "response", "req", "request", "client", "conn", "connection"
-]
-
-# Naming conventions for nested scopes
-naming_conventions = """
+
+## COMMON VARIABLE NAMES TO BE EXTRA CAREFUL WITH
+err, error, ctx, context, v, value, i, idx, index,
+n, num, count, size, len, length, k, key, val,
+result, res, data, item, obj, object, msg, message,
+resp, response, req, request, client, conn, connection
+
+## NAMING CONVENTIONS FOR NESTED SCOPES
When dealing with nested scopes, use these naming patterns:
- Outer scope: 'err', 'ctx', 'v', 'i'
- Inner scope: 'innerErr', 'innerCtx', 'item', 'idx'
- Deep nested: 'deepErr', 'deepCtx', 'subItem', 'subIdx'
- Or use descriptive prefixes: 'processErr', 'validateErr', 'parseErr'
-"""
-# Import organization rules
-import_organization = """
-Follow this exact import order:
-1. Standard library imports
-2. Default imports (third-party)
-3. github.com/apache/skywalking-banyandb/ prefix imports
+## USAGE INSTRUCTIONS FOR AI ASSISTANTS
-Use these specific aliases:
-- commonv1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1
-- databasev1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1
-- modelv1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1
-- propertyv1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/property/v1
-- measurev1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1
-- streamv1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/stream/v1
-- clusterv1 for github.com/apache/skywalking-banyandb/api/proto/banyandb/cluster/v1
-- pbv1 for github.com/apache/skywalking-banyandb/pkg/pb/v1
-"""
-
-# Error handling patterns
-error_handling = """
-Follow these error handling patterns:
-1. Always check errors immediately after function calls
-2. Use descriptive error variable names to avoid shadowing
-3. Wrap errors with context using fmt.Errorf and %w verb
-4. Follow the errname convention for error types
-5. Use proper error return patterns
-6. Avoid ignoring errors unless explicitly documented
-"""
-
-# Documentation standards
-documentation = """
-Follow these documentation standards:
-1. Add top-level comments for all exported functions and types
-2. Use proper sentence structure and punctuation
-3. Describe what the function does, not how it does it
-4. Include parameter and return value descriptions for complex functions
-5. Follow the godot scope: toplevel setting
-"""
\ No newline at end of file
+### For Claude:
+- Use this file as a reference when generating Go code
+- Follow all formatting, naming, and error handling patterns
+- Pay special attention to variable shadowing prevention
+- Use the provided import aliases and organization rules
+
+### For Cursor:
+- This file can be used as .cursorrules
+- Follow all the patterns and guidelines specified
+
+### For GitHub Copilot:
+- Use this as a reference for code generation
+- Follow the coding standards and patterns
+
+### For Other LLMs:
+- Use this as a comprehensive coding guide
+- Follow all specified patterns and conventions
+- Pay attention to the project-specific requirements
\ No newline at end of file
diff --git a/CHANGES.md b/CHANGES.md
index 5cc0071b..b3f0134d 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -21,6 +21,7 @@ Release Notes.
- Implement fadvise for large files to prevent page cache pollution.
- Data Model: Introduce the `Trace` data model to store the trace/span data.
- Push down aggregation for topN query.
+- Introduce write queue mechanism in liaison nodes to efficiently synchronize stream and measure partition folders, improving write throughput and consistency
### Bug Fixes
@@ -29,6 +30,10 @@ Release Notes.
- Fix the crash when collecting the metrics from a closed segment.
- Fix topN parsing panic when the criteria is set.
+### Document
+
+- Introduce AI_CODING_GUIDELINES.md to provide guidelines for using AI assistants (like Claude, Cursor, GitHub Copilot) in development, ensuring generated code follows project standards around variable shadowing, imports, error handling, code style and documentation
+
## 0.8.0
### Features
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 2b23eca2..d24dadc0 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -93,6 +93,17 @@ make lint
If you found some errors in the output of the above command, try to `make format` to fix some obvious style issues. As for the complicated errors, please correct them manually.
+## AI-Assisted Development
+
+If you're using AI assistants (like Claude, Cursor, GitHub Copilot, etc.) to help with code generation, please refer to our [AI Coding Guidelines](AI_CODING_GUIDELINES.md) to ensure the generated code follows our project's coding standards and linting rules.
+
+The guidelines cover:
+- Variable shadowing prevention
+- Import organization and aliases
+- Error handling patterns
+- Code style and documentation standards
+- Common patterns to avoid and preferred alternatives
+
## Update licenses
If you import new dependencies or upgrade an existing one, trigger the licenses generator
diff --git a/api/data/data.go b/api/data/data.go
index 065d7949..dc3ce00b 100644
--- a/api/data/data.go
+++ b/api/data/data.go
@@ -30,18 +30,21 @@ import (
var (
// TopicMap is the map of topic name to topic.
TopicMap = map[string]bus.Topic{
- TopicStreamWrite.String(): TopicStreamWrite,
- TopicStreamQuery.String(): TopicStreamQuery,
- TopicMeasureWrite.String(): TopicMeasureWrite,
- TopicMeasureQuery.String(): TopicMeasureQuery,
- TopicTopNQuery.String(): TopicTopNQuery,
- TopicPropertyDelete.String(): TopicPropertyDelete,
- TopicPropertyQuery.String(): TopicPropertyQuery,
- TopicPropertyUpdate.String(): TopicPropertyUpdate,
- TopicStreamPartSync.String(): TopicStreamPartSync,
- TopicPropertyRepair.String(): TopicPropertyRepair,
- TopicStreamSeriesIndexWrite.String(): TopicStreamSeriesIndexWrite,
- TopicStreamLocalIndexWrite.String(): TopicStreamLocalIndexWrite,
+ TopicStreamWrite.String(): TopicStreamWrite,
+ TopicStreamQuery.String(): TopicStreamQuery,
+ TopicMeasureWrite.String(): TopicMeasureWrite,
+ TopicMeasureQuery.String(): TopicMeasureQuery,
+ TopicTopNQuery.String(): TopicTopNQuery,
+ TopicPropertyDelete.String(): TopicPropertyDelete,
+ TopicPropertyQuery.String(): TopicPropertyQuery,
+ TopicPropertyUpdate.String(): TopicPropertyUpdate,
+ TopicStreamPartSync.String(): TopicStreamPartSync,
+ TopicMeasurePartSync.String(): TopicMeasurePartSync,
+ TopicMeasureSeriesIndexInsert.String(): TopicMeasureSeriesIndexInsert,
+ TopicMeasureSeriesIndexUpdate.String(): TopicMeasureSeriesIndexUpdate,
+ TopicPropertyRepair.String(): TopicPropertyRepair,
+ TopicStreamSeriesIndexWrite.String(): TopicStreamSeriesIndexWrite,
+ TopicStreamLocalIndexWrite.String(): TopicStreamLocalIndexWrite,
}
// TopicRequestMap is the map of topic name to request message.
@@ -74,6 +77,15 @@ var (
TopicStreamPartSync: func() proto.Message {
return nil
},
+ TopicMeasurePartSync: func() proto.Message {
+ return nil
+ },
+ TopicMeasureSeriesIndexInsert: func() proto.Message {
+ return nil
+ },
+ TopicMeasureSeriesIndexUpdate: func() proto.Message {
+ return nil
+ },
TopicPropertyRepair: func() proto.Message {
return &propertyv1.InternalRepairRequest{}
},
diff --git a/api/data/measure.go b/api/data/measure.go
index 512848e3..41d08b24 100644
--- a/api/data/measure.go
+++ b/api/data/measure.go
@@ -57,3 +57,30 @@ var MeasureDeleteExpiredSegmentsKindVersion = common.KindVersion{
// TopicMeasureDeleteExpiredSegments is the measure delete topic.
var TopicMeasureDeleteExpiredSegments = bus.BiTopic(MeasureDeleteExpiredSegmentsKindVersion.String())
+
+// MeasurePartSyncKindVersion is the version tag of measure part sync kind.
+var MeasurePartSyncKindVersion = common.KindVersion{
+ Version: "v1",
+ Kind: "measure-part-sync",
+}
+
+// TopicMeasurePartSync is the measure part sync topic.
+var TopicMeasurePartSync = bus.BiTopic(MeasurePartSyncKindVersion.String())
+
+// MeasureSeriesIndexInsertKindVersion is the version tag of measure series index insert kind.
+var MeasureSeriesIndexInsertKindVersion = common.KindVersion{
+ Version: "v1",
+ Kind: "measure-series-index-insert",
+}
+
+// TopicMeasureSeriesIndexInsert is the measure series index insert topic.
+var TopicMeasureSeriesIndexInsert = bus.BiTopic(MeasureSeriesIndexInsertKindVersion.String())
+
+// MeasureSeriesIndexUpdateKindVersion is the version tag of measure series index update kind.
+var MeasureSeriesIndexUpdateKindVersion = common.KindVersion{
+ Version: "v1",
+ Kind: "measure-series-index-update",
+}
+
+// TopicMeasureSeriesIndexUpdate is the measure series index update topic.
+var TopicMeasureSeriesIndexUpdate = bus.BiTopic(MeasureSeriesIndexUpdateKindVersion.String())
diff --git a/banyand/backup/lifecycle/steps.go b/banyand/backup/lifecycle/steps.go
index e94f12d3..f5c43b8d 100644
--- a/banyand/backup/lifecycle/steps.go
+++ b/banyand/backup/lifecycle/steps.go
@@ -100,7 +100,7 @@ func (l *lifecycleService) setupQuerySvc(ctx context.Context, streamDir, measure
}
}
if measureDir != "" {
- measureSVC, err = measure.NewReadonlyService(l.metadata, l.omr, l.pm)
+ measureSVC, err = measure.NewReadonlyDataSVC(l.metadata, l.omr, l.pm)
if err != nil {
return nil, nil, err
}
diff --git a/banyand/dquery/dquery.go b/banyand/dquery/dquery.go
index c6f9ee41..a9809fa6 100644
--- a/banyand/dquery/dquery.go
+++ b/banyand/dquery/dquery.go
@@ -29,8 +29,6 @@ import (
"github.com/apache/skywalking-banyandb/api/common"
"github.com/apache/skywalking-banyandb/api/data"
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
- databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
- measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
"github.com/apache/skywalking-banyandb/banyand/measure"
"github.com/apache/skywalking-banyandb/banyand/metadata"
@@ -41,7 +39,6 @@ import (
"github.com/apache/skywalking-banyandb/pkg/logger"
"github.com/apache/skywalking-banyandb/pkg/query/executor"
"github.com/apache/skywalking-banyandb/pkg/run"
- "github.com/apache/skywalking-banyandb/pkg/schema"
)
const (
@@ -49,17 +46,11 @@ const (
hotStageName = "hot"
)
-var (
- _ run.Service = (*queryService)(nil)
- distributedQueryScope = observability.RootScope.SubScope("dquery")
- streamScope = distributedQueryScope.SubScope("stream")
- measureScope = distributedQueryScope.SubScope("measure")
-)
+var _ run.Service = (*queryService)(nil)
// Service is the interface for distributed query service.
type Service interface {
run.Unit
- measure.TopNService
}
var _ Service = (*queryService)(nil)
@@ -79,7 +70,8 @@ type queryService struct {
}
// NewService return a new query service.
-func NewService(metaService metadata.Repo, pipeline queue.Server, broadcaster bus.Broadcaster, qClient queue.Client, omr observability.MetricsRegistry,
+func NewService(metaService metadata.Repo, pipeline queue.Server, broadcaster bus.Broadcaster, omr observability.MetricsRegistry,
+ streamSchemaSVC stream.Service, measureSchemaSVC measure.Service,
) (Service, error) {
svc := &queryService{
metaService: metaService,
@@ -88,13 +80,14 @@ func NewService(metaService metadata.Repo, pipeline queue.Server, broadcaster bu
omr: omr,
}
svc.sqp = &streamQueryProcessor{
- queryService: svc,
- broadcaster: broadcaster,
+ queryService: svc,
+ streamService: streamSchemaSVC,
+ broadcaster: broadcaster,
}
svc.mqp = &measureQueryProcessor{
- queryService: svc,
- qClient: qClient,
- broadcaster: broadcaster,
+ queryService: svc,
+ measureService: measureSchemaSVC,
+ broadcaster: broadcaster,
}
svc.tqp = &topNQueryProcessor{
queryService: svc,
@@ -130,10 +123,6 @@ func (q *queryService) PreRun(ctx context.Context) error {
}
q.log = logger.GetLogger(moduleName)
- q.sqp.streamService = stream.NewPortableRepository(q.metaService, q.log,
- schema.NewMetrics(q.omr.With(streamScope)))
- q.mqp.measureService = measure.NewPortableRepository(q.metaService, q.log,
- schema.NewMetrics(q.omr.With(measureScope)), q.mqp.qClient)
q.tqp.measureService = q.mqp.measureService
return multierr.Combine(
q.pipeline.Subscribe(data.TopicStreamQuery, q.sqp),
@@ -143,8 +132,6 @@ func (q *queryService) PreRun(ctx context.Context) error {
}
func (q *queryService) GracefulStop() {
- q.sqp.streamService.Close()
- q.mqp.measureService.Close()
q.closer.Done()
q.closer.CloseThenWait()
}
@@ -153,14 +140,6 @@ func (q *queryService) Serve() run.StopNotify {
return q.closer.CloseNotify()
}
-func (q *queryService) InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
- if q.mqp == nil || q.mqp.measureService == nil {
- q.log.Error().Msg("measure query processor or measure service is not initialized")
- return
- }
- q.mqp.measureService.InFlow(stm, seriesID, shardID, entityValues, dp)
-}
-
func (q *queryService) parseNodeSelector(stages []string, resource *commonv1.ResourceOpts) ([]string, bool) {
if len(stages) == 0 {
stages = resource.DefaultStages
diff --git a/banyand/dquery/measure.go b/banyand/dquery/measure.go
index b03665bf..77a66584 100644
--- a/banyand/dquery/measure.go
+++ b/banyand/dquery/measure.go
@@ -27,7 +27,6 @@ import (
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
"github.com/apache/skywalking-banyandb/banyand/measure"
- "github.com/apache/skywalking-banyandb/banyand/queue"
"github.com/apache/skywalking-banyandb/pkg/bus"
"github.com/apache/skywalking-banyandb/pkg/logger"
"github.com/apache/skywalking-banyandb/pkg/query"
@@ -37,8 +36,7 @@ import (
)
type measureQueryProcessor struct {
- measureService measure.SchemaService
- qClient queue.Client
+ measureService measure.Service
broadcaster bus.Broadcaster
*queryService
*bus.UnImplementedHealthyListener
diff --git a/banyand/dquery/stream.go b/banyand/dquery/stream.go
index 98feb553..646415a9 100644
--- a/banyand/dquery/stream.go
+++ b/banyand/dquery/stream.go
@@ -35,7 +35,7 @@ import (
)
type streamQueryProcessor struct {
- streamService stream.SchemaService
+ streamService stream.Service
broadcaster bus.Broadcaster
*queryService
*bus.UnImplementedHealthyListener
diff --git a/banyand/dquery/topn.go b/banyand/dquery/topn.go
index 1bb3ad99..80f59d34 100644
--- a/banyand/dquery/topn.go
+++ b/banyand/dquery/topn.go
@@ -41,7 +41,7 @@ import (
const defaultTopNQueryTimeout = 10 * time.Second
type topNQueryProcessor struct {
- measureService measure.SchemaService
+ measureService measure.Service
broadcaster bus.Broadcaster
*queryService
*bus.UnImplementedHealthyListener
diff --git a/banyand/liaison/grpc/discovery.go b/banyand/liaison/grpc/discovery.go
index f9542a70..859bd7b5 100644
--- a/banyand/liaison/grpc/discovery.go
+++ b/banyand/liaison/grpc/discovery.go
@@ -281,15 +281,6 @@ func (e *entityRepo) getLocator(id identity) (partition.Locator, bool) {
return el, true
}
-// loadMeasure retrieves the measure from the entityRepo by its metadata.
-func (e *entityRepo) loadMeasure(metadata *commonv1.Metadata) (*databasev1.Measure, bool) {
- id := getID(metadata)
- e.RWMutex.RLock()
- defer e.RWMutex.RUnlock()
- measure, ok := e.measureMap[id]
- return measure, ok
-}
-
var _ schema.EventHandler = (*shardingKeyRepo)(nil)
type shardingKeyRepo struct {
diff --git a/banyand/liaison/grpc/measure.go b/banyand/liaison/grpc/measure.go
index a4e84fa2..d020849a 100644
--- a/banyand/liaison/grpc/measure.go
+++ b/banyand/liaison/grpc/measure.go
@@ -25,14 +25,12 @@ import (
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
- "google.golang.org/protobuf/proto"
"github.com/apache/skywalking-banyandb/api/common"
"github.com/apache/skywalking-banyandb/api/data"
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
- "github.com/apache/skywalking-banyandb/banyand/measure"
"github.com/apache/skywalking-banyandb/banyand/queue"
"github.com/apache/skywalking-banyandb/pkg/accesslog"
"github.com/apache/skywalking-banyandb/pkg/bus"
@@ -377,103 +375,3 @@ type succeedSentMessage struct {
nodes []string
messageID uint64
}
-
-type measureRedirectWriteCallback struct {
- pipeline queue.Client
- nodeRegistry NodeRegistry
- topNService measure.TopNService
- l *logger.Logger
- groupRepo *groupRepo
- entityRepo *entityRepo
- writeTimeout time.Duration
- maxDiskUsagePercent int
-}
-
-func (r *measureRedirectWriteCallback) CheckHealth() *common.Error {
- if r.maxDiskUsagePercent < 1 {
- return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "measure is readonly because \"measure-max-disk-usage-percent\" is 0")
- }
- return nil
-}
-
-func (r *measureRedirectWriteCallback) Rev(ctx context.Context, message bus.Message) (resp bus.Message) {
- events, ok := message.Data().([]any)
- if !ok {
- r.l.Warn().Msg("invalid event data type")
- return
- }
- if len(events) < 1 {
- r.l.Warn().Msg("empty event")
- return
- }
-
- publisher := r.pipeline.NewBatchPublisher(r.writeTimeout)
- defer func() {
- _, err := publisher.Close()
- if err != nil {
- r.l.Error().Err(err).Msg("failed to close publisher")
- }
- }()
-
- for i := range events {
- var writeEvent *measurev1.InternalWriteRequest
- switch e := events[i].(type) {
- case *measurev1.InternalWriteRequest:
- writeEvent = e
- case []byte:
- writeEvent = &measurev1.InternalWriteRequest{}
- if err := proto.Unmarshal(e, writeEvent); err != nil {
- r.l.Error().Err(err).RawJSON("written", e).Msg("fail to unmarshal event")
- continue
- }
- default:
- r.l.Warn().Msg("invalid event data type")
- continue
- }
-
- metadata := writeEvent.Request.GetMetadata()
- if metadata == nil {
- r.l.Warn().Msg("metadata is nil in InternalWriteRequest")
- continue
- }
-
- group := metadata.GetGroup()
- measureName := metadata.GetName()
- shardID := writeEvent.GetShardId()
-
- copies, ok := r.groupRepo.copies(group)
- if !ok {
- r.l.Error().Str("group", group).Msg("failed to get group copies")
- continue
- }
-
- for copyIdx := range copies {
- nodeID, err := r.nodeRegistry.Locate(group, measureName, shardID, copyIdx)
- if err != nil {
- r.l.Error().Err(err).Str("group", group).Str("measure", measureName).Uint32("shard", shardID).Uint32("copy", copyIdx).Msg("failed to locate node")
- continue
- }
-
- msg := bus.NewBatchMessageWithNode(bus.MessageID(time.Now().UnixNano()), nodeID, writeEvent)
- if _, err := publisher.Publish(ctx, data.TopicMeasureWrite, msg); err != nil {
- r.l.Error().Err(err).Str("node", nodeID).Msg("failed to publish message")
- }
- }
- stm, ok := r.entityRepo.loadMeasure(metadata)
- if !ok {
- r.l.Error().RawJSON("written", logger.Proto(writeEvent)).Msg("failed to load measure schema")
- continue
- }
- series := &pbv1.Series{
- Subject: metadata.Name,
- EntityValues: writeEvent.EntityValues,
- }
- if err := series.Marshal(); err != nil {
- r.l.Error().Err(err).RawJSON("written", logger.Proto(writeEvent)).Msg("failed to marshal series")
- continue
- }
- r.topNService.InFlow(stm, uint64(series.ID), writeEvent.ShardId, writeEvent.EntityValues, writeEvent.Request.DataPoint)
- }
-
- return
-}
diff --git a/banyand/liaison/grpc/registry_test.go b/banyand/liaison/grpc/registry_test.go
index 76bf9048..287d3f35 100644
--- a/banyand/liaison/grpc/registry_test.go
+++ b/banyand/liaison/grpc/registry_test.go
@@ -182,10 +182,10 @@ func setupForRegistry() func() {
metricSvc := observability.NewMetricService(metaSvc, pipeline, "standalone", nil)
nr := grpc.NewLocalNodeRegistry()
- tcp := grpc.NewServer(context.TODO(), pipeline, pipeline, pipeline, nil, metaSvc, grpc.NodeRegistries{
+ tcp := grpc.NewServer(context.TODO(), pipeline, pipeline, pipeline, metaSvc, grpc.NodeRegistries{
MeasureLiaisonNodeRegistry: nr,
PropertyNodeRegistry: nr,
- }, metricSvc, nil, pipeline)
+ }, metricSvc)
preloadStreamSvc := &preloadStreamService{metaSvc: metaSvc}
var flags []string
metaPath, metaDeferFunc, err := test.NewSpace()
diff --git a/banyand/liaison/grpc/server.go b/banyand/liaison/grpc/server.go
index 0ee0965a..5f5e3a76 100644
--- a/banyand/liaison/grpc/server.go
+++ b/banyand/liaison/grpc/server.go
@@ -35,13 +35,11 @@ import (
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/status"
- "github.com/apache/skywalking-banyandb/api/data"
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
propertyv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/property/v1"
streamv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/stream/v1"
- "github.com/apache/skywalking-banyandb/banyand/measure"
"github.com/apache/skywalking-banyandb/banyand/metadata"
"github.com/apache/skywalking-banyandb/banyand/metadata/schema"
"github.com/apache/skywalking-banyandb/banyand/observability"
@@ -74,18 +72,13 @@ type Server interface {
type NodeRegistries struct {
StreamLiaisonNodeRegistry NodeRegistry
MeasureLiaisonNodeRegistry NodeRegistry
- MeasureDataNodeRegistry NodeRegistry
PropertyNodeRegistry NodeRegistry
}
type server struct {
databasev1.UnimplementedSnapshotServiceServer
- topNPipeline queue.Server
- omr observability.MetricsRegistry
- tire2Server queue.Server
- schemaRepo metadata.Repo
- measureCallback *measureRedirectWriteCallback
- topNHandler *topNHandler
+ omr observability.MetricsRegistry
+ schemaRepo metadata.Repo
*topNAggregationRegistryServer
*groupRegistryServer
stopCh chan struct{}
@@ -115,9 +108,8 @@ type server struct {
}
// NewServer returns a new gRPC server.
-func NewServer(_ context.Context, tir1Client, tir2Client, broadcaster queue.Client, topNPipeline queue.Server,
- schemaRegistry metadata.Repo, nr NodeRegistries, omr observability.MetricsRegistry, topNService measure.TopNService,
- tire2Server queue.Server,
+func NewServer(_ context.Context, tir1Client, tir2Client, broadcaster queue.Client,
+ schemaRegistry metadata.Repo, nr NodeRegistries, omr observability.MetricsRegistry,
) Server {
gr := &groupRepo{resourceOpts: make(map[string]*commonv1.ResourceOpts)}
er := &entityRepo{entitiesMap: make(map[identity]partition.Locator), measureMap: make(map[identity]*databasev1.Measure)}
@@ -133,18 +125,10 @@ func NewServer(_ context.Context, tir1Client, tir2Client, broadcaster queue.Clie
}
s := &server{
- omr: omr,
- streamSVC: streamSVC,
- measureSVC: measureSVC,
- groupRepo: gr,
- tire2Server: tire2Server,
- measureCallback: &measureRedirectWriteCallback{
- pipeline: tir2Client,
- groupRepo: gr,
- entityRepo: er,
- nodeRegistry: nr.MeasureDataNodeRegistry,
- topNService: topNService,
- },
+ omr: omr,
+ streamSVC: streamSVC,
+ measureSVC: measureSVC,
+ groupRepo: gr,
streamRegistryServer: &streamRegistryServer{
schemaRegistry: schemaRegistry,
},
@@ -172,8 +156,7 @@ func NewServer(_ context.Context, tir1Client, tir2Client, broadcaster queue.Clie
propertyRegistryServer: &propertyRegistryServer{
schemaRegistry: schemaRegistry,
},
- topNPipeline: topNPipeline,
- schemaRepo: schemaRegistry,
+ schemaRepo: schemaRegistry,
}
s.accessLogRecorders = []accessLogRecorder{streamSVC, measureSVC}
@@ -185,7 +168,6 @@ func (s *server) PreRun(_ context.Context) error {
s.streamSVC.setLogger(s.log.Named("stream-t1"))
s.measureSVC.setLogger(s.log)
s.propertyServer.SetLogger(s.log)
- s.measureCallback.l = s.log.Named("measure-t2")
components := []*discoveryService{
s.streamSVC.discoveryService,
s.measureSVC.discoveryService,
@@ -198,9 +180,6 @@ func (s *server) PreRun(_ context.Context) error {
return err
}
}
- if err := s.tire2Server.Subscribe(data.TopicMeasureWrite, s.measureCallback); err != nil {
- return err
- }
if s.enableIngestionAccessLog {
for _, alr := range s.accessLogRecorders {
@@ -230,19 +209,6 @@ func (s *server) PreRun(_ context.Context) error {
return err
}
}
-
- if s.topNPipeline != nil {
- topNHandler := &topNHandler{
- nodeRegistry: s.measureCallback.nodeRegistry,
- pipeline: s.measureCallback.pipeline,
- l: s.log.Named("topNHandler"),
- }
- if err := s.topNPipeline.Subscribe(data.TopicMeasureWrite, topNHandler); err != nil {
- s.log.Error().Err(err).Msg("Failed to subscribe to topN pipeline")
- return err
- }
- s.topNHandler = topNHandler
- }
return nil
}
@@ -270,13 +236,11 @@ func (s *server) FlagSet() *run.FlagSet {
fs.BoolVar(&s.enableIngestionAccessLog, "enable-ingestion-access-log", false, "enable ingestion access log")
fs.StringVar(&s.accessLogRootPath, "access-log-root-path", "", "access log root path")
fs.DurationVar(&s.streamSVC.writeTimeout, "stream-write-timeout", 15*time.Second, "timeout for writing stream among liaison nodes")
- fs.DurationVar(&s.measureCallback.writeTimeout, "measure-write-data-timeout", 15*time.Second, "timeout for writing measure data to the data nodes")
fs.DurationVar(&s.measureSVC.writeTimeout, "measure-write-timeout", 15*time.Second, "timeout for writing measure among liaison nodes")
fs.DurationVar(&s.measureSVC.maxWaitDuration, "measure-metadata-cache-wait-duration", 0,
"the maximum duration to wait for metadata cache to load (for testing purposes)")
fs.DurationVar(&s.streamSVC.maxWaitDuration, "stream-metadata-cache-wait-duration", 0,
"the maximum duration to wait for metadata cache to load (for testing purposes)")
- fs.IntVar(&s.measureCallback.maxDiskUsagePercent, "liaison-measure-max-disk-usage-percent", 95, "the maximum disk usage percentage allowed")
fs.IntVar(&s.propertyServer.repairQueueCount, "property-repair-queue-count", 128, "the number of queues for property repair")
return fs
}
diff --git a/banyand/measure/datapoints.go b/banyand/measure/datapoints.go
index 4359cbfc..ebc4da16 100644
--- a/banyand/measure/datapoints.go
+++ b/banyand/measure/datapoints.go
@@ -24,6 +24,7 @@ import (
"github.com/apache/skywalking-banyandb/api/common"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
+ "github.com/apache/skywalking-banyandb/banyand/internal/wqueue"
"github.com/apache/skywalking-banyandb/pkg/index"
pbv1 "github.com/apache/skywalking-banyandb/pkg/pb/v1"
"github.com/apache/skywalking-banyandb/pkg/pool"
@@ -220,18 +221,26 @@ func releaseDataPoints(v *dataPoints) {
var dataPointsPool = pool.Register[*dataPoints]("measure-dataPoints")
type dataPointsInTable struct {
- tsTable *tsTable
- dataPoints *dataPoints
- timeRange timestamp.TimeRange
+ segment storage.Segment[*tsTable, option]
+ tsTable *tsTable
+ dataPoints *dataPoints
+ metadataDocMap map[uint64]int
+ indexModeDocMap map[uint64]int
+ timeRange timestamp.TimeRange
+ metadataDocs index.Documents
+ indexModeDocs index.Documents
+ shardID common.ShardID
}
type dataPointsInGroup struct {
- tsdb storage.TSDB[*tsTable, option]
- metadataDocs index.Documents
- indexModeDocs index.Documents
- metadataDocMap map[uint64]int
- indexModeDocMap map[uint64]int
- tables []*dataPointsInTable
- segments []storage.Segment[*tsTable, option]
- latestTS int64
+ tsdb storage.TSDB[*tsTable, option]
+ tables []*dataPointsInTable
+ segments []storage.Segment[*tsTable, option]
+ latestTS int64
+}
+
+type dataPointsInQueue struct {
+ name string
+ queue *wqueue.Queue[*tsTable, option]
+ tables []*dataPointsInTable
}
diff --git a/banyand/measure/flusher.go b/banyand/measure/flusher.go
index 7bc40af0..4bc14826 100644
--- a/banyand/measure/flusher.go
+++ b/banyand/measure/flusher.go
@@ -185,3 +185,46 @@ func (tst *tsTable) persistSnapshot(snapshot *snapshot) {
tst.mustWriteSnapshot(snapshot.epoch, partNames)
tst.gc.registerSnapshot(snapshot)
}
+
+// flusherLoopNoMerger is like flusherLoop but does not perform merging, only flushing.
+func (tst *tsTable) flusherLoopNoMerger(flushCh chan *flusherIntroduction, introducerWatcher, flusherWatcher watcher.Channel, epoch uint64) {
+ defer tst.loopCloser.Done()
+ epochWatcher := introducerWatcher.Add(epoch, tst.loopCloser.CloseNotify())
+ if epochWatcher == nil {
+ return
+ }
+ var flusherWatchers watcher.Epochs
+
+ for {
+ select {
+ case <-tst.loopCloser.CloseNotify():
+ return
+ case e := <-flusherWatcher:
+ flusherWatchers.Add(e)
+ case <-epochWatcher.Watch():
+ if func() bool {
+ tst.incTotalFlushLoopStarted(1)
+ start := time.Now()
+ defer func() {
+ tst.incTotalFlushLoopFinished(1)
+ tst.incTotalFlushLatency(time.Since(start).Seconds())
+ }()
+ curSnapshot := tst.currentSnapshot()
+ if curSnapshot != nil {
+ defer curSnapshot.decRef()
+ tst.flush(curSnapshot, flushCh)
+ epoch = curSnapshot.epoch
+ if tst.currentEpoch() != epoch {
+ tst.incTotalFlushLoopProgress(1)
+ return false
+ }
+ }
+ epochWatcher = introducerWatcher.Add(epoch, tst.loopCloser.CloseNotify())
+ return epochWatcher == nil
+ }() {
+ return
+ }
+ }
+ flusherWatchers.Notify(epoch)
+ }
+}
diff --git a/banyand/measure/introducer.go b/banyand/measure/introducer.go
index ab3a9771..b28538ca 100644
--- a/banyand/measure/introducer.go
+++ b/banyand/measure/introducer.go
@@ -142,6 +142,38 @@ func (tst *tsTable) introducerLoop(flushCh chan *flusherIntroduction, mergeCh ch
}
}
+func (tst *tsTable) introducerLoopWithSync(flushCh chan *flusherIntroduction, syncCh chan *syncIntroduction, watcherCh watcher.Channel, epoch uint64) {
+ var introducerWatchers watcher.Epochs
+ defer tst.loopCloser.Done()
+ for {
+ select {
+ case <-tst.loopCloser.CloseNotify():
+ return
+ case next := <-tst.introductions:
+ tst.incTotalIntroduceLoopStarted("mem")
+ tst.introduceMemPart(next, epoch)
+ tst.incTotalIntroduceLoopFinished("mem")
+ epoch++
+ case next := <-flushCh:
+ tst.incTotalIntroduceLoopStarted("flush")
+ tst.introduceFlushed(next, epoch)
+ tst.incTotalIntroduceLoopFinished("flush")
+ tst.gc.clean()
+ epoch++
+ case next := <-syncCh:
+ tst.incTotalIntroduceLoopStarted("sync")
+ tst.introduceSync(next, epoch)
+ tst.incTotalIntroduceLoopFinished("sync")
+ tst.gc.clean()
+ epoch++
+ case epochWatcher := <-watcherCh:
+ introducerWatchers.Add(epochWatcher)
+ }
+ curEpoch := tst.currentEpoch()
+ introducerWatchers.Notify(curEpoch)
+ }
+}
+
func (tst *tsTable) introduceMemPart(nextIntroduction *introduction, epoch uint64) {
cur := tst.currentSnapshot()
if cur != nil {
@@ -190,6 +222,21 @@ func (tst *tsTable) introduceMerged(nextIntroduction *mergerIntroduction, epoch
}
}
+func (tst *tsTable) introduceSync(nextIntroduction *syncIntroduction, epoch uint64) {
+ cur := tst.currentSnapshot()
+ if cur == nil {
+ tst.l.Panic().Msg("current snapshot is nil")
+ return
+ }
+ defer cur.decRef()
+ nextSnp := cur.remove(epoch, nextIntroduction.synced)
+ nextSnp.creator = snapshotCreatorSyncer
+ tst.replaceSnapshot(&nextSnp, true)
+ if nextIntroduction.applied != nil {
+ close(nextIntroduction.applied)
+ }
+}
+
func (tst *tsTable) replaceSnapshot(next *snapshot, persisted bool) {
tst.Lock()
defer tst.Unlock()
diff --git a/banyand/measure/measure.go b/banyand/measure/measure.go
index 84af87d4..7f9c00fc 100644
--- a/banyand/measure/measure.go
+++ b/banyand/measure/measure.go
@@ -27,6 +27,7 @@ import (
databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
"github.com/apache/skywalking-banyandb/banyand/protector"
+ "github.com/apache/skywalking-banyandb/banyand/queue"
"github.com/apache/skywalking-banyandb/pkg/logger"
"github.com/apache/skywalking-banyandb/pkg/partition"
"github.com/apache/skywalking-banyandb/pkg/run"
@@ -45,8 +46,9 @@ const (
)
type option struct {
- mergePolicy *mergePolicy
protector protector.Memory
+ tire2Client queue.Client
+ mergePolicy *mergePolicy
seriesCacheMaxSize run.Bytes
flushTimeout time.Duration
}
diff --git a/banyand/measure/measure_suite_test.go b/banyand/measure/measure_suite_test.go
index ca55ec80..e4dd13b3 100644
--- a/banyand/measure/measure_suite_test.go
+++ b/banyand/measure/measure_suite_test.go
@@ -78,7 +78,7 @@ func setUp() (*services, func()) {
metricSvc := observability.NewMetricService(metadataService, pipeline, "test", nil)
pm := protector.NewMemory(metricSvc)
// Init Measure Service
- measureService, err := measure.NewService(metadataService, pipeline, nil, metricSvc, pm)
+ measureService, err := measure.NewStandalone(metadataService, pipeline, nil, metricSvc, pm)
gomega.Expect(err).NotTo(gomega.HaveOccurred())
preloadMeasureSvc := &preloadMeasureService{metaSvc: metadataService}
querySvc, err := query.NewService(context.TODO(), nil, measureService, metadataService, pipeline)
diff --git a/banyand/measure/metadata.go b/banyand/measure/metadata.go
index a53930af..fd4155f6 100644
--- a/banyand/measure/metadata.go
+++ b/banyand/measure/metadata.go
@@ -20,7 +20,6 @@ package measure
import (
"context"
"fmt"
- "io"
"path"
"strings"
"sync"
@@ -35,6 +34,8 @@ import (
databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
"github.com/apache/skywalking-banyandb/api/validate"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
+ "github.com/apache/skywalking-banyandb/banyand/internal/wqueue"
+ "github.com/apache/skywalking-banyandb/banyand/liaison/grpc"
"github.com/apache/skywalking-banyandb/banyand/metadata"
"github.com/apache/skywalking-banyandb/banyand/metadata/schema"
"github.com/apache/skywalking-banyandb/banyand/observability"
@@ -42,6 +43,7 @@ import (
"github.com/apache/skywalking-banyandb/banyand/queue"
"github.com/apache/skywalking-banyandb/banyand/queue/pub"
"github.com/apache/skywalking-banyandb/pkg/logger"
+ "github.com/apache/skywalking-banyandb/pkg/meter"
resourceSchema "github.com/apache/skywalking-banyandb/pkg/schema"
"github.com/apache/skywalking-banyandb/pkg/timestamp"
)
@@ -71,7 +73,6 @@ var (
// SchemaService allows querying schema information.
type SchemaService interface {
Query
- TopNService
Close()
}
type schemaRepo struct {
@@ -83,7 +84,7 @@ type schemaRepo struct {
path string
}
-func newSchemaRepo(path string, svc *service, nodeLabels map[string]string) *schemaRepo {
+func newSchemaRepo(path string, svc *standalone, nodeLabels map[string]string) *schemaRepo {
sr := &schemaRepo{
path: path,
l: svc.l,
@@ -100,21 +101,21 @@ func newSchemaRepo(path string, svc *service, nodeLabels map[string]string) *sch
return sr
}
-// NewPortableRepository creates a new portable repository.
-func NewPortableRepository(metadata metadata.Repo, l *logger.Logger, metrics *resourceSchema.Metrics, topNQueue queue.Client) SchemaService {
- r := &schemaRepo{
- l: l,
- metadata: metadata,
- pipeline: topNQueue,
- Repository: resourceSchema.NewPortableRepository(
- metadata,
- l,
- newPortableSupplier(metadata, l),
- metrics,
- ),
- }
- r.start()
- return r
+func newLiaisonSchemaRepo(path string, svc *liaison, measureDataNodeRegistry grpc.NodeRegistry, pipeline queue.Client) *schemaRepo {
+ sr := &schemaRepo{
+ path: path,
+ l: svc.l,
+ metadata: svc.metadata,
+ pipeline: pipeline,
+ }
+ sr.Repository = resourceSchema.NewRepository(
+ svc.metadata,
+ svc.l,
+ newQueueSupplier(path, svc, measureDataNodeRegistry),
+ resourceSchema.NewMetrics(svc.omr.With(metadataScope)),
+ )
+ sr.start()
+ return sr
}
func (sr *schemaRepo) start() {
@@ -311,6 +312,18 @@ func (sr *schemaRepo) loadTSDB(groupName string) (storage.TSDB[*tsTable, option]
return db.(storage.TSDB[*tsTable, option]), nil
}
+func (sr *schemaRepo) loadQueue(groupName string) (*wqueue.Queue[*tsTable, option], error) {
+ g, ok := sr.LoadGroup(groupName)
+ if !ok {
+ return nil, fmt.Errorf("group %s not found", groupName)
+ }
+ db := g.SupplyTSDB()
+ if db == nil {
+ return nil, fmt.Errorf("queue for group %s not found", groupName)
+ }
+ return db.(*wqueue.Queue[*tsTable, option]), nil
+}
+
func (sr *schemaRepo) createTopNResultMeasure(ctx context.Context, measureSchemaRegistry schema.Measure, group string) {
md := GetTopNSchemaMetadata(group)
operation := func() error {
@@ -372,16 +385,16 @@ var _ resourceSchema.ResourceSupplier = (*supplier)(nil)
type supplier struct {
metadata metadata.Repo
omr observability.MetricsRegistry
- l *logger.Logger
c storage.Cache
pm protector.Memory
+ l *logger.Logger
schemaRepo *schemaRepo
nodeLabels map[string]string
path string
option option
}
-func newSupplier(path string, svc *service, sr *schemaRepo, nodeLabels map[string]string) *supplier {
+func newSupplier(path string, svc *standalone, sr *schemaRepo, nodeLabels map[string]string) *supplier {
if svc.pm == nil {
svc.l.Panic().Msg("CRITICAL: svc.pm is nil in newSupplier")
}
@@ -478,33 +491,127 @@ func (s *supplier) OpenDB(groupSchema *commonv1.Group) (resourceSchema.DB, error
opts, s.c, group)
}
-type portableSupplier struct {
- metadata metadata.Repo
- l *logger.Logger
+type queueSupplier struct {
+ metadata metadata.Repo
+ omr observability.MetricsRegistry
+ pm protector.Memory
+ measureDataNodeRegistry grpc.NodeRegistry
+ l *logger.Logger
+ schemaRepo *schemaRepo
+ path string
+ option option
}
-func newPortableSupplier(metadata metadata.Repo, l *logger.Logger) *portableSupplier {
- return &portableSupplier{
- metadata: metadata,
- l: l,
+func newQueueSupplier(path string, svc *liaison, measureDataNodeRegistry grpc.NodeRegistry) *queueSupplier {
+ opt := svc.option
+ opt.protector = svc.pm
+
+ if opt.protector == nil {
+ svc.l.Panic().Msg("CRITICAL: opt.protector is still nil after assignment")
+ }
+ return &queueSupplier{
+ metadata: svc.metadata,
+ omr: svc.omr,
+ pm: svc.pm,
+ measureDataNodeRegistry: measureDataNodeRegistry,
+ l: svc.l,
+ path: path,
+ option: opt,
}
}
-func (s *portableSupplier) ResourceSchema(md *commonv1.Metadata) (resourceSchema.ResourceSchema, error) {
+func (s *queueSupplier) OpenResource(spec resourceSchema.Resource) (resourceSchema.IndexListener, error) {
+ measureSchema := spec.Schema().(*databasev1.Measure)
+ return openMeasure(measureSpec{
+ schema: measureSchema,
+ }, s.l, nil, s.pm, s.schemaRepo)
+}
+
+func (s *queueSupplier) ResourceSchema(md *commonv1.Metadata) (resourceSchema.ResourceSchema, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
return s.metadata.MeasureRegistry().GetMeasure(ctx, md)
}
-func (*portableSupplier) OpenDB(_ *commonv1.Group) (io.Closer, error) {
- panic("do not support open db")
+func (s *queueSupplier) OpenDB(groupSchema *commonv1.Group) (resourceSchema.DB, error) {
+ name := groupSchema.Metadata.Name
+ p := common.Position{
+ Module: "measure",
+ Database: name,
+ }
+ ro := groupSchema.ResourceOpts
+ if ro == nil {
+ return nil, fmt.Errorf("no resource opts in group %s", name)
+ }
+ shardNum := ro.ShardNum
+ group := groupSchema.Metadata.Name
+ opts := wqueue.Opts[*tsTable, option]{
+ Group: group,
+ ShardNum: shardNum,
+ SegmentInterval: storage.MustToIntervalRule(ro.SegmentInterval),
+ Location: path.Join(s.path, group),
+ Option: s.option,
+ Metrics: s.newMetrics(p),
+ SubQueueCreator: newWriteQueue,
+ GetNodes: func(shardID common.ShardID) []string {
+ copies := ro.Replicas + 1
+ nodes := make([]string, 0, copies)
+ for i := uint32(0); i < copies; i++ {
+ nodeID, err := s.measureDataNodeRegistry.Locate(group, "", uint32(shardID), i)
+ if err != nil {
+ s.l.Error().Err(err).Str("group", group).Uint32("shard", uint32(shardID)).Uint32("copy", i).Msg("failed to locate node")
+ return nil
+ }
+ nodes = append(nodes, nodeID)
+ }
+ return nodes
+ },
+ }
+ return wqueue.Open(
+ common.SetPosition(context.Background(), func(_ common.Position) common.Position {
+ return p
+ }),
+ opts, group,
+ )
}
-func (s *portableSupplier) OpenResource(spec resourceSchema.Resource) (resourceSchema.IndexListener, error) {
- measureSchema := spec.Schema().(*databasev1.Measure)
- return openMeasure(measureSpec{
- schema: measureSchema,
- }, s.l, nil, nil, nil)
+func (s *queueSupplier) newMetrics(p common.Position) storage.Metrics {
+ factory := s.omr.With(measureScope.ConstLabels(meter.ToLabelPairs(common.DBLabelNames(), p.DBLabelValues())))
+ return &metrics{
+ totalWritten: factory.NewCounter("total_written"),
+ totalBatch: factory.NewCounter("total_batch"),
+ totalBatchIntroLatency: factory.NewCounter("total_batch_intro_time"),
+ totalIntroduceLoopStarted: factory.NewCounter("total_introduce_loop_started", "phase"),
+ totalIntroduceLoopFinished: factory.NewCounter("total_introduce_loop_finished", "phase"),
+ totalFlushLoopStarted: factory.NewCounter("total_flush_loop_started"),
+ totalFlushLoopFinished: factory.NewCounter("total_flush_loop_finished"),
+ totalFlushLoopErr: factory.NewCounter("total_flush_loop_err"),
+ totalMergeLoopStarted: factory.NewCounter("total_merge_loop_started"),
+ totalMergeLoopFinished: factory.NewCounter("total_merge_loop_finished"),
+ totalMergeLoopErr: factory.NewCounter("total_merge_loop_err"),
+ totalFlushLoopProgress: factory.NewCounter("total_flush_loop_progress"),
+ totalFlushed: factory.NewCounter("total_flushed"),
+ totalFlushedMemParts: factory.NewCounter("total_flushed_mem_parts"),
+ totalFlushPauseCompleted: factory.NewCounter("total_flush_pause_completed"),
+ totalFlushPauseBreak: factory.NewCounter("total_flush_pause_break"),
+ totalFlushIntroLatency: factory.NewCounter("total_flush_intro_latency"),
+ totalFlushLatency: factory.NewCounter("total_flush_latency"),
+ totalMergedParts: factory.NewCounter("total_merged_parts", "type"),
+ totalMergeLatency: factory.NewCounter("total_merge_latency", "type"),
+ totalMerged: factory.NewCounter("total_merged", "type"),
+ tbMetrics: tbMetrics{
+ totalMemParts: factory.NewGauge("total_mem_part", common.ShardLabelNames()...),
+ totalMemElements: factory.NewGauge("total_mem_elements", common.ShardLabelNames()...),
+ totalMemBlocks: factory.NewGauge("total_mem_blocks", common.ShardLabelNames()...),
+ totalMemPartBytes: factory.NewGauge("total_mem_part_bytes", common.ShardLabelNames()...),
+ totalMemPartUncompressedBytes: factory.NewGauge("total_mem_part_uncompressed_bytes", common.ShardLabelNames()...),
+ totalFileParts: factory.NewGauge("total_file_parts", common.ShardLabelNames()...),
+ totalFileElements: factory.NewGauge("total_file_elements", common.ShardLabelNames()...),
+ totalFileBlocks: factory.NewGauge("total_file_blocks", common.ShardLabelNames()...),
+ totalFilePartBytes: factory.NewGauge("total_file_part_bytes", common.ShardLabelNames()...),
+ totalFilePartUncompressedBytes: factory.NewGauge("total_file_part_uncompressed_bytes", common.ShardLabelNames()...),
+ },
+ }
}
// GetTopNSchema returns the schema of the topN result measure.
diff --git a/banyand/measure/metrics.go b/banyand/measure/metrics.go
index 1b85934c..aab606c2 100644
--- a/banyand/measure/metrics.go
+++ b/banyand/measure/metrics.go
@@ -357,7 +357,7 @@ type tbMetrics struct {
totalFilePartUncompressedBytes meter.Gauge
}
-func (s *service) createNativeObservabilityGroup(ctx context.Context) error {
+func (s *standalone) createNativeObservabilityGroup(ctx context.Context) error {
if !s.omr.NativeEnabled() {
return nil
}
diff --git a/banyand/measure/part.go b/banyand/measure/part.go
index 05836dc4..c94f92fa 100644
--- a/banyand/measure/part.go
+++ b/banyand/measure/part.go
@@ -18,7 +18,9 @@
package measure
import (
+ "encoding/json"
"fmt"
+ "io"
"path"
"path/filepath"
"sort"
@@ -27,6 +29,8 @@ import (
"github.com/apache/skywalking-banyandb/api/common"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
"github.com/apache/skywalking-banyandb/pkg/bytes"
+ "github.com/apache/skywalking-banyandb/pkg/compress/zstd"
+ "github.com/apache/skywalking-banyandb/pkg/encoding"
"github.com/apache/skywalking-banyandb/pkg/fs"
"github.com/apache/skywalking-banyandb/pkg/logger"
"github.com/apache/skywalking-banyandb/pkg/pool"
@@ -126,17 +130,275 @@ func (mp *memPart) reset() {
mp.timestamps.Reset()
mp.fieldValues.Reset()
if mp.tagFamilies != nil {
- for _, tf := range mp.tagFamilies {
+ for k, tf := range mp.tagFamilies {
tf.Reset()
+ delete(mp.tagFamilies, k)
}
}
if mp.tagFamilyMetadata != nil {
- for _, tfh := range mp.tagFamilyMetadata {
+ for k, tfh := range mp.tagFamilyMetadata {
tfh.Reset()
+ delete(mp.tagFamilyMetadata, k)
}
}
}
+// Marshal serializes the memPart to []byte for network transmission.
+// The format is:
+// - partMetadata (JSON)
+// - meta buffer length + data
+// - primary buffer length + data
+// - timestamps buffer length + data
+// - fieldValues buffer length + data
+// - tagFamilies count + (name + buffer length + data) for each
+// - tagFamilyMetadata count + (name + buffer length + data) for each.
+func (mp *memPart) Marshal() ([]byte, error) {
+ bb := bigValuePool.Generate()
+ defer bigValuePool.Release(bb)
+
+ // Marshal partMetadata as JSON
+ metadataBytes, err := json.Marshal(mp.partMetadata)
+ if err != nil {
+ return nil, fmt.Errorf("cannot marshal partMetadata: %w", err)
+ }
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(metadataBytes)))
+ bb.Buf = append(bb.Buf, metadataBytes...)
+
+ // Marshal meta buffer
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.meta.Buf)))
+ bb.Buf = append(bb.Buf, mp.meta.Buf...)
+
+ // Marshal primary buffer
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.primary.Buf)))
+ bb.Buf = append(bb.Buf, mp.primary.Buf...)
+
+ // Marshal timestamps buffer
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.timestamps.Buf)))
+ bb.Buf = append(bb.Buf, mp.timestamps.Buf...)
+
+ // Marshal fieldValues buffer
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.fieldValues.Buf)))
+ bb.Buf = append(bb.Buf, mp.fieldValues.Buf...)
+
+ // Marshal tagFamilies
+ if mp.tagFamilies == nil {
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, 0)
+ } else {
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.tagFamilies)))
+ for name, tf := range mp.tagFamilies {
+ // Write name length and name
+ nameBytes := []byte(name)
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(nameBytes)))
+ bb.Buf = append(bb.Buf, nameBytes...)
+ // Write buffer length and data
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(tf.Buf)))
+ bb.Buf = append(bb.Buf, tf.Buf...)
+ }
+ }
+
+ // Marshal tagFamilyMetadata
+ if mp.tagFamilyMetadata == nil {
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, 0)
+ } else {
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(mp.tagFamilyMetadata)))
+ for name, tfh := range mp.tagFamilyMetadata {
+ // Write name length and name
+ nameBytes := []byte(name)
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(nameBytes)))
+ bb.Buf = append(bb.Buf, nameBytes...)
+ // Write buffer length and data
+ bb.Buf = encoding.VarUint64ToBytes(bb.Buf, uint64(len(tfh.Buf)))
+ bb.Buf = append(bb.Buf, tfh.Buf...)
+ }
+ }
+
+ result := make([]byte, len(bb.Buf))
+ copy(result, bb.Buf)
+ return result, nil
+}
+
+// Unmarshal deserializes the memPart from []byte.
+func (mp *memPart) Unmarshal(data []byte) error {
+ mp.reset()
+
+ // Unmarshal partMetadata
+ tail, metadataLen := encoding.BytesToVarUint64(data)
+ if uint64(len(tail)) < metadataLen {
+ return fmt.Errorf("insufficient data for partMetadata: need %d bytes, have %d", metadataLen, len(tail))
+ }
+ metadataBytes := tail[:metadataLen]
+ tail = tail[metadataLen:]
+ if err := json.Unmarshal(metadataBytes, &mp.partMetadata); err != nil {
+ return fmt.Errorf("cannot unmarshal partMetadata: %w", err)
+ }
+
+ // Unmarshal meta buffer
+ tail, metaLen := encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < metaLen {
+ return fmt.Errorf("insufficient data for meta buffer: need %d bytes, have %d", metaLen, len(tail))
+ }
+ mp.meta.Buf = append(mp.meta.Buf[:0], tail[:metaLen]...)
+ tail = tail[metaLen:]
+
+ // Unmarshal primary buffer
+ tail, primaryLen := encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < primaryLen {
+ return fmt.Errorf("insufficient data for primary buffer: need %d bytes, have %d", primaryLen, len(tail))
+ }
+ mp.primary.Buf = append(mp.primary.Buf[:0], tail[:primaryLen]...)
+ tail = tail[primaryLen:]
+
+ // Unmarshal timestamps buffer
+ tail, timestampsLen := encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < timestampsLen {
+ return fmt.Errorf("insufficient data for timestamps buffer: need %d bytes, have %d", timestampsLen, len(tail))
+ }
+ mp.timestamps.Buf = append(mp.timestamps.Buf[:0], tail[:timestampsLen]...)
+ tail = tail[timestampsLen:]
+
+ // Unmarshal fieldValues buffer
+ tail, fieldValuesLen := encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < fieldValuesLen {
+ return fmt.Errorf("insufficient data for fieldValues buffer: need %d bytes, have %d", fieldValuesLen, len(tail))
+ }
+ mp.fieldValues.Buf = append(mp.fieldValues.Buf[:0], tail[:fieldValuesLen]...)
+ tail = tail[fieldValuesLen:]
+
+ var nameLen, bufLen uint64
+ // Unmarshal tagFamilies
+ tail, tagFamiliesCount := encoding.BytesToVarUint64(tail)
+ if tagFamiliesCount > 0 {
+ mp.tagFamilies = make(map[string]*bytes.Buffer)
+ for i := uint64(0); i < tagFamiliesCount; i++ {
+ // Read name length and name
+ tail, nameLen = encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < nameLen {
+ return fmt.Errorf("insufficient data for tag family name: need %d bytes, have %d", nameLen, len(tail))
+ }
+ name := string(tail[:nameLen])
+ tail = tail[nameLen:]
+
+ // Read buffer length and data
+ tail, bufLen = encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < bufLen {
+ return fmt.Errorf("insufficient data for tag family buffer: need %d bytes, have %d", bufLen, len(tail))
+ }
+ tf := &bytes.Buffer{}
+ tf.Buf = append(tf.Buf[:0], tail[:bufLen]...)
+ mp.tagFamilies[name] = tf
+ tail = tail[bufLen:]
+ }
+ }
+
+ // Unmarshal tagFamilyMetadata
+ tail, tagFamilyMetadataCount := encoding.BytesToVarUint64(tail)
+ if tagFamilyMetadataCount > 0 {
+ mp.tagFamilyMetadata = make(map[string]*bytes.Buffer)
+ for i := uint64(0); i < tagFamilyMetadataCount; i++ {
+ // Read name length and name
+ tail, nameLen = encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < nameLen {
+ return fmt.Errorf("insufficient data for tag family metadata name: need %d bytes, have %d", nameLen, len(tail))
+ }
+ name := string(tail[:nameLen])
+ tail = tail[nameLen:]
+
+ // Read buffer length and data
+ tail, bufLen = encoding.BytesToVarUint64(tail)
+ if uint64(len(tail)) < bufLen {
+ return fmt.Errorf("insufficient data for tag family metadata buffer: need %d bytes, have %d", bufLen, len(tail))
+ }
+ tfh := &bytes.Buffer{}
+ tfh.Buf = append(tfh.Buf[:0], tail[:bufLen]...)
+ mp.tagFamilyMetadata[name] = tfh
+ tail = tail[bufLen:]
+ }
+ }
+
+ if len(tail) > 0 {
+ return fmt.Errorf("unexpected trailing data: %d bytes", len(tail))
+ }
+
+ return nil
+}
+
+func (mp *memPart) mustInitFromPart(p *part) {
+ mp.reset()
+
+ // Copy part metadata
+ mp.partMetadata = p.partMetadata
+
+ // Read primary data
+ sr := p.primary.SequentialRead()
+ data, err := io.ReadAll(sr)
+ if err != nil {
+ logger.Panicf("cannot read primary data from %s: %s", p.primary.Path(), err)
+ }
+ fs.MustClose(sr)
+ mp.primary.Buf = append(mp.primary.Buf[:0], data...)
+
+ // Read timestamps data
+ sr = p.timestamps.SequentialRead()
+ data, err = io.ReadAll(sr)
+ if err != nil {
+ logger.Panicf("cannot read timestamps data from %s: %s", p.timestamps.Path(), err)
+ }
+ fs.MustClose(sr)
+ mp.timestamps.Buf = append(mp.timestamps.Buf[:0], data...)
+
+ // Read fieldValues data
+ sr = p.fieldValues.SequentialRead()
+ data, err = io.ReadAll(sr)
+ if err != nil {
+ logger.Panicf("cannot read fieldValues data from %s: %s", p.fieldValues.Path(), err)
+ }
+ fs.MustClose(sr)
+ mp.fieldValues.Buf = append(mp.fieldValues.Buf[:0], data...)
+
+ // Read tag families data
+ if p.tagFamilies != nil {
+ mp.tagFamilies = make(map[string]*bytes.Buffer)
+ mp.tagFamilyMetadata = make(map[string]*bytes.Buffer)
+
+ for name, reader := range p.tagFamilies {
+ sr = reader.SequentialRead()
+ data, err = io.ReadAll(sr)
+ if err != nil {
+ logger.Panicf("cannot read tag family data from %s: %s", reader.Path(), err)
+ }
+ fs.MustClose(sr)
+
+ mp.tagFamilies[name] = &bytes.Buffer{}
+ mp.tagFamilies[name].Buf = append(mp.tagFamilies[name].Buf[:0], data...)
+ }
+
+ for name, reader := range p.tagFamilyMetadata {
+ sr = reader.SequentialRead()
+ data, err = io.ReadAll(sr)
+ if err != nil {
+ logger.Panicf("cannot read tag family metadata from %s: %s", reader.Path(), err)
+ }
+ fs.MustClose(sr)
+
+ mp.tagFamilyMetadata[name] = &bytes.Buffer{}
+ mp.tagFamilyMetadata[name].Buf = append(mp.tagFamilyMetadata[name].Buf[:0], data...)
+ }
+ }
+
+ // Encode primaryBlockMetadata to memPart.meta
+ // This is the reverse process of mustReadPrimaryBlockMetadata
+ bb := bigValuePool.Generate()
+ defer bigValuePool.Release(bb)
+
+ // Marshal all primaryBlockMetadata entries
+ for _, pbm := range p.primaryBlockMetadata {
+ bb.Buf = pbm.marshal(bb.Buf)
+ }
+
+ // Compress the marshaled data
+ mp.meta.Buf = zstd.Compress(mp.meta.Buf[:0], bb.Buf, 1)
+}
+
func (mp *memPart) mustInitFromDataPoints(dps *dataPoints) {
mp.reset()
diff --git a/banyand/measure/snapshot.go b/banyand/measure/snapshot.go
index 4151d6de..3a3a9972 100644
--- a/banyand/measure/snapshot.go
+++ b/banyand/measure/snapshot.go
@@ -55,6 +55,7 @@ const (
snapshotCreatorFlusher
snapshotCreatorMerger
snapshotCreatorMergedFlusher
+ snapshotCreatorSyncer
)
type snapshot struct {
@@ -198,7 +199,7 @@ func (tst *tsTable) createMetadata(dst string, snapshot *snapshot) {
}
}
-func (s *service) takeGroupSnapshot(dstDir string, groupName string) error {
+func (s *standalone) takeGroupSnapshot(dstDir string, groupName string) error {
group, ok := s.schemaRepo.LoadGroup(groupName)
if !ok {
return errors.Errorf("group %s not found", groupName)
@@ -216,7 +217,7 @@ func (s *service) takeGroupSnapshot(dstDir string, groupName string) error {
type snapshotListener struct {
*bus.UnImplementedHealthyListener
- s *service
+ s *standalone
snapshotSeq uint64
snapshotMux sync.Mutex
}
diff --git a/banyand/measure/service.go b/banyand/measure/svc_data.go
similarity index 55%
copy from banyand/measure/service.go
copy to banyand/measure/svc_data.go
index 9938529d..3cbd4878 100644
--- a/banyand/measure/service.go
+++ b/banyand/measure/svc_data.go
@@ -19,72 +19,63 @@ package measure
import (
"context"
+ "fmt"
"path"
"path/filepath"
"strings"
+ "sync"
"time"
"github.com/pkg/errors"
+ "go.uber.org/multierr"
"github.com/apache/skywalking-banyandb/api/common"
"github.com/apache/skywalking-banyandb/api/data"
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
- modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
"github.com/apache/skywalking-banyandb/banyand/metadata"
+ "github.com/apache/skywalking-banyandb/banyand/metadata/schema"
"github.com/apache/skywalking-banyandb/banyand/observability"
"github.com/apache/skywalking-banyandb/banyand/protector"
"github.com/apache/skywalking-banyandb/banyand/queue"
"github.com/apache/skywalking-banyandb/pkg/bus"
"github.com/apache/skywalking-banyandb/pkg/fs"
"github.com/apache/skywalking-banyandb/pkg/logger"
+ "github.com/apache/skywalking-banyandb/pkg/meter/native"
"github.com/apache/skywalking-banyandb/pkg/run"
resourceSchema "github.com/apache/skywalking-banyandb/pkg/schema"
"github.com/apache/skywalking-banyandb/pkg/timestamp"
)
-var (
- errEmptyRootPath = errors.New("root path is empty")
- // ErrMeasureNotExist denotes a measure doesn't exist in the metadata repo.
- ErrMeasureNotExist = errors.New("measure doesn't exist")
+const (
+ serviceName = "measure"
)
-// Service allows inspecting the measure data points.
-type Service interface {
- run.PreRunner
- run.Config
- run.Service
- Query
- TopNService
-}
-
-var _ Service = (*service)(nil)
+var _ Service = (*dataSVC)(nil)
-type service struct {
- writeListener bus.MessageListener
+type dataSVC struct {
lfs fs.FileSystem
+ c storage.Cache
pipeline queue.Server
- localPipeline queue.Queue
- metricPipeline queue.Server
omr observability.MetricsRegistry
metadata metadata.Repo
pm protector.Memory
+ metricPipeline queue.Server
schemaRepo *schemaRepo
l *logger.Logger
- c storage.Cache
cm *cacheMetrics
root string
- snapshotDir string
dataPath string
+ snapshotDir string
option option
cc storage.CacheConfig
maxDiskUsagePercent int
maxFileSnapshotNum int
}
-func (s *service) Measure(metadata *commonv1.Metadata) (Measure, error) {
+func (s *dataSVC) Measure(metadata *commonv1.Metadata) (Measure, error) {
sm, ok := s.schemaRepo.loadMeasure(metadata)
if !ok {
return nil, errors.WithStack(ErrMeasureNotExist)
@@ -92,15 +83,15 @@ func (s *service) Measure(metadata *commonv1.Metadata) (Measure, error) {
return sm, nil
}
-func (s *service) LoadGroup(name string) (resourceSchema.Group, bool) {
+func (s *dataSVC) LoadGroup(name string) (resourceSchema.Group, bool) {
return s.schemaRepo.LoadGroup(name)
}
-func (s *service) GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange {
+func (s *dataSVC) GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange {
return s.schemaRepo.GetRemovalSegmentsTimeRange(group)
}
-func (s *service) FlagSet() *run.FlagSet {
+func (s *dataSVC) FlagSet() *run.FlagSet {
flagS := run.NewFlagSet("storage")
flagS.StringVar(&s.root, "measure-root-path", "/tmp", "the root path of measure")
flagS.StringVar(&s.dataPath, "measure-data-path", "", "the data directory path of measure. If not set, <measure-root-path>/measure/data will be used")
@@ -118,7 +109,7 @@ func (s *service) FlagSet() *run.FlagSet {
return flagS
}
-func (s *service) Validate() error {
+func (s *dataSVC) Validate() error {
if s.root == "" {
return errEmptyRootPath
}
@@ -140,15 +131,15 @@ func (s *service) Validate() error {
return nil
}
-func (s *service) Name() string {
- return "measure"
+func (s *dataSVC) Name() string {
+ return serviceName
}
-func (s *service) Role() databasev1.Role {
+func (s *dataSVC) Role() databasev1.Role {
return databasev1.Role_ROLE_DATA
}
-func (s *service) PreRun(ctx context.Context) error {
+func (s *dataSVC) PreRun(ctx context.Context) error {
s.l = logger.GetLogger(s.Name())
s.l.Info().Msg("memory protector is initialized in PreRun")
s.lfs = fs.NewLocalFileSystemWithLoggerAndLimit(s.l, s.pm.GetLimit())
@@ -161,14 +152,13 @@ func (s *service) PreRun(ctx context.Context) error {
if !strings.HasPrefix(filepath.VolumeName(s.dataPath), filepath.VolumeName(path)) {
observability.UpdatePath(s.dataPath)
}
- s.localPipeline = queue.Local()
val := ctx.Value(common.ContextNodeKey)
if val == nil {
return errors.New("node id is empty")
}
s.c = storage.NewServiceCacheWithConfig(s.cc)
node := val.(common.Node)
- s.schemaRepo = newSchemaRepo(s.dataPath, s, node.Labels)
+ s.schemaRepo = newDataSchemaRepo(s.dataPath, s, node.Labels)
s.cm = newCacheMetrics(s.omr)
observability.MetricsCollector.Register("measure_cache", s.collectCacheMetrics)
@@ -177,55 +167,57 @@ func (s *service) PreRun(ctx context.Context) error {
return nil
}
- if err := s.createNativeObservabilityGroup(ctx); err != nil {
+ if err := s.createDataNativeObservabilityGroup(ctx); err != nil {
+ return err
+ }
+
+ if err := s.pipeline.Subscribe(data.TopicSnapshot, &dataSnapshotListener{s: s}); err != nil {
return err
}
- if err := s.pipeline.Subscribe(data.TopicSnapshot, &snapshotListener{s: s}); err != nil {
+ if err := s.pipeline.Subscribe(data.TopicMeasureDeleteExpiredSegments, &dataDeleteStreamSegmentsListener{s: s}); err != nil {
return err
}
- if err := s.pipeline.Subscribe(data.TopicMeasureDeleteExpiredSegments, &deleteStreamSegmentsListener{s: s}); err != nil {
+ err := s.pipeline.Subscribe(data.TopicMeasurePartSync, setUpSyncCallback(s.l, s.schemaRepo))
+ if err != nil {
+ return err
+ }
+ err = s.pipeline.Subscribe(data.TopicMeasureSeriesIndexInsert, setUpIndexCallback(s.l, s.schemaRepo, data.TopicMeasureSeriesIndexInsert))
+ if err != nil {
+ return err
+ }
+ err = s.pipeline.Subscribe(data.TopicMeasureSeriesIndexUpdate, setUpIndexCallback(s.l, s.schemaRepo, data.TopicMeasureSeriesIndexUpdate))
+ if err != nil {
return err
}
- s.writeListener = setUpWriteCallback(s.l, s.schemaRepo, s.maxDiskUsagePercent)
+ writeListener := setUpWriteCallback(s.l, s.schemaRepo, s.maxDiskUsagePercent)
+ err = s.pipeline.Subscribe(data.TopicMeasureWrite, writeListener)
+ if err != nil {
+ return err
+ }
// only subscribe metricPipeline for data node
if s.metricPipeline != nil {
- err := s.metricPipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
+ err = s.metricPipeline.Subscribe(data.TopicMeasureWrite, writeListener)
if err != nil {
return err
}
}
- err := s.pipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
- if err != nil {
- return err
- }
- return s.localPipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
+ return nil
}
-func (s *service) Serve() run.StopNotify {
+func (s *dataSVC) Serve() run.StopNotify {
return s.schemaRepo.StopCh()
}
-func (s *service) GracefulStop() {
+func (s *dataSVC) GracefulStop() {
observability.MetricsCollector.Unregister("measure_cache")
s.schemaRepo.Close()
s.c.Close()
- if s.localPipeline != nil {
- s.localPipeline.GracefulStop()
- }
}
-func (s *service) InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
- if s.schemaRepo == nil {
- s.l.Error().Msg("schema repository is not initialized")
- return
- }
- s.schemaRepo.InFlow(stm, seriesID, shardID, entityValues, dp)
-}
-
-func (s *service) collectCacheMetrics() {
+func (s *dataSVC) collectCacheMetrics() {
if s.cm == nil || s.c == nil {
return
}
@@ -246,9 +238,53 @@ func (s *service) collectCacheMetrics() {
s.cm.size.Set(float64(size))
}
-// NewService returns a new service.
-func NewService(metadata metadata.Repo, pipeline queue.Server, metricPipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
- return &service{
+func (s *dataSVC) createDataNativeObservabilityGroup(ctx context.Context) error {
+ if !s.omr.NativeEnabled() {
+ return nil
+ }
+ g := &commonv1.Group{
+ Metadata: &commonv1.Metadata{
+ Name: native.ObservabilityGroupName,
+ },
+ Catalog: commonv1.Catalog_CATALOG_MEASURE,
+ ResourceOpts: &commonv1.ResourceOpts{
+ ShardNum: 1,
+ SegmentInterval: &commonv1.IntervalRule{
+ Unit: commonv1.IntervalRule_UNIT_DAY,
+ Num: 1,
+ },
+ Ttl: &commonv1.IntervalRule{
+ Unit: commonv1.IntervalRule_UNIT_DAY,
+ Num: 1,
+ },
+ },
+ }
+ if err := s.metadata.GroupRegistry().CreateGroup(ctx, g); err != nil &&
+ !errors.Is(err, schema.ErrGRPCAlreadyExists) {
+ return errors.WithMessage(err, "failed to create native observability group")
+ }
+ return nil
+}
+
+func (s *dataSVC) takeGroupSnapshot(dstDir string, groupName string) error {
+ group, ok := s.schemaRepo.LoadGroup(groupName)
+ if !ok {
+ return errors.Errorf("group %s not found", groupName)
+ }
+ db := group.SupplyTSDB()
+ if db == nil {
+ return errors.Errorf("group %s has no tsdb", group.GetSchema().Metadata.Name)
+ }
+ tsdb := db.(storage.TSDB[*tsTable, option])
+ if err := tsdb.TakeFileSnapshot(dstDir); err != nil {
+ return errors.WithMessagef(err, "snapshot %s fail to take file snapshot for group %s", dstDir, group.GetSchema().Metadata.Name)
+ }
+ return nil
+}
+
+// NewDataSVC returns a new data service.
+func NewDataSVC(metadata metadata.Repo, pipeline queue.Server, metricPipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
+ return &dataSVC{
metadata: metadata,
pipeline: pipeline,
metricPipeline: metricPipeline,
@@ -257,21 +293,119 @@ func NewService(metadata metadata.Repo, pipeline queue.Server, metricPipeline qu
}, nil
}
-// NewReadonlyService returns a new readonly service.
-func NewReadonlyService(metadata metadata.Repo, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
- return &service{
+// NewReadonlyDataSVC returns a new readonly data service.
+func NewReadonlyDataSVC(metadata metadata.Repo, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
+ return &dataSVC{
metadata: metadata,
omr: omr,
pm: pm,
}, nil
}
-type deleteStreamSegmentsListener struct {
+func newDataSchemaRepo(path string, svc *dataSVC, nodeLabels map[string]string) *schemaRepo {
+ sr := &schemaRepo{
+ path: path,
+ l: svc.l,
+ metadata: svc.metadata,
+ }
+ sr.Repository = resourceSchema.NewRepository(
+ svc.metadata,
+ svc.l,
+ newDataSupplier(path, svc, sr, nodeLabels),
+ resourceSchema.NewMetrics(svc.omr.With(metadataScope)),
+ )
+ sr.start()
+ return sr
+}
+
+func newDataSupplier(path string, svc *dataSVC, sr *schemaRepo, nodeLabels map[string]string) *supplier {
+ if svc.pm == nil {
+ svc.l.Panic().Msg("CRITICAL: svc.pm is nil in newSupplier")
+ }
+ opt := svc.option
+ opt.protector = svc.pm
+
+ if opt.protector == nil {
+ svc.l.Panic().Msg("CRITICAL: opt.protector is still nil after assignment")
+ }
+ return &supplier{
+ path: path,
+ metadata: svc.metadata,
+ l: svc.l,
+ c: svc.c,
+ option: opt,
+ omr: svc.omr,
+ pm: svc.pm,
+ schemaRepo: sr,
+ nodeLabels: nodeLabels,
+ }
+}
+
+type dataSnapshotListener struct {
+ *bus.UnImplementedHealthyListener
+ s *dataSVC
+ snapshotSeq uint64
+ snapshotMux sync.Mutex
+}
+
+func (d *dataSnapshotListener) Rev(ctx context.Context, message bus.Message) bus.Message {
+ groups := message.Data().([]*databasev1.SnapshotRequest_Group)
+ var gg []resourceSchema.Group
+ if len(groups) == 0 {
+ gg = d.s.schemaRepo.LoadAllGroups()
+ } else {
+ for _, g := range groups {
+ if g.Catalog != commonv1.Catalog_CATALOG_MEASURE {
+ continue
+ }
+ group, ok := d.s.schemaRepo.LoadGroup(g.Group)
+ if !ok {
+ continue
+ }
+ gg = append(gg, group)
+ }
+ }
+ if len(gg) == 0 {
+ return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), nil)
+ }
+ d.snapshotMux.Lock()
+ defer d.snapshotMux.Unlock()
+ storage.DeleteStaleSnapshots(d.s.snapshotDir, d.s.maxFileSnapshotNum, d.s.lfs)
+ sn := d.snapshotName()
+ var err error
+ for _, g := range gg {
+ select {
+ case <-ctx.Done():
+ return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), nil)
+ default:
+ }
+ if errGroup := d.s.takeGroupSnapshot(filepath.Join(d.s.snapshotDir, sn, g.GetSchema().Metadata.Name), g.GetSchema().Metadata.Name); err != nil {
+ d.s.l.Error().Err(errGroup).Str("group", g.GetSchema().Metadata.Name).Msg("fail to take group snapshot")
+ err = multierr.Append(err, errGroup)
+ continue
+ }
+ }
+ snp := &databasev1.Snapshot{
+ Name: sn,
+ Catalog: commonv1.Catalog_CATALOG_MEASURE,
+ }
+ if err != nil {
+ snp.Error = err.Error()
+ }
+ return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), snp)
+}
+
+func (d *dataSnapshotListener) snapshotName() string {
+ d.snapshotSeq++
+ return fmt.Sprintf("%s-%08X", time.Now().UTC().Format("20060102150405"), d.snapshotSeq)
+}
+
+type dataDeleteStreamSegmentsListener struct {
*bus.UnImplementedHealthyListener
- s *service
+ s *dataSVC
}
-func (d *deleteStreamSegmentsListener) Rev(_ context.Context, message bus.Message) bus.Message {
+func (d *dataDeleteStreamSegmentsListener) Rev(_ context.Context, message bus.Message) bus.Message {
req := message.Data().(*measurev1.DeleteExpiredSegmentsRequest)
if req == nil {
return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), int64(0))
diff --git a/banyand/measure/svc_liaison.go b/banyand/measure/svc_liaison.go
new file mode 100644
index 00000000..33cb3cde
--- /dev/null
+++ b/banyand/measure/svc_liaison.go
@@ -0,0 +1,165 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package measure
+
+import (
+ "context"
+ "path"
+ "path/filepath"
+ "strings"
+
+ "github.com/pkg/errors"
+
+ "github.com/apache/skywalking-banyandb/api/common"
+ "github.com/apache/skywalking-banyandb/api/data"
+ commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
+ databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
+ measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
+ modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
+ "github.com/apache/skywalking-banyandb/banyand/internal/storage"
+ "github.com/apache/skywalking-banyandb/banyand/liaison/grpc"
+ "github.com/apache/skywalking-banyandb/banyand/metadata"
+ "github.com/apache/skywalking-banyandb/banyand/observability"
+ "github.com/apache/skywalking-banyandb/banyand/protector"
+ "github.com/apache/skywalking-banyandb/banyand/queue"
+ "github.com/apache/skywalking-banyandb/pkg/fs"
+ "github.com/apache/skywalking-banyandb/pkg/logger"
+ "github.com/apache/skywalking-banyandb/pkg/node"
+ "github.com/apache/skywalking-banyandb/pkg/run"
+ resourceSchema "github.com/apache/skywalking-banyandb/pkg/schema"
+ "github.com/apache/skywalking-banyandb/pkg/timestamp"
+)
+
+type liaison struct {
+ pm protector.Memory
+ metadata metadata.Repo
+ pipeline queue.Server
+ omr observability.MetricsRegistry
+ lfs fs.FileSystem
+ dataNodeSelector node.Selector
+ l *logger.Logger
+ schemaRepo *schemaRepo
+ dataPath string
+ root string
+ option option
+ maxDiskUsagePercent int
+}
+
+func (s *liaison) Measure(metadata *commonv1.Metadata) (Measure, error) {
+ sm, ok := s.schemaRepo.loadMeasure(metadata)
+ if !ok {
+ return nil, errors.WithStack(ErrMeasureNotExist)
+ }
+ return sm, nil
+}
+
+func (s *liaison) LoadGroup(name string) (resourceSchema.Group, bool) {
+ return s.schemaRepo.LoadGroup(name)
+}
+
+func (s *liaison) GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange {
+ return s.schemaRepo.GetRemovalSegmentsTimeRange(group)
+}
+
+func (s *liaison) FlagSet() *run.FlagSet {
+ flagS := run.NewFlagSet("storage")
+ flagS.StringVar(&s.root, "measure-root-path", "/tmp", "the root path of measure")
+ flagS.StringVar(&s.dataPath, "measure-data-path", "", "the data directory path of measure. If not set, <measure-root-path>/measure/data will be used")
+ flagS.DurationVar(&s.option.flushTimeout, "measure-flush-timeout", defaultFlushTimeout, "the memory data timeout of measure")
+ flagS.IntVar(&s.maxDiskUsagePercent, "measure-max-disk-usage-percent", 95, "the maximum disk usage percentage allowed")
+ return flagS
+}
+
+func (s *liaison) Validate() error {
+ if s.root == "" {
+ return errEmptyRootPath
+ }
+ if s.maxDiskUsagePercent < 0 {
+ return errors.New("measure-max-disk-usage-percent must be greater than or equal to 0")
+ }
+ if s.maxDiskUsagePercent > 100 {
+ return errors.New("measure-max-disk-usage-percent must be less than or equal to 100")
+ }
+ return nil
+}
+
+func (s *liaison) Name() string {
+ return serviceName
+}
+
+func (s *liaison) Role() databasev1.Role {
+ return databasev1.Role_ROLE_LIAISON
+}
+
+func (s *liaison) PreRun(ctx context.Context) error {
+ s.l = logger.GetLogger(s.Name())
+ s.l.Info().Msg("memory protector is initialized in PreRun")
+ s.lfs = fs.NewLocalFileSystemWithLoggerAndLimit(s.l, s.pm.GetLimit())
+ path := path.Join(s.root, s.Name())
+ observability.UpdatePath(path)
+ val := ctx.Value(common.ContextNodeKey)
+ if val == nil {
+ return errors.New("node id is empty")
+ }
+ if s.dataPath == "" {
+ s.dataPath = filepath.Join(path, storage.DataDir)
+ }
+ if !strings.HasPrefix(filepath.VolumeName(s.dataPath), filepath.VolumeName(path)) {
+ observability.UpdatePath(s.dataPath)
+ }
+ topNResultPipeline := queue.Local()
+ measureDataNodeRegistry := grpc.NewClusterNodeRegistry(data.TopicMeasurePartSync, s.option.tire2Client, s.dataNodeSelector)
+ s.schemaRepo = newLiaisonSchemaRepo(s.dataPath, s, measureDataNodeRegistry, topNResultPipeline)
+ writeListener := setUpWriteQueueCallback(s.l, s.schemaRepo, s.maxDiskUsagePercent, s.option.tire2Client)
+ if err := s.pipeline.Subscribe(data.TopicMeasureWrite, writeListener); err != nil {
+ return err
+ }
+ return topNResultPipeline.Subscribe(data.TopicMeasureWrite, writeListener)
+}
+
+func (s *liaison) Serve() run.StopNotify {
+ return s.schemaRepo.StopCh()
+}
+
+func (s *liaison) GracefulStop() {
+ s.schemaRepo.Close()
+}
+
+func (s *liaison) InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
+ if s.schemaRepo == nil {
+ s.l.Error().Msg("schema repository is not initialized")
+ return
+ }
+ s.schemaRepo.inFlow(stm, seriesID, shardID, entityValues, dp)
+}
+
+// NewLiaison creates a new measure liaison service with the given dependencies.
+func NewLiaison(metadata metadata.Repo, pipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory,
+ dataNodeSelector node.Selector, tire2Client queue.Client,
+) (Service, error) {
+ return &liaison{
+ metadata: metadata,
+ pipeline: pipeline,
+ omr: omr,
+ pm: pm,
+ dataNodeSelector: dataNodeSelector,
+ option: option{
+ tire2Client: tire2Client,
+ },
+ }, nil
+}
diff --git a/banyand/measure/service.go b/banyand/measure/svc_standalone.go
similarity index 71%
rename from banyand/measure/service.go
rename to banyand/measure/svc_standalone.go
index 9938529d..727f27c5 100644
--- a/banyand/measure/service.go
+++ b/banyand/measure/svc_standalone.go
@@ -30,14 +30,11 @@ import (
"github.com/apache/skywalking-banyandb/api/data"
commonv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/common/v1"
databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
- measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
- modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
"github.com/apache/skywalking-banyandb/banyand/internal/storage"
"github.com/apache/skywalking-banyandb/banyand/metadata"
"github.com/apache/skywalking-banyandb/banyand/observability"
"github.com/apache/skywalking-banyandb/banyand/protector"
"github.com/apache/skywalking-banyandb/banyand/queue"
- "github.com/apache/skywalking-banyandb/pkg/bus"
"github.com/apache/skywalking-banyandb/pkg/fs"
"github.com/apache/skywalking-banyandb/pkg/logger"
"github.com/apache/skywalking-banyandb/pkg/run"
@@ -57,24 +54,22 @@ type Service interface {
run.Config
run.Service
Query
- TopNService
}
-var _ Service = (*service)(nil)
+var _ Service = (*standalone)(nil)
-type service struct {
- writeListener bus.MessageListener
+type standalone struct {
lfs fs.FileSystem
+ c storage.Cache
pipeline queue.Server
localPipeline queue.Queue
metricPipeline queue.Server
omr observability.MetricsRegistry
metadata metadata.Repo
pm protector.Memory
- schemaRepo *schemaRepo
- l *logger.Logger
- c storage.Cache
cm *cacheMetrics
+ l *logger.Logger
+ schemaRepo *schemaRepo
root string
snapshotDir string
dataPath string
@@ -84,7 +79,7 @@ type service struct {
maxFileSnapshotNum int
}
-func (s *service) Measure(metadata *commonv1.Metadata) (Measure, error) {
+func (s *standalone) Measure(metadata *commonv1.Metadata) (Measure, error) {
sm, ok := s.schemaRepo.loadMeasure(metadata)
if !ok {
return nil, errors.WithStack(ErrMeasureNotExist)
@@ -92,15 +87,15 @@ func (s *service) Measure(metadata *commonv1.Metadata) (Measure, error) {
return sm, nil
}
-func (s *service) LoadGroup(name string) (resourceSchema.Group, bool) {
+func (s *standalone) LoadGroup(name string) (resourceSchema.Group, bool) {
return s.schemaRepo.LoadGroup(name)
}
-func (s *service) GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange {
+func (s *standalone) GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange {
return s.schemaRepo.GetRemovalSegmentsTimeRange(group)
}
-func (s *service) FlagSet() *run.FlagSet {
+func (s *standalone) FlagSet() *run.FlagSet {
flagS := run.NewFlagSet("storage")
flagS.StringVar(&s.root, "measure-root-path", "/tmp", "the root path of measure")
flagS.StringVar(&s.dataPath, "measure-data-path", "", "the data directory path of measure. If not set, <measure-root-path>/measure/data will be used")
@@ -118,7 +113,7 @@ func (s *service) FlagSet() *run.FlagSet {
return flagS
}
-func (s *service) Validate() error {
+func (s *standalone) Validate() error {
if s.root == "" {
return errEmptyRootPath
}
@@ -140,15 +135,15 @@ func (s *service) Validate() error {
return nil
}
-func (s *service) Name() string {
- return "measure"
+func (s *standalone) Name() string {
+ return serviceName
}
-func (s *service) Role() databasev1.Role {
+func (s *standalone) Role() databasev1.Role {
return databasev1.Role_ROLE_DATA
}
-func (s *service) PreRun(ctx context.Context) error {
+func (s *standalone) PreRun(ctx context.Context) error {
s.l = logger.GetLogger(s.Name())
s.l.Info().Msg("memory protector is initialized in PreRun")
s.lfs = fs.NewLocalFileSystemWithLoggerAndLimit(s.l, s.pm.GetLimit())
@@ -185,30 +180,26 @@ func (s *service) PreRun(ctx context.Context) error {
return err
}
- if err := s.pipeline.Subscribe(data.TopicMeasureDeleteExpiredSegments, &deleteStreamSegmentsListener{s: s}); err != nil {
- return err
- }
-
- s.writeListener = setUpWriteCallback(s.l, s.schemaRepo, s.maxDiskUsagePercent)
+ writeListener := setUpWriteCallback(s.l, s.schemaRepo, s.maxDiskUsagePercent)
// only subscribe metricPipeline for data node
if s.metricPipeline != nil {
- err := s.metricPipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
+ err := s.metricPipeline.Subscribe(data.TopicMeasureWrite, writeListener)
if err != nil {
return err
}
}
- err := s.pipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
+ err := s.pipeline.Subscribe(data.TopicMeasureWrite, writeListener)
if err != nil {
return err
}
- return s.localPipeline.Subscribe(data.TopicMeasureWrite, s.writeListener)
+ return s.localPipeline.Subscribe(data.TopicMeasureWrite, writeListener)
}
-func (s *service) Serve() run.StopNotify {
+func (s *standalone) Serve() run.StopNotify {
return s.schemaRepo.StopCh()
}
-func (s *service) GracefulStop() {
+func (s *standalone) GracefulStop() {
observability.MetricsCollector.Unregister("measure_cache")
s.schemaRepo.Close()
s.c.Close()
@@ -217,15 +208,7 @@ func (s *service) GracefulStop() {
}
}
-func (s *service) InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
- if s.schemaRepo == nil {
- s.l.Error().Msg("schema repository is not initialized")
- return
- }
- s.schemaRepo.InFlow(stm, seriesID, shardID, entityValues, dp)
-}
-
-func (s *service) collectCacheMetrics() {
+func (s *standalone) collectCacheMetrics() {
if s.cm == nil || s.c == nil {
return
}
@@ -246,9 +229,9 @@ func (s *service) collectCacheMetrics() {
s.cm.size.Set(float64(size))
}
-// NewService returns a new service.
-func NewService(metadata metadata.Repo, pipeline queue.Server, metricPipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
- return &service{
+// NewStandalone returns a new standalone service.
+func NewStandalone(metadata metadata.Repo, pipeline queue.Server, metricPipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
+ return &standalone{
metadata: metadata,
pipeline: pipeline,
metricPipeline: metricPipeline,
@@ -256,32 +239,3 @@ func NewService(metadata metadata.Repo, pipeline queue.Server, metricPipeline qu
pm: pm,
}, nil
}
-
-// NewReadonlyService returns a new readonly service.
-func NewReadonlyService(metadata metadata.Repo, omr observability.MetricsRegistry, pm protector.Memory) (Service, error) {
- return &service{
- metadata: metadata,
- omr: omr,
- pm: pm,
- }, nil
-}
-
-type deleteStreamSegmentsListener struct {
- *bus.UnImplementedHealthyListener
- s *service
-}
-
-func (d *deleteStreamSegmentsListener) Rev(_ context.Context, message bus.Message) bus.Message {
- req := message.Data().(*measurev1.DeleteExpiredSegmentsRequest)
- if req == nil {
- return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), int64(0))
- }
-
- db, err := d.s.schemaRepo.loadTSDB(req.Group)
- if err != nil {
- d.s.l.Error().Err(err).Str("group", req.Group).Msg("failed to load tsdb")
- return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), int64(0))
- }
- deleted := db.DeleteExpiredSegments(timestamp.NewSectionTimeRange(req.TimeRange.Begin.AsTime(), req.TimeRange.End.AsTime()))
- return bus.NewMessage(bus.MessageID(time.Now().UnixNano()), deleted)
-}
diff --git a/banyand/stream/syncer.go b/banyand/measure/syncer.go
similarity index 69%
copy from banyand/stream/syncer.go
copy to banyand/measure/syncer.go
index 0883340e..f3e3fbe6 100644
--- a/banyand/stream/syncer.go
+++ b/banyand/measure/syncer.go
@@ -15,11 +15,10 @@
// specific language governing permissions and limitations
// under the License.
-package stream
+package measure
import (
"context"
- "errors"
"fmt"
"time"
@@ -27,9 +26,40 @@ import (
"github.com/apache/skywalking-banyandb/pkg/bus"
"github.com/apache/skywalking-banyandb/pkg/convert"
"github.com/apache/skywalking-banyandb/pkg/encoding"
+ "github.com/apache/skywalking-banyandb/pkg/pool"
"github.com/apache/skywalking-banyandb/pkg/watcher"
)
+type syncIntroduction struct {
+ synced map[uint64]struct{}
+ applied chan struct{}
+}
+
+func (i *syncIntroduction) reset() {
+ for k := range i.synced {
+ delete(i.synced, k)
+ }
+ i.applied = nil
+}
+
+var syncIntroductionPool = pool.Register[*syncIntroduction]("measure-sync-introduction")
+
+func generateSyncIntroduction() *syncIntroduction {
+ v := syncIntroductionPool.Get()
+ if v == nil {
+ return &syncIntroduction{
+ synced: make(map[uint64]struct{}),
+ }
+ }
+ i := v
+ i.reset()
+ return i
+}
+
+func releaseSyncIntroduction(i *syncIntroduction) {
+ syncIntroductionPool.Put(i)
+}
+
func (tst *tsTable) syncLoop(syncCh chan *syncIntroduction, flusherNotifier watcher.Channel) {
defer tst.loopCloser.Done()
@@ -52,18 +82,22 @@ func (tst *tsTable) syncLoop(syncCh chan *syncIntroduction, flusherNotifier watc
}
defer curSnapshot.decRef()
if curSnapshot.epoch != epoch {
- tst.incTotalSyncLoopStarted(1)
- defer tst.incTotalSyncLoopFinished(1)
+ // Use existing metric methods for measure
+ tst.incTotalFlushLoopStarted(1)
+ defer tst.incTotalFlushLoopFinished(1)
var err error
if err = tst.syncSnapshot(curSnapshot, syncCh); err != nil {
- if errors.Is(err, errClosed) {
+ if tst.loopCloser.Closed() {
return true
}
- tst.l.Logger.Warn().Err(err).Msgf("cannot sync snapshot: %d", curSnapshot.epoch)
- tst.incTotalSyncLoopErr(1)
+ tst.l.Error().Err(err).Msgf("cannot sync snapshot: %d", curSnapshot.epoch)
+ tst.incTotalFlushLoopErr(1)
return false
}
epoch = curSnapshot.epoch
+ if tst.currentEpoch() != epoch {
+ return false
+ }
}
ew = flusherNotifier.Add(epoch, tst.loopCloser.CloseNotify())
return ew == nil
@@ -75,10 +109,9 @@ func (tst *tsTable) syncLoop(syncCh chan *syncIntroduction, flusherNotifier watc
}
func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntroduction) error {
- // Get all parts from the current snapshot
var partsToSync []*part
for _, pw := range curSnapshot.parts {
- if pw.p.partMetadata.TotalCount > 0 {
+ if pw.mp == nil && pw.p.partMetadata.TotalCount > 0 {
partsToSync = append(partsToSync, pw.p)
}
}
@@ -86,13 +119,8 @@ func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntrodu
if len(partsToSync) == 0 {
return nil
}
- nodes := tst.getNodes()
- if len(nodes) == 0 {
- return fmt.Errorf("no nodes to sync parts")
- }
// Sort parts from old to new (by part ID)
- // Parts with lower IDs are older
for i := 0; i < len(partsToSync); i++ {
for j := i + 1; j < len(partsToSync); j++ {
if partsToSync[i].partMetadata.ID > partsToSync[j].partMetadata.ID {
@@ -101,46 +129,51 @@ func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntrodu
}
}
- // Send parts through pipeline
+ nodes := tst.getNodes
+ if nodes == nil {
+ nodes = func() []string { return nil }
+ }
+ allNodes := nodes()
+ if len(allNodes) == 0 {
+ return fmt.Errorf("no nodes to sync parts")
+ }
+
ctx := context.Background()
- for _, node := range nodes {
+ for _, node := range allNodes {
for _, part := range partsToSync {
- // Convert part to memPart
memPart := generateMemPart()
memPart.mustInitFromPart(part)
- // Marshal memPart to []byte
partData, err := memPart.Marshal()
+ releaseMemPart(memPart)
if err != nil {
- releaseMemPart(memPart)
return fmt.Errorf("failed to marshal part %d: %w", part.partMetadata.ID, err)
}
- // Encode group, shardID and prepend to partData
combinedData := make([]byte, 0, len(partData)+len(tst.group)+4)
combinedData = encoding.EncodeBytes(combinedData, convert.StringToBytes(tst.group))
combinedData = encoding.Uint32ToBytes(combinedData, uint32(tst.shardID))
combinedData = append(combinedData, partData...)
- // Create message with combined data (group + shardID + partData)
message := bus.NewMessageWithNode(bus.MessageID(time.Now().UnixNano()), node, combinedData)
- _, err = tst.option.tire2Client.Publish(ctx, data.TopicStreamPartSync, message)
-
- releaseMemPart(memPart)
-
- if err != nil {
- return fmt.Errorf("failed to publish part %d to pipeline: %w", part.partMetadata.ID, err)
+ f, publishErr := tst.option.tire2Client.Publish(ctx, data.TopicMeasurePartSync, message)
+ if publishErr != nil {
+ return fmt.Errorf("failed to publish part %d: %w", part.partMetadata.ID, publishErr)
+ }
+ if f != nil {
+ _, publishErr = f.GetAll()
+ if publishErr != nil {
+ tst.l.Warn().Err(publishErr).Msgf("failed to get all sync results: %v", publishErr)
+ }
}
}
}
- // Construct syncIntroduction to remove synced parts from snapshot
si := generateSyncIntroduction()
defer releaseSyncIntroduction(si)
si.applied = make(chan struct{})
- // Mark all synced parts for removal
for _, part := range partsToSync {
si.synced[part.partMetadata.ID] = struct{}{}
}
diff --git a/banyand/measure/topn.go b/banyand/measure/topn.go
index 8ae4ad47..2af2bae1 100644
--- a/banyand/measure/topn.go
+++ b/banyand/measure/topn.go
@@ -66,13 +66,7 @@ var (
_ flow.Sink = (*topNStreamingProcessor)(nil)
)
-// TopNService is the interface for top N service to write measures to the top N flow.
-type TopNService interface {
- // InFlow is called when a measure is written to the top N flow.
- InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue)
-}
-
-func (sr *schemaRepo) InFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
+func (sr *schemaRepo) inFlow(stm *databasev1.Measure, seriesID uint64, shardID uint32, entityValues []*modelv1.TagValue, dp *measurev1.DataPointValue) {
if p, _ := sr.topNProcessorMap.Load(getKey(stm.GetMetadata())); p != nil {
p.(*topNProcessorManager).onMeasureWrite(seriesID, shardID, &measurev1.InternalWriteRequest{
Request: &measurev1.WriteRequest{
diff --git a/banyand/measure/tstable.go b/banyand/measure/tstable.go
index a0e5c0e0..7f7a288a 100644
--- a/banyand/measure/tstable.go
+++ b/banyand/measure/tstable.go
@@ -45,9 +45,10 @@ const (
snapshotSuffix = ".snp"
)
-func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
- l *logger.Logger, _ timestamp.TimeRange, option option, m any,
-) (*tsTable, error) {
+// initTSTable initializes a tsTable and loads parts/snapshots, but does not start any background loops.
+func initTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
+ l *logger.Logger, option option, m any,
+) (*tsTable, uint64) {
if option.protector == nil {
logger.GetLogger("measure").
Panic().
@@ -59,17 +60,15 @@ func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
option: option,
l: l,
p: p,
+ pm: option.protector,
}
- tst.pm = option.protector
if m != nil {
tst.metrics = m.(*metrics)
}
tst.gc.init(&tst)
ee := fileSystem.ReadDir(rootPath)
if len(ee) == 0 {
- t := &tst
- t.startLoop(uint64(time.Now().UnixNano()))
- return t, nil
+ return &tst, uint64(time.Now().UnixNano())
}
var loadedParts []uint64
var loadedSnapshots []uint64
@@ -88,7 +87,6 @@ func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
needToDelete = append(needToDelete, ee[i].Name())
continue
}
-
loadedParts = append(loadedParts, p)
continue
}
@@ -108,20 +106,38 @@ func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
fileSystem.MustRMAll(filepath.Join(rootPath, needToDelete[i]))
}
if len(loadedParts) == 0 || len(loadedSnapshots) == 0 {
- t := &tst
- t.startLoop(uint64(time.Now().UnixNano()))
- return t, nil
+ return &tst, uint64(time.Now().UnixNano())
}
sort.Slice(loadedSnapshots, func(i, j int) bool {
return loadedSnapshots[i] > loadedSnapshots[j]
})
epoch := loadedSnapshots[0]
- t := &tst
- t.loadSnapshot(epoch, loadedParts)
+ tst.loadSnapshot(epoch, loadedParts)
+ return &tst, epoch
+}
+
+// newTSTable creates a new tsTable and starts the background loop.
+func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
+ l *logger.Logger, _ timestamp.TimeRange, option option, m any,
+) (*tsTable, error) {
+ t, epoch := initTSTable(fileSystem, rootPath, p, l, option, m)
t.startLoop(epoch)
return t, nil
}
+// startLoopNoMerge starts the background loops without the merge loop.
+func (tst *tsTable) startLoopNoMerge(cur uint64) {
+ tst.loopCloser = run.NewCloser(1 + 3)
+ tst.introductions = make(chan *introduction)
+ flushCh := make(chan *flusherIntroduction)
+ syncCh := make(chan *syncIntroduction)
+ introducerWatcher := make(watcher.Channel, 1)
+ flusherWatcher := make(watcher.Channel, 1)
+ go tst.introducerLoopWithSync(flushCh, syncCh, introducerWatcher, cur+1)
+ go tst.flusherLoopNoMerger(flushCh, introducerWatcher, flusherWatcher, cur)
+ go tst.syncLoop(syncCh, flusherWatcher)
+}
+
type tsTable struct {
fileSystem fs.FileSystem
l *logger.Logger
@@ -133,9 +149,12 @@ type tsTable struct {
option option
pm protector.Memory
root string
+ getNodes func() []string
+ group string
gc garbageCleaner
curPartID uint64
sync.RWMutex
+ shardID common.ShardID
}
func (tst *tsTable) loadSnapshot(epoch uint64, loadedParts []uint64) {
@@ -265,6 +284,10 @@ func (tst *tsTable) mustAddDataPoints(dps *dataPoints) {
mp := generateMemPart()
mp.mustInitFromDataPoints(dps)
+ tst.mustAddMemPart(mp)
+}
+
+func (tst *tsTable) mustAddMemPart(mp *memPart) {
p := openMemPart(mp)
ind := generateIntroduction()
@@ -272,8 +295,8 @@ func (tst *tsTable) mustAddDataPoints(dps *dataPoints) {
ind.applied = make(chan struct{})
ind.memPart = newPartWrapper(mp, p)
ind.memPart.p.partMetadata.ID = atomic.AddUint64(&tst.curPartID, 1)
-
startTime := time.Now()
+ totalCount := mp.partMetadata.TotalCount
select {
case tst.introductions <- ind:
case <-tst.loopCloser.CloseNotify():
@@ -283,7 +306,7 @@ func (tst *tsTable) mustAddDataPoints(dps *dataPoints) {
case <-ind.applied:
case <-tst.loopCloser.CloseNotify():
}
- tst.incTotalWritten(len(dps.timestamps))
+ tst.incTotalWritten(int(totalCount))
tst.incTotalBatch(1)
tst.incTotalBatchIntroLatency(time.Since(startTime).Seconds())
}
diff --git a/banyand/stream/wqueue.go b/banyand/measure/wqueue.go
similarity index 92%
copy from banyand/stream/wqueue.go
copy to banyand/measure/wqueue.go
index e0852463..c176c50c 100644
--- a/banyand/stream/wqueue.go
+++ b/banyand/measure/wqueue.go
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-package stream
+package measure
import (
"fmt"
@@ -35,10 +35,7 @@ type NodeSelector interface {
func newWriteQueue(fileSystem fs.FileSystem, rootPath string, p common.Position,
l *logger.Logger, option option, m any, group string, shardID common.ShardID, getNodes func() []string,
) (*tsTable, error) {
- t, epoch, err := initTSTable(fileSystem, rootPath, p, l, option, m)
- if err != nil {
- return nil, err
- }
+ t, epoch := initTSTable(fileSystem, rootPath, p, l, option, m)
t.getNodes = getNodes
t.group = group
t.shardID = shardID
diff --git a/banyand/measure/wqueue_test.go b/banyand/measure/wqueue_test.go
new file mode 100644
index 00000000..ad8b2160
--- /dev/null
+++ b/banyand/measure/wqueue_test.go
@@ -0,0 +1,90 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package measure
+
+import (
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+ "go.uber.org/mock/gomock"
+
+ "github.com/apache/skywalking-banyandb/api/common"
+ "github.com/apache/skywalking-banyandb/banyand/protector"
+ "github.com/apache/skywalking-banyandb/banyand/queue"
+ "github.com/apache/skywalking-banyandb/pkg/bus"
+ "github.com/apache/skywalking-banyandb/pkg/fs"
+ "github.com/apache/skywalking-banyandb/pkg/logger"
+ "github.com/apache/skywalking-banyandb/pkg/test"
+ "github.com/apache/skywalking-banyandb/pkg/test/flags"
+)
+
+func Test_newWriteQueue(t *testing.T) {
+ tmpPath, defFn := test.Space(require.New(t))
+ defer defFn()
+
+ // Create mock tire2Client
+ ctrl := gomock.NewController(t)
+ defer ctrl.Finish()
+ mockTire2Client := queue.NewMockClient(ctrl)
+ mockTire2Client.EXPECT().
+ Publish(gomock.Any(), gomock.Any(), gomock.Any()).
+ Return(bus.Future(nil), nil).
+ AnyTimes()
+
+ // Create tsTable using newWriteQueue
+ opts := option{
+ tire2Client: mockTire2Client,
+ protector: protector.Nop{},
+ }
+
+ tst, err := newWriteQueue(
+ fs.NewLocalFileSystem(),
+ tmpPath,
+ common.Position{},
+ logger.GetLogger("test"),
+ opts,
+ nil,
+ "test-group",
+ common.ShardID(1),
+ func() []string { return []string{"node1", "node2"} },
+ )
+ require.NoError(t, err)
+ require.NotNil(t, tst)
+ defer tst.Close()
+
+ // Wait for the creator to be set to snapshotCreatorMemPart
+ assert.Eventually(t, func() bool {
+ tst.mustAddDataPoints(dpsTS1)
+ time.Sleep(100 * time.Millisecond)
+ s := tst.currentSnapshot()
+ if s == nil {
+ return false
+ }
+ defer s.decRef()
+ return s.creator == snapshotCreatorSyncer
+ }, flags.EventuallyTimeout, 500*time.Millisecond, "wait for snapshot creator to be set to mem part")
+
+ // Verify that the tsTable has the expected group and shardID
+ assert.Equal(t, "test-group", tst.group)
+ assert.Equal(t, common.ShardID(1), tst.shardID)
+
+ // Verify that the tire2Client was properly set
+ assert.Equal(t, mockTire2Client, tst.option.tire2Client)
+}
diff --git a/banyand/stream/write_standalone.go b/banyand/measure/write_data.go
similarity index 51%
copy from banyand/stream/write_standalone.go
copy to banyand/measure/write_data.go
index 6d927b7e..641fc25f 100644
--- a/banyand/stream/write_standalone.go
+++ b/banyand/measure/write_data.go
@@ -15,13 +15,14 @@
// specific language governing permissions and limitations
// under the License.
-package stream
+package measure
import (
"context"
"time"
"github.com/apache/skywalking-banyandb/api/common"
+ "github.com/apache/skywalking-banyandb/api/data"
"github.com/apache/skywalking-banyandb/pkg/bus"
"github.com/apache/skywalking-banyandb/pkg/encoding"
"github.com/apache/skywalking-banyandb/pkg/index"
@@ -102,49 +103,53 @@ func (s *syncCallback) Rev(_ context.Context, message bus.Message) (resp bus.Mes
tsTable.mustAddMemPart(memPart)
tsdb.Tick(memPart.partMetadata.MaxTimestamp)
- s.l.Info().
- Str("group", group).
- Uint32("shardID", shardID).
- Uint64("partID", memPart.partMetadata.ID).
- Msg("successfully stored memPart to tsTable")
+ if e := s.l.Debug(); e.Enabled() {
+ e.
+ Str("group", group).
+ Uint32("shardID", shardID).
+ Uint64("partID", memPart.partMetadata.ID).
+ Msg("successfully stored memPart to tsTable")
+ }
return
}
-type seriesIndexCallback struct {
+type indexCallback struct {
l *logger.Logger
schemaRepo *schemaRepo
+ topic bus.Topic
}
-func setUpSeriesIndexCallback(l *logger.Logger, schemaRepo *schemaRepo) bus.MessageListener {
- return &seriesIndexCallback{
+func setUpIndexCallback(l *logger.Logger, schemaRepo *schemaRepo, topic bus.Topic) bus.MessageListener {
+ return &indexCallback{
l: l,
schemaRepo: schemaRepo,
+ topic: topic,
}
}
-func (s *seriesIndexCallback) CheckHealth() *common.Error {
+func (i *indexCallback) CheckHealth() *common.Error {
return nil
}
-func (s *seriesIndexCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- data, ok := message.Data().([]byte)
+func (i *indexCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
+ msgData, ok := message.Data().([]byte)
if !ok {
- s.l.Warn().Msg("invalid series index message data type")
+ i.l.Warn().Msg("invalid index insert message data type")
return
}
// Decode group name (first part of the message)
- tail, groupBytes, err := encoding.DecodeBytes(data)
+ tail, groupBytes, err := encoding.DecodeBytes(msgData)
if err != nil {
- s.l.Error().Err(err).Msg("failed to decode group name from series index message")
+ i.l.Error().Err(err).Msg("failed to decode group name from index insert message")
return
}
group := string(groupBytes)
// Decode timestamp (uint64) - second part of the message
if len(tail) < 8 {
- s.l.Error().Msg("insufficient data for timestamp in series index message")
+ i.l.Error().Msg("insufficient data for timestamp in index insert message")
return
}
timestamp := encoding.BytesToInt64(tail[:8])
@@ -153,19 +158,19 @@ func (s *seriesIndexCallback) Rev(_ context.Context, message bus.Message) (resp
// Unmarshal Documents from the remaining data
var documents index.Documents
if err = documents.Unmarshal(tail); err != nil {
- s.l.Error().Err(err).Msg("failed to unmarshal documents from series index message")
+ i.l.Error().Err(err).Msg("failed to unmarshal documents from index insert message")
return
}
if len(documents) == 0 {
- s.l.Warn().Msg("empty documents in series index message")
+ i.l.Warn().Msg("empty documents in index insert message")
return
}
// Get TSDB by group name using schemaRepo
- tsdb, err := s.schemaRepo.loadTSDB(group)
+ tsdb, err := i.schemaRepo.loadTSDB(group)
if err != nil {
- s.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
+ i.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
return
}
@@ -173,111 +178,29 @@ func (s *seriesIndexCallback) Rev(_ context.Context, message bus.Message) (resp
segmentTime := time.Unix(0, timestamp)
segment, err := tsdb.CreateSegmentIfNotExist(segmentTime)
if err != nil {
- s.l.Error().Err(err).Str("group", group).Time("segmentTime", segmentTime).Msg("failed to create segment")
+ i.l.Error().Err(err).Str("group", group).Time("segmentTime", segmentTime).Msg("failed to create segment")
return
}
defer segment.DecRef()
- // Insert all documents into the segment's index
- if err := segment.IndexDB().Insert(documents); err != nil {
- s.l.Error().Err(err).Str("group", group).Int("documentCount", len(documents)).Msg("failed to insert documents to index")
- return
- }
-
- s.l.Info().
- Str("group", group).
- Int("documentCount", len(documents)).
- Time("segmentTime", segmentTime).
- Msg("successfully inserted documents to series index")
-
- return
-}
-
-type localIndexCallback struct {
- l *logger.Logger
- schemaRepo *schemaRepo
-}
-
-func setUpLocalIndexCallback(l *logger.Logger, schemaRepo *schemaRepo) bus.MessageListener {
- return &localIndexCallback{
- l: l,
- schemaRepo: schemaRepo,
- }
-}
-
-func (l *localIndexCallback) CheckHealth() *common.Error {
- return nil
-}
-
-func (l *localIndexCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- data, ok := message.Data().([]byte)
- if !ok {
- l.l.Warn().Msg("invalid local index message data type")
- return
- }
-
- // Decode group name (first part of the message)
- tail, groupBytes, err := encoding.DecodeBytes(data)
- if err != nil {
- l.l.Error().Err(err).Msg("failed to decode group name from local index message")
- return
- }
- group := string(groupBytes)
-
- // Decode shardID (uint32)
- if len(tail) < 4 {
- l.l.Error().Msg("insufficient data for shardID in local index message")
- return
+ switch i.topic {
+ case data.TopicMeasureSeriesIndexInsert:
+ err = segment.IndexDB().Insert(documents)
+ case data.TopicMeasureSeriesIndexUpdate:
+ err = segment.IndexDB().Update(documents)
}
- shardID := encoding.BytesToUint32(tail[:4])
- tail = tail[4:]
-
- // Unmarshal Documents from the remaining data
- var documents index.Documents
- if err = documents.Unmarshal(tail); err != nil {
- l.l.Error().Err(err).Msg("failed to unmarshal documents from local index message")
- return
- }
-
- if len(documents) == 0 {
- l.l.Warn().Msg("empty documents in local index message")
- return
- }
-
- // Get TSDB by group name using schemaRepo
- tsdb, err := l.schemaRepo.loadTSDB(group)
- if err != nil {
- l.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
- return
- }
-
- // Use the first document's timestamp to get the segment
- firstDocTime := time.Unix(0, documents[0].Timestamp)
- segment, err := tsdb.CreateSegmentIfNotExist(firstDocTime)
if err != nil {
- l.l.Error().Err(err).Str("group", group).Time("docTime", firstDocTime).Msg("failed to create segment for documents")
- return
- }
- defer segment.DecRef()
-
- // Get the tsTable using shardID
- tsTable, err := segment.CreateTSTableIfNotExist(common.ShardID(shardID))
- if err != nil {
- l.l.Error().Err(err).Str("group", group).Uint32("shardID", shardID).Msg("failed to create ts table")
+ i.l.Error().Err(err).Str("group", group).Int("documentCount", len(documents)).Msg("failed to insert documents to index")
return
}
- // Insert documents into tsTable.index
- if err := tsTable.Index().Write(documents); err != nil {
- l.l.Error().Err(err).Str("group", group).Uint32("shardID", shardID).Int("documentCount", len(documents)).Msg("failed to insert documents to tsTable index")
- return
+ if e := i.l.Debug(); e.Enabled() {
+ e.
+ Str("group", group).
+ Int("documentCount", len(documents)).
+ Time("segmentTime", segmentTime).
+ Msg("successfully inserted documents to index")
}
- l.l.Info().
- Str("group", group).
- Uint32("shardID", shardID).
- Int("documentCount", len(documents)).
- Msg("successfully inserted documents to tsTable index")
-
return
}
diff --git a/banyand/measure/write_liaison.go b/banyand/measure/write_liaison.go
new file mode 100644
index 00000000..36fcd7aa
--- /dev/null
+++ b/banyand/measure/write_liaison.go
@@ -0,0 +1,216 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package measure
+
+import (
+ "context"
+ "fmt"
+ "time"
+
+ "google.golang.org/protobuf/proto"
+
+ "github.com/apache/skywalking-banyandb/api/common"
+ "github.com/apache/skywalking-banyandb/api/data"
+ measurev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/measure/v1"
+ modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
+ "github.com/apache/skywalking-banyandb/banyand/internal/wqueue"
+ "github.com/apache/skywalking-banyandb/banyand/observability"
+ "github.com/apache/skywalking-banyandb/banyand/queue"
+ "github.com/apache/skywalking-banyandb/pkg/bus"
+ "github.com/apache/skywalking-banyandb/pkg/convert"
+ "github.com/apache/skywalking-banyandb/pkg/encoding"
+ "github.com/apache/skywalking-banyandb/pkg/index"
+ "github.com/apache/skywalking-banyandb/pkg/logger"
+ "github.com/apache/skywalking-banyandb/pkg/timestamp"
+)
+
+func setUpWriteQueueCallback(l *logger.Logger, schemaRepo *schemaRepo, maxDiskUsagePercent int, tire2Client queue.Client) bus.MessageListener {
+ if maxDiskUsagePercent > 100 {
+ maxDiskUsagePercent = 100
+ }
+ return &writeQueueCallback{
+ l: l,
+ schemaRepo: schemaRepo,
+ maxDiskUsagePercent: maxDiskUsagePercent,
+ tire2Client: tire2Client,
+ }
+}
+
+type writeQueueCallback struct {
+ tire2Client queue.Client
+ l *logger.Logger
+ schemaRepo *schemaRepo
+ maxDiskUsagePercent int
+}
+
+func (w *writeQueueCallback) CheckHealth() *common.Error {
+ if w.maxDiskUsagePercent < 1 {
+ return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "measure is readonly because \"measure-max-disk-usage-percent\" is 0")
+ }
+ diskPercent := observability.GetPathUsedPercent(w.schemaRepo.path)
+ if diskPercent < w.maxDiskUsagePercent {
+ return nil
+ }
+ w.l.Warn().Int("maxPercent", w.maxDiskUsagePercent).Int("diskPercent", diskPercent).Msg("disk usage is too high, stop writing")
+ return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "disk usage is too high, stop writing")
+}
+
+func (w *writeQueueCallback) Rev(ctx context.Context, message bus.Message) (resp bus.Message) {
+ events, ok := message.Data().([]any)
+ if !ok {
+ w.l.Warn().Msg("invalid event data type")
+ return
+ }
+ if len(events) < 1 {
+ w.l.Warn().Msg("empty event")
+ return
+ }
+ groups := make(map[string]*dataPointsInQueue)
+ for i := range events {
+ var writeEvent *measurev1.InternalWriteRequest
+ switch e := events[i].(type) {
+ case *measurev1.InternalWriteRequest:
+ writeEvent = e
+ case []byte:
+ writeEvent = &measurev1.InternalWriteRequest{}
+ if err := proto.Unmarshal(e, writeEvent); err != nil {
+ w.l.Error().Err(err).RawJSON("written", e).Msg("fail to unmarshal event")
+ continue
+ }
+ default:
+ w.l.Warn().Msg("invalid event data type")
+ continue
+ }
+ var err error
+ if groups, err = w.handle(groups, writeEvent); err != nil {
+ w.l.Error().Err(err).Msg("cannot handle write event")
+ groups = make(map[string]*dataPointsInQueue)
+ continue
+ }
+ }
+ for i := range groups {
+ g := groups[i]
+ for j := range g.tables {
+ es := g.tables[j]
+ if es.tsTable != nil && es.dataPoints != nil {
+ es.tsTable.mustAddDataPoints(es.dataPoints)
+ releaseDataPoints(es.dataPoints)
+ }
+ nodes := g.queue.GetNodes(es.shardID)
+ if len(nodes) == 0 {
+ w.l.Warn().Uint32("shardID", uint32(es.shardID)).Msg("no nodes found for shard")
+ continue
+ }
+ sendDocuments := func(topic bus.Topic, docs index.Documents) {
+ seriesDocData, marshalErr := docs.Marshal()
+ if marshalErr != nil {
+ w.l.Error().Err(marshalErr).Uint32("shardID", uint32(es.shardID)).Msg("failed to marshal series documents")
+ return
+ }
+ // Encode group name, start timestamp from timeRange, and prepend to docData
+ combinedData := make([]byte, 0, len(seriesDocData)+len(g.name)+8)
+ combinedData = encoding.EncodeBytes(combinedData, convert.StringToBytes(g.name))
+ combinedData = encoding.Int64ToBytes(combinedData, es.timeRange.Start.UnixNano())
+ combinedData = append(combinedData, seriesDocData...)
+
+ // Send to all nodes for this shard
+ for _, node := range nodes {
+ message := bus.NewMessageWithNode(bus.MessageID(time.Now().UnixNano()), node, combinedData)
+ _, publishErr := w.tire2Client.Publish(ctx, topic, message)
+ if publishErr != nil {
+ w.l.Error().Err(publishErr).Str("node", node).Uint32("shardID", uint32(es.shardID)).Msg("failed to publish series index to node")
+ }
+ }
+ }
+ if len(es.metadataDocs) > 0 {
+ sendDocuments(data.TopicMeasureSeriesIndexInsert, es.metadataDocs)
+ }
+ if len(es.indexModeDocs) > 0 {
+ sendDocuments(data.TopicMeasureSeriesIndexUpdate, es.indexModeDocs)
+ }
+ }
+ }
+ return
+}
+
+func (w *writeQueueCallback) handle(dst map[string]*dataPointsInQueue, writeEvent *measurev1.InternalWriteRequest) (map[string]*dataPointsInQueue, error) {
+ req := writeEvent.Request
+ t := req.DataPoint.Timestamp.AsTime().Local()
+ if err := timestamp.Check(t); err != nil {
+ return nil, fmt.Errorf("invalid timestamp: %w", err)
+ }
+ ts := t.UnixNano()
+
+ gn := req.Metadata.Group
+ queue, err := w.schemaRepo.loadQueue(gn)
+ if err != nil {
+ return nil, fmt.Errorf("cannot load tsdb for group %s: %w", gn, err)
+ }
+ dpg, ok := dst[gn]
+ if !ok {
+ dpg = &dataPointsInQueue{
+ name: gn,
+ queue: queue,
+ tables: make([]*dataPointsInTable, 0),
+ }
+ dst[gn] = dpg
+ }
+
+ var dpt *dataPointsInTable
+ shardID := common.ShardID(writeEvent.ShardId)
+ for i := range dpg.tables {
+ if dpg.tables[i].shardID == shardID && dpg.tables[i].timeRange.Contains(ts) {
+ dpt = dpg.tables[i]
+ break
+ }
+ }
+ stm, ok := w.schemaRepo.loadMeasure(req.GetMetadata())
+ if !ok {
+ return nil, fmt.Errorf("cannot find measure definition: %s", req.GetMetadata())
+ }
+ fLen := len(req.DataPoint.GetTagFamilies())
+ if fLen < 1 {
+ return nil, fmt.Errorf("%s has no tag family", req.Metadata)
+ }
+ if fLen > len(stm.schema.GetTagFamilies()) {
+ return nil, fmt.Errorf("%s has more tag families than %s", req.Metadata, stm.schema)
+ }
+ is := stm.indexSchema.Load().(indexSchema)
+ if len(is.indexRuleLocators.TagFamilyTRule) != len(stm.GetSchema().GetTagFamilies()) {
+ logger.Panicf("metadata crashed, tag family rule length %d, tag family length %d",
+ len(is.indexRuleLocators.TagFamilyTRule), len(stm.GetSchema().GetTagFamilies()))
+ }
+
+ if dpt == nil {
+ var shard *wqueue.Shard[*tsTable]
+ shard, err = queue.GetOrCreateShard(shardID)
+ if err != nil {
+ return nil, fmt.Errorf("cannot get or create shard: %w", err)
+ }
+ dpt = newDpt(nil, queue.GetTimeRange(t), stm.schema.IndexMode, shard.SubQueue())
+ dpt.shardID = shardID
+ dpg.tables = append(dpg.tables, dpt)
+ }
+
+ sid, err := processDataPoint(dpt, req, writeEvent, stm, is, ts)
+ if err != nil {
+ return nil, err
+ }
+ w.schemaRepo.inFlow(stm.GetSchema(), sid, writeEvent.ShardId, writeEvent.EntityValues, req.DataPoint)
+ return dst, nil
+}
diff --git a/banyand/measure/write.go b/banyand/measure/write_standalone.go
similarity index 85%
rename from banyand/measure/write.go
rename to banyand/measure/write_standalone.go
index e35d39d0..7fbd5685 100644
--- a/banyand/measure/write.go
+++ b/banyand/measure/write_standalone.go
@@ -21,7 +21,6 @@ import (
"bytes"
"context"
"fmt"
- "time"
"google.golang.org/protobuf/proto"
@@ -71,6 +70,54 @@ func (w *writeCallback) CheckHealth() *common.Error {
return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "disk usage is too high, stop writing")
}
+func processDataPoint(dpt *dataPointsInTable, req *measurev1.WriteRequest, writeEvent *measurev1.InternalWriteRequest,
+ stm *measure, is indexSchema, ts int64,
+) (uint64, error) {
+ series := &pbv1.Series{
+ Subject: req.Metadata.Name,
+ EntityValues: writeEvent.EntityValues,
+ }
+ if err := series.Marshal(); err != nil {
+ return 0, fmt.Errorf("cannot marshal series: %w", err)
+ }
+
+ if stm.schema.IndexMode {
+ fields := handleIndexMode(stm.schema, req, is.indexRuleLocators)
+ fields = appendEntityTagsToIndexFields(fields, stm, series)
+ doc := index.Document{
+ DocID: uint64(series.ID),
+ EntityValues: series.Buffer,
+ Fields: fields,
+ Version: req.DataPoint.Version,
+ Timestamp: ts,
+ }
+
+ if pos, exists := dpt.indexModeDocMap[doc.DocID]; exists {
+ dpt.indexModeDocs[pos] = doc
+ } else {
+ dpt.indexModeDocMap[doc.DocID] = len(dpt.indexModeDocs)
+ dpt.indexModeDocs = append(dpt.indexModeDocs, doc)
+ }
+ return uint64(series.ID), nil
+ }
+
+ fields := appendDataPoints(dpt, ts, series.ID, stm.GetSchema(), req, is.indexRuleLocators)
+
+ doc := index.Document{
+ DocID: uint64(series.ID),
+ EntityValues: series.Buffer,
+ Fields: fields,
+ }
+
+ if pos, exists := dpt.metadataDocMap[doc.DocID]; exists {
+ dpt.metadataDocs[pos] = doc
+ } else {
+ dpt.metadataDocMap[doc.DocID] = len(dpt.metadataDocs)
+ dpt.metadataDocs = append(dpt.metadataDocs, doc)
+ }
+ return uint64(series.ID), nil
+}
+
func (w *writeCallback) handle(dst map[string]*dataPointsInGroup, writeEvent *measurev1.InternalWriteRequest) (map[string]*dataPointsInGroup, error) {
req := writeEvent.Request
t := req.DataPoint.Timestamp.AsTime().Local()
@@ -87,11 +134,9 @@ func (w *writeCallback) handle(dst map[string]*dataPointsInGroup, writeEvent *me
dpg, ok := dst[gn]
if !ok {
dpg = &dataPointsInGroup{
- tsdb: tsdb,
- tables: make([]*dataPointsInTable, 0),
- segments: make([]storage.Segment[*tsTable, option], 0),
- metadataDocMap: make(map[uint64]int),
- indexModeDocMap: make(map[uint64]int),
+ tsdb: tsdb,
+ tables: make([]*dataPointsInTable, 0),
+ segments: make([]storage.Segment[*tsTable, option], 0),
}
dst[gn] = dpg
}
@@ -125,53 +170,33 @@ func (w *writeCallback) handle(dst map[string]*dataPointsInGroup, writeEvent *me
shardID := common.ShardID(writeEvent.ShardId)
if dpt == nil {
- if dpt, err = w.newDpt(tsdb, dpg, t, ts, shardID, stm.schema.IndexMode); err != nil {
- return nil, fmt.Errorf("cannot create data points in table: %w", err)
+ var segment storage.Segment[*tsTable, option]
+ for _, seg := range dpg.segments {
+ if seg.GetTimeRange().Contains(ts) {
+ segment = seg
+ }
}
- }
-
- series := &pbv1.Series{
- Subject: req.Metadata.Name,
- EntityValues: writeEvent.EntityValues,
- }
- if err := series.Marshal(); err != nil {
- return nil, fmt.Errorf("cannot marshal series: %w", err)
- }
-
- if stm.schema.IndexMode {
- fields := handleIndexMode(stm.schema, req, is.indexRuleLocators)
- fields = w.appendEntityTagsToIndexFields(fields, stm, series)
- doc := index.Document{
- DocID: uint64(series.ID),
- EntityValues: series.Buffer,
- Fields: fields,
- Version: req.DataPoint.Version,
- Timestamp: ts,
+ if segment == nil {
+ segment, err = tsdb.CreateSegmentIfNotExist(t)
+ if err != nil {
+ return nil, fmt.Errorf("cannot create segment: %w", err)
+ }
+ dpg.segments = append(dpg.segments, segment)
}
-
- if pos, exists := dpg.indexModeDocMap[doc.DocID]; exists {
- dpg.indexModeDocs[pos] = doc
- } else {
- dpg.indexModeDocMap[doc.DocID] = len(dpg.indexModeDocs)
- dpg.indexModeDocs = append(dpg.indexModeDocs, doc)
+ var tstb *tsTable
+ tstb, err = segment.CreateTSTableIfNotExist(shardID)
+ if err != nil {
+ return nil, fmt.Errorf("cannot create ts table: %w", err)
}
- return dst, nil
+ dpt = newDpt(segment, segment.GetTimeRange(), stm.schema.IndexMode, tstb)
+ dpg.tables = append(dpg.tables, dpt)
}
- fields := appendDataPoints(dpt, ts, series.ID, stm.GetSchema(), req, is.indexRuleLocators)
-
- doc := index.Document{
- DocID: uint64(series.ID),
- EntityValues: series.Buffer,
- Fields: fields,
- }
-
- if pos, exists := dpg.metadataDocMap[doc.DocID]; exists {
- dpg.metadataDocs[pos] = doc
- } else {
- dpg.metadataDocMap[doc.DocID] = len(dpg.metadataDocs)
- dpg.metadataDocs = append(dpg.metadataDocs, doc)
+ sid, err := processDataPoint(dpt, req, writeEvent, stm, is, ts)
+ if err != nil {
+ return nil, err
}
+ w.schemaRepo.inFlow(stm.GetSchema(), sid, writeEvent.ShardId, writeEvent.EntityValues, req.DataPoint)
return dst, nil
}
@@ -209,39 +234,20 @@ func appendDataPoints(dest *dataPointsInTable, ts int64, sid common.SeriesID, sc
return fields
}
-func (w *writeCallback) newDpt(tsdb storage.TSDB[*tsTable, option], dpg *dataPointsInGroup,
- t time.Time, ts int64, shardID common.ShardID, indexMode bool,
-) (*dataPointsInTable, error) {
- var segment storage.Segment[*tsTable, option]
- for _, seg := range dpg.segments {
- if seg.GetTimeRange().Contains(ts) {
- segment = seg
- }
- }
- if segment == nil {
- var err error
- segment, err = tsdb.CreateSegmentIfNotExist(t)
- if err != nil {
- return nil, fmt.Errorf("cannot create segment: %w", err)
- }
- dpg.segments = append(dpg.segments, segment)
+func newDpt(segment storage.Segment[*tsTable, option], timeRange timestamp.TimeRange,
+ indexMode bool, tstb *tsTable,
+) *dataPointsInTable {
+ dpt := &dataPointsInTable{
+ timeRange: timeRange,
+ tsTable: tstb,
+ metadataDocMap: make(map[uint64]int),
+ segment: segment,
}
if indexMode {
- return &dataPointsInTable{
- timeRange: segment.GetTimeRange(),
- }, nil
+ dpt.indexModeDocMap = make(map[uint64]int)
}
- tstb, err := segment.CreateTSTableIfNotExist(shardID)
- if err != nil {
- return nil, fmt.Errorf("cannot create ts table: %w", err)
- }
- dpt := &dataPointsInTable{
- timeRange: segment.GetTimeRange(),
- tsTable: tstb,
- }
- dpg.tables = append(dpg.tables, dpt)
- return dpt, nil
+ return dpt
}
func handleTagFamily(schema *databasev1.Measure, req *measurev1.WriteRequest, locator partition.IndexRuleLocator) ([]nameValues, []index.Field) {
@@ -363,7 +369,7 @@ func handleIndexMode(schema *databasev1.Measure, req *measurev1.WriteRequest, lo
return fields
}
-func (w *writeCallback) appendEntityTagsToIndexFields(fields []index.Field, stm *measure, series *pbv1.Series) []index.Field {
+func appendEntityTagsToIndexFields(fields []index.Field, stm *measure, series *pbv1.Series) []index.Field {
f := index.NewStringField(subjectField, series.Subject)
f.Index = true
f.NoSort = true
@@ -435,24 +441,24 @@ func (w *writeCallback) Rev(_ context.Context, message bus.Message) (resp bus.Me
g := groups[i]
for j := range g.tables {
dps := g.tables[j]
- if dps.tsTable != nil {
+ if dps.tsTable != nil && dps.dataPoints != nil {
dps.tsTable.mustAddDataPoints(dps.dataPoints)
}
if dps.dataPoints != nil {
releaseDataPoints(dps.dataPoints)
}
- }
- for _, segment := range g.segments {
- if len(g.metadataDocs) > 0 {
- if err := segment.IndexDB().Insert(g.metadataDocs); err != nil {
+ if len(dps.metadataDocs) > 0 {
+ if err := dps.segment.IndexDB().Insert(dps.metadataDocs); err != nil {
w.l.Error().Err(err).Msg("cannot write metadata")
}
}
- if len(g.indexModeDocs) > 0 {
- if err := segment.IndexDB().Update(g.indexModeDocs); err != nil {
+ if len(dps.indexModeDocs) > 0 {
+ if err := dps.segment.IndexDB().Update(dps.indexModeDocs); err != nil {
w.l.Error().Err(err).Msg("cannot write index")
}
}
+ }
+ for _, segment := range g.segments {
segment.DecRef()
}
g.tsdb.Tick(g.latestTS)
diff --git a/banyand/stream/metadata.go b/banyand/stream/metadata.go
index eb7d3b0e..112db8b7 100644
--- a/banyand/stream/metadata.go
+++ b/banyand/stream/metadata.go
@@ -20,7 +20,6 @@ package stream
import (
"context"
"fmt"
- "io"
"path"
"time"
@@ -90,22 +89,6 @@ func newLiaisonSchemaRepo(path string, svc *liaison, streamDataNodeRegistry grpc
return sr
}
-// NewPortableRepository creates a new portable repository.
-func NewPortableRepository(metadata metadata.Repo, l *logger.Logger, metrics *resourceSchema.Metrics) SchemaService {
- r := &schemaRepo{
- l: l,
- metadata: metadata,
- Repository: resourceSchema.NewPortableRepository(
- metadata,
- l,
- newPortableSupplier(metadata, l),
- metrics,
- ),
- }
- r.start()
- return r
-}
-
func (sr *schemaRepo) start() {
sr.Watcher()
sr.metadata.
@@ -380,35 +363,6 @@ func (s *supplier) OpenDB(groupSchema *commonv1.Group) (resourceSchema.DB, error
)
}
-type portableSupplier struct {
- metadata metadata.Repo
- l *logger.Logger
-}
-
-func newPortableSupplier(metadata metadata.Repo, l *logger.Logger) *portableSupplier {
- return &portableSupplier{
- metadata: metadata,
- l: l,
- }
-}
-
-func (s *portableSupplier) ResourceSchema(md *commonv1.Metadata) (resourceSchema.ResourceSchema, error) {
- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
- defer cancel()
- return s.metadata.StreamRegistry().GetStream(ctx, md)
-}
-
-func (*portableSupplier) OpenDB(_ *commonv1.Group) (io.Closer, error) {
- panic("do not support open db")
-}
-
-func (s *portableSupplier) OpenResource(spec resourceSchema.Resource) (resourceSchema.IndexListener, error) {
- streamSchema := spec.Schema().(*databasev1.Stream)
- return openStream(streamSpec{
- schema: streamSchema,
- }, s.l, nil, nil), nil
-}
-
var _ resourceSchema.ResourceSupplier = (*queueSupplier)(nil)
type queueSupplier struct {
@@ -473,6 +427,7 @@ func (s *queueSupplier) OpenDB(groupSchema *commonv1.Group) (resourceSchema.DB,
opts := wqueue.Opts[*tsTable, option]{
Group: group,
ShardNum: shardNum,
+ SegmentInterval: storage.MustToIntervalRule(ro.SegmentInterval),
Location: path.Join(s.path, group),
Option: s.option,
Metrics: s.newMetrics(p),
diff --git a/banyand/stream/metrics.go b/banyand/stream/metrics.go
index 5a63c8e8..ec9d87f7 100644
--- a/banyand/stream/metrics.go
+++ b/banyand/stream/metrics.go
@@ -401,7 +401,9 @@ func (tst *tsTable) Collect(m storage.Metrics) {
metrics.totalFileBlocks.Set(float64(totalFileBlocks), tst.p.ShardLabelValues()...)
metrics.totalFilePartBytes.Set(float64(totalFilePartBytes), tst.p.ShardLabelValues()...)
metrics.totalFilePartUncompressedBytes.Set(float64(totalFilePartUncompressedBytes), tst.p.ShardLabelValues()...)
- tst.index.collectMetrics(tst.p.SegLabelValues()...)
+ if tst.index != nil {
+ tst.index.collectMetrics(tst.p.SegLabelValues()...)
+ }
}
func (tst *tsTable) deleteMetrics() {
diff --git a/banyand/stream/part.go b/banyand/stream/part.go
index 54d598b9..15620714 100644
--- a/banyand/stream/part.go
+++ b/banyand/stream/part.go
@@ -136,18 +136,21 @@ func (mp *memPart) reset() {
mp.primary.Reset()
mp.timestamps.Reset()
if mp.tagFamilies != nil {
- for _, tf := range mp.tagFamilies {
+ for k, tf := range mp.tagFamilies {
tf.Reset()
+ delete(mp.tagFamilies, k)
}
}
if mp.tagFamilyMetadata != nil {
- for _, tfh := range mp.tagFamilyMetadata {
+ for k, tfh := range mp.tagFamilyMetadata {
tfh.Reset()
+ delete(mp.tagFamilyMetadata, k)
}
}
if mp.tagFamilyFilter != nil {
- for _, tff := range mp.tagFamilyFilter {
+ for k, tff := range mp.tagFamilyFilter {
tff.Reset()
+ delete(mp.tagFamilyFilter, k)
}
}
}
diff --git a/banyand/stream/snapshot.go b/banyand/stream/snapshot.go
index c41a63cf..3e3731e8 100644
--- a/banyand/stream/snapshot.go
+++ b/banyand/stream/snapshot.go
@@ -194,6 +194,9 @@ func parseSnapshot(name string) (uint64, error) {
}
func (tst *tsTable) TakeFileSnapshot(dst string) error {
+ if tst.index == nil {
+ return fmt.Errorf("cannot take file snapshot: index is not initialized for this tsTable")
+ }
indexDir := filepath.Join(dst, filepath.Base(tst.index.location))
tst.fileSystem.MkdirPanicIfExist(indexDir, storage.DirPerm)
if err := tst.index.store.TakeFileSnapshot(indexDir); err != nil {
diff --git a/banyand/stream/syncer.go b/banyand/stream/syncer.go
index 0883340e..8f7060d8 100644
--- a/banyand/stream/syncer.go
+++ b/banyand/stream/syncer.go
@@ -19,7 +19,6 @@ package stream
import (
"context"
- "errors"
"fmt"
"time"
@@ -56,7 +55,7 @@ func (tst *tsTable) syncLoop(syncCh chan *syncIntroduction, flusherNotifier watc
defer tst.incTotalSyncLoopFinished(1)
var err error
if err = tst.syncSnapshot(curSnapshot, syncCh); err != nil {
- if errors.Is(err, errClosed) {
+ if tst.loopCloser.Closed() {
return true
}
tst.l.Logger.Warn().Err(err).Msgf("cannot sync snapshot: %d", curSnapshot.epoch)
@@ -64,6 +63,9 @@ func (tst *tsTable) syncLoop(syncCh chan *syncIntroduction, flusherNotifier watc
return false
}
epoch = curSnapshot.epoch
+ if tst.currentEpoch() != epoch {
+ return false
+ }
}
ew = flusherNotifier.Add(epoch, tst.loopCloser.CloseNotify())
return ew == nil
@@ -78,7 +80,7 @@ func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntrodu
// Get all parts from the current snapshot
var partsToSync []*part
for _, pw := range curSnapshot.parts {
- if pw.p.partMetadata.TotalCount > 0 {
+ if pw.mp == nil && pw.p.partMetadata.TotalCount > 0 {
partsToSync = append(partsToSync, pw.p)
}
}
@@ -112,8 +114,8 @@ func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntrodu
// Marshal memPart to []byte
partData, err := memPart.Marshal()
+ releaseMemPart(memPart)
if err != nil {
- releaseMemPart(memPart)
return fmt.Errorf("failed to marshal part %d: %w", part.partMetadata.ID, err)
}
@@ -125,13 +127,16 @@ func (tst *tsTable) syncSnapshot(curSnapshot *snapshot, syncCh chan *syncIntrodu
// Create message with combined data (group + shardID + partData)
message := bus.NewMessageWithNode(bus.MessageID(time.Now().UnixNano()), node, combinedData)
- _, err = tst.option.tire2Client.Publish(ctx, data.TopicStreamPartSync, message)
-
- releaseMemPart(memPart)
-
+ f, err := tst.option.tire2Client.Publish(ctx, data.TopicStreamPartSync, message)
if err != nil {
return fmt.Errorf("failed to publish part %d to pipeline: %w", part.partMetadata.ID, err)
}
+ if f != nil {
+ _, err = f.GetAll()
+ if err != nil {
+ tst.l.Warn().Err(err).Msgf("failed to get all sync results: %v", err)
+ }
+ }
}
}
diff --git a/banyand/stream/tstable.go b/banyand/stream/tstable.go
index 93163649..817ef7eb 100644
--- a/banyand/stream/tstable.go
+++ b/banyand/stream/tstable.go
@@ -186,7 +186,7 @@ func (tst *tsTable) mustReadSnapshot(snapshot uint64) []uint64 {
// initTSTable initializes a tsTable and loads parts/snapshots, but does not start any background loops.
func initTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
- l *logger.Logger, option option, m any,
+ l *logger.Logger, option option, m any, initIndex bool,
) (*tsTable, uint64, error) {
if option.protector == nil {
logger.GetLogger("stream").
@@ -206,11 +206,13 @@ func initTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
tst.metrics = m.(*metrics)
indexMetrics = tst.metrics.indexMetrics
}
- index, err := newElementIndex(context.TODO(), rootPath, option.elementIndexFlushTimeout.Nanoseconds()/int64(time.Second), indexMetrics)
- if err != nil {
- return nil, 0, err
+ if initIndex {
+ index, err := newElementIndex(context.TODO(), rootPath, option.elementIndexFlushTimeout.Nanoseconds()/int64(time.Second), indexMetrics)
+ if err != nil {
+ return nil, 0, err
+ }
+ tst.index = index
}
- tst.index = index
tst.gc.init(&tst)
ee := fileSystem.ReadDir(rootPath)
if len(ee) == 0 {
@@ -268,7 +270,7 @@ func initTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
l *logger.Logger, _ timestamp.TimeRange, option option, m any,
) (*tsTable, error) {
- t, epoch, err := initTSTable(fileSystem, rootPath, p, l, option, m)
+ t, epoch, err := initTSTable(fileSystem, rootPath, p, l, option, m, true)
if err != nil {
return nil, err
}
@@ -277,6 +279,9 @@ func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position,
}
func (tst *tsTable) Index() *elementIndex {
+ if tst.index == nil {
+ logger.Panicf("index is not initialized for this tsTable")
+ }
return tst.index
}
@@ -289,11 +294,17 @@ func (tst *tsTable) Close() error {
defer tst.Unlock()
tst.deleteMetrics()
if tst.snapshot == nil {
- return tst.index.Close()
+ if tst.index != nil {
+ return tst.index.Close()
+ }
+ return nil
}
tst.snapshot.decRef()
tst.snapshot = nil
- return tst.index.Close()
+ if tst.index != nil {
+ return tst.index.Close()
+ }
+ return nil
}
func (tst *tsTable) mustAddMemPart(mp *memPart) {
@@ -305,6 +316,7 @@ func (tst *tsTable) mustAddMemPart(mp *memPart) {
ind.memPart = newPartWrapper(mp, p)
ind.memPart.p.partMetadata.ID = atomic.AddUint64(&tst.curPartID, 1)
startTime := time.Now()
+ totalCount := mp.partMetadata.TotalCount
select {
case tst.introductions <- ind:
case <-tst.loopCloser.CloseNotify():
@@ -314,7 +326,7 @@ func (tst *tsTable) mustAddMemPart(mp *memPart) {
case <-ind.applied:
case <-tst.loopCloser.CloseNotify():
}
- tst.incTotalWritten(int(mp.partMetadata.TotalCount))
+ tst.incTotalWritten(int(totalCount))
tst.incTotalBatch(1)
tst.incTotalBatchIntroLatency(time.Since(startTime).Seconds())
}
diff --git a/banyand/stream/wqueue.go b/banyand/stream/wqueue.go
index e0852463..63618054 100644
--- a/banyand/stream/wqueue.go
+++ b/banyand/stream/wqueue.go
@@ -35,7 +35,7 @@ type NodeSelector interface {
func newWriteQueue(fileSystem fs.FileSystem, rootPath string, p common.Position,
l *logger.Logger, option option, m any, group string, shardID common.ShardID, getNodes func() []string,
) (*tsTable, error) {
- t, epoch, err := initTSTable(fileSystem, rootPath, p, l, option, m)
+ t, epoch, err := initTSTable(fileSystem, rootPath, p, l, option, m, false)
if err != nil {
return nil, err
}
diff --git a/banyand/stream/write.go b/banyand/stream/write.go
deleted file mode 100644
index 09756c45..00000000
--- a/banyand/stream/write.go
+++ /dev/null
@@ -1,430 +0,0 @@
-// Licensed to Apache Software Foundation (ASF) under one or more contributor
-// license agreements. See the NOTICE file distributed with
-// this work for additional information regarding copyright
-// ownership. Apache Software Foundation (ASF) licenses this file to you under
-// the Apache License, Version 2.0 (the "License"); you may
-// not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-package stream
-
-import (
- "bytes"
- "context"
- "fmt"
- "strings"
- "time"
-
- "google.golang.org/protobuf/proto"
-
- "github.com/apache/skywalking-banyandb/api/common"
- databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
- modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
- streamv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/stream/v1"
- "github.com/apache/skywalking-banyandb/banyand/internal/storage"
- "github.com/apache/skywalking-banyandb/banyand/observability"
- "github.com/apache/skywalking-banyandb/pkg/bus"
- "github.com/apache/skywalking-banyandb/pkg/convert"
- "github.com/apache/skywalking-banyandb/pkg/index"
- "github.com/apache/skywalking-banyandb/pkg/logger"
- pbv1 "github.com/apache/skywalking-banyandb/pkg/pb/v1"
- "github.com/apache/skywalking-banyandb/pkg/timestamp"
-)
-
-type writeCallback struct {
- l *logger.Logger
- schemaRepo *schemaRepo
- maxDiskUsagePercent int
-}
-
-func setUpWriteCallback(l *logger.Logger, schemaRepo *schemaRepo, maxDiskUsagePercent int) bus.MessageListener {
- if maxDiskUsagePercent > 100 {
- maxDiskUsagePercent = 100
- }
- return &writeCallback{
- l: l,
- schemaRepo: schemaRepo,
- maxDiskUsagePercent: maxDiskUsagePercent,
- }
-}
-
-func (w *writeCallback) CheckHealth() *common.Error {
- if w.maxDiskUsagePercent < 1 {
- return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "stream is readonly because \"stream-max-disk-usage-percent\" is 0")
- }
- diskPercent := observability.GetPathUsedPercent(w.schemaRepo.path)
- if diskPercent < w.maxDiskUsagePercent {
- return nil
- }
- w.l.Warn().Int("maxPercent", w.maxDiskUsagePercent).Int("diskPercent", diskPercent).Msg("disk usage is too high, stop writing")
- return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "disk usage is too high, stop writing")
-}
-
-func (w *writeCallback) handle(dst map[string]*elementsInGroup, writeEvent *streamv1.InternalWriteRequest,
- docIDBuilder *strings.Builder,
-) (map[string]*elementsInGroup, error) {
- t := writeEvent.Request.Element.Timestamp.AsTime().Local()
- if err := timestamp.Check(t); err != nil {
- return nil, fmt.Errorf("invalid timestamp: %w", err)
- }
- ts := t.UnixNano()
- eg, err := w.prepareElementsInGroup(dst, writeEvent, ts)
- if err != nil {
- return nil, err
- }
- et, err := w.prepareElementsInTable(eg, writeEvent, ts)
- if err != nil {
- return nil, err
- }
- err = processElements(w.schemaRepo, et.elements, writeEvent, docIDBuilder, ts, &et.docs, &et.seriesDocs)
- if err != nil {
- return nil, err
- }
- return dst, nil
-}
-
-func (w *writeCallback) prepareElementsInGroup(dst map[string]*elementsInGroup, writeEvent *streamv1.InternalWriteRequest, ts int64) (*elementsInGroup, error) {
- gn := writeEvent.Request.Metadata.Group
- tsdb, err := w.schemaRepo.loadTSDB(gn)
- if err != nil {
- return nil, fmt.Errorf("cannot load tsdb for group %s: %w", gn, err)
- }
-
- eg, ok := dst[gn]
- if !ok {
- eg = &elementsInGroup{
- tsdb: tsdb,
- tables: make([]*elementsInTable, 0),
- segments: make([]storage.Segment[*tsTable, option], 0),
- }
- dst[gn] = eg
- }
- if eg.latestTS < ts {
- eg.latestTS = ts
- }
- return eg, nil
-}
-
-func (w *writeCallback) prepareElementsInTable(eg *elementsInGroup, writeEvent *streamv1.InternalWriteRequest, ts int64) (*elementsInTable, error) {
- var et *elementsInTable
- for i := range eg.tables {
- if eg.tables[i].timeRange.Contains(ts) {
- et = eg.tables[i]
- break
- }
- }
-
- if et == nil {
- var segment storage.Segment[*tsTable, option]
- for _, seg := range eg.segments {
- if seg.GetTimeRange().Contains(ts) {
- segment = seg
- break
- }
- }
- if segment == nil {
- var err error
- segment, err = eg.tsdb.CreateSegmentIfNotExist(time.Unix(0, ts))
- if err != nil {
- return nil, fmt.Errorf("cannot create segment: %w", err)
- }
- eg.segments = append(eg.segments, segment)
- }
-
- shardID := common.ShardID(writeEvent.ShardId)
- tstb, err := segment.CreateTSTableIfNotExist(shardID)
- if err != nil {
- return nil, fmt.Errorf("cannot create ts table: %w", err)
- }
-
- et = &elementsInTable{
- timeRange: segment.GetTimeRange(),
- tsTable: tstb,
- elements: generateElements(),
- segment: segment,
- seriesDocs: seriesDoc{
- docs: make(index.Documents, 0),
- docIDsAdded: make(map[uint64]struct{}),
- },
- }
- et.elements.reset()
- eg.tables = append(eg.tables, et)
- }
- return et, nil
-}
-
-func processElements(schemaRepo *schemaRepo, elements *elements, writeEvent *streamv1.InternalWriteRequest,
- docIDBuilder *strings.Builder, ts int64, tableDocs *index.Documents, seriesDocs *seriesDoc,
-) error {
- req := writeEvent.Request
-
- elements.timestamps = append(elements.timestamps, ts)
- docIDBuilder.Reset()
- docIDBuilder.WriteString(req.Metadata.Name)
- docIDBuilder.WriteByte('|')
- docIDBuilder.WriteString(req.Element.ElementId)
- eID := convert.HashStr(docIDBuilder.String())
- elements.elementIDs = append(elements.elementIDs, eID)
-
- stm, ok := schemaRepo.loadStream(writeEvent.GetRequest().GetMetadata())
- if !ok {
- return fmt.Errorf("cannot find stream definition: %s", writeEvent.GetRequest().GetMetadata())
- }
-
- fLen := len(req.Element.GetTagFamilies())
- if fLen < 1 {
- return fmt.Errorf("%s has no tag family", req)
- }
- if fLen > len(stm.schema.GetTagFamilies()) {
- return fmt.Errorf("%s has more tag families than %s", req.Metadata, stm.schema)
- }
-
- series := &pbv1.Series{
- Subject: req.Metadata.Name,
- EntityValues: writeEvent.EntityValues,
- }
- if err := series.Marshal(); err != nil {
- return fmt.Errorf("cannot marshal series: %w", err)
- }
- elements.seriesIDs = append(elements.seriesIDs, series.ID)
-
- is := stm.indexSchema.Load().(indexSchema)
- tagFamilies := make([]tagValues, 0, len(stm.schema.TagFamilies))
- indexedTags := make(map[string]map[string]struct{})
- var fields []index.Field
-
- if len(is.indexRuleLocators.TagFamilyTRule) != len(stm.GetSchema().GetTagFamilies()) {
- return fmt.Errorf("metadata crashed, tag family rule length %d, tag family length %d",
- len(is.indexRuleLocators.TagFamilyTRule), len(stm.GetSchema().GetTagFamilies()))
- }
-
- for i := range stm.GetSchema().GetTagFamilies() {
- var tagFamily *modelv1.TagFamilyForWrite
- if len(req.Element.TagFamilies) <= i {
- tagFamily = pbv1.NullTagFamily
- } else {
- tagFamily = req.Element.TagFamilies[i]
- }
- tfr := is.indexRuleLocators.TagFamilyTRule[i]
- tagFamilySpec := stm.GetSchema().GetTagFamilies()[i]
- tf := tagValues{
- tag: tagFamilySpec.Name,
- }
- indexedTags[tagFamilySpec.Name] = make(map[string]struct{})
-
- for j := range tagFamilySpec.Tags {
- var tagValue *modelv1.TagValue
- if tagFamily == pbv1.NullTagFamily || len(tagFamily.Tags) <= j {
- tagValue = pbv1.NullTagValue
- } else {
- tagValue = tagFamily.Tags[j]
- }
-
- t := tagFamilySpec.Tags[j]
- indexed := false
- if r, ok := tfr[t.Name]; ok && tagValue != pbv1.NullTagValue {
- if r.GetType() == databasev1.IndexRule_TYPE_INVERTED {
- fields = appendField(fields, index.FieldKey{
- IndexRuleID: r.GetMetadata().GetId(),
- Analyzer: r.Analyzer,
- SeriesID: series.ID,
- }, t.Type, tagValue, r.GetNoSort())
- } else if r.GetType() == databasev1.IndexRule_TYPE_SKIPPING {
- indexed = true
- }
- }
- _, isEntity := is.indexRuleLocators.EntitySet[t.Name]
- if tagFamilySpec.Tags[j].IndexedOnly || isEntity {
- continue
- }
- tv := encodeTagValue(t.Name, t.Type, tagValue)
- tv.indexed = indexed
- tf.values = append(tf.values, tv)
- }
- if len(tf.values) > 0 {
- tagFamilies = append(tagFamilies, tf)
- }
- }
- elements.tagFamilies = append(elements.tagFamilies, tagFamilies)
-
- *tableDocs = append(*tableDocs, index.Document{
- DocID: eID,
- Fields: fields,
- Timestamp: ts,
- })
-
- docID := uint64(series.ID)
- if _, existed := seriesDocs.docIDsAdded[docID]; !existed {
- seriesDocs.docs = append(seriesDocs.docs, index.Document{
- DocID: docID,
- EntityValues: series.Buffer,
- })
- seriesDocs.docIDsAdded[docID] = struct{}{}
- }
-
- return nil
-}
-
-func (w *writeCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- events, ok := message.Data().([]any)
- if !ok {
- w.l.Warn().Msg("invalid event data type")
- return
- }
- if len(events) < 1 {
- w.l.Warn().Msg("empty event")
- return
- }
- groups := make(map[string]*elementsInGroup)
- var builder strings.Builder
- for i := range events {
- var writeEvent *streamv1.InternalWriteRequest
- switch e := events[i].(type) {
- case *streamv1.InternalWriteRequest:
- writeEvent = e
- case []byte:
- writeEvent = &streamv1.InternalWriteRequest{}
- if err := proto.Unmarshal(e, writeEvent); err != nil {
- w.l.Error().Err(err).RawJSON("written", e).Msg("fail to unmarshal event")
- continue
- }
- default:
- w.l.Warn().Msg("invalid event data type")
- continue
- }
- var err error
- if groups, err = w.handle(groups, writeEvent, &builder); err != nil {
- w.l.Error().Err(err).Msg("cannot handle write event")
- groups = make(map[string]*elementsInGroup)
- continue
- }
- }
- for i := range groups {
- g := groups[i]
- for j := range g.tables {
- es := g.tables[j]
- es.tsTable.mustAddElements(es.elements)
- releaseElements(es.elements)
- if len(es.docs) > 0 {
- index := es.tsTable.Index()
- if err := index.Write(es.docs); err != nil {
- w.l.Error().Err(err).Msg("cannot write element index")
- }
- }
- if len(es.seriesDocs.docs) > 0 {
- if err := es.segment.IndexDB().Insert(es.seriesDocs.docs); err != nil {
- w.l.Error().Err(err).Msg("cannot write series index")
- }
- }
- }
- if len(g.segments) > 0 {
- for _, segment := range g.segments {
- segment.DecRef()
- }
- }
- g.tsdb.Tick(g.latestTS)
- }
- return
-}
-
-func encodeTagValue(name string, tagType databasev1.TagType, tagVal *modelv1.TagValue) *tagValue {
- tv := generateTagValue()
- tv.tag = name
- switch tagType {
- case databasev1.TagType_TAG_TYPE_INT:
- tv.valueType = pbv1.ValueTypeInt64
- if tagVal.GetInt() != nil {
- tv.value = convert.Int64ToBytes(tagVal.GetInt().GetValue())
- }
- case databasev1.TagType_TAG_TYPE_STRING:
- tv.valueType = pbv1.ValueTypeStr
- if tagVal.GetStr() != nil {
- tv.value = convert.StringToBytes(tagVal.GetStr().GetValue())
- }
- case databasev1.TagType_TAG_TYPE_DATA_BINARY:
- tv.valueType = pbv1.ValueTypeBinaryData
- if tagVal.GetBinaryData() != nil {
- tv.value = bytes.Clone(tagVal.GetBinaryData())
- }
- case databasev1.TagType_TAG_TYPE_INT_ARRAY:
- tv.valueType = pbv1.ValueTypeInt64Arr
- if tagVal.GetIntArray() == nil {
- return tv
- }
- tv.valueArr = make([][]byte, len(tagVal.GetIntArray().Value))
- for i := range tagVal.GetIntArray().Value {
- tv.valueArr[i] = convert.Int64ToBytes(tagVal.GetIntArray().Value[i])
- }
- case databasev1.TagType_TAG_TYPE_STRING_ARRAY:
- tv.valueType = pbv1.ValueTypeStrArr
- if tagVal.GetStrArray() == nil {
- return tv
- }
- tv.valueArr = make([][]byte, len(tagVal.GetStrArray().Value))
- for i := range tagVal.GetStrArray().Value {
- tv.valueArr[i] = []byte(tagVal.GetStrArray().Value[i])
- }
- default:
- logger.Panicf("unsupported tag value type: %T", tagVal.GetValue())
- }
- return tv
-}
-
-func appendField(dest []index.Field, fieldKey index.FieldKey, tagType databasev1.TagType, tagVal *modelv1.TagValue, noSort bool) []index.Field {
- switch tagType {
- case databasev1.TagType_TAG_TYPE_INT:
- v := tagVal.GetInt()
- if v == nil {
- return dest
- }
- f := index.NewIntField(fieldKey, v.Value)
- f.NoSort = noSort
- dest = append(dest, f)
- case databasev1.TagType_TAG_TYPE_STRING:
- v := tagVal.GetStr()
- if v == nil {
- return dest
- }
- f := index.NewStringField(fieldKey, v.Value)
- f.NoSort = noSort
- dest = append(dest, f)
- case databasev1.TagType_TAG_TYPE_DATA_BINARY:
- v := tagVal.GetBinaryData()
- if v == nil {
- return dest
- }
- f := index.NewBytesField(fieldKey, v)
- f.NoSort = noSort
- dest = append(dest, f)
- case databasev1.TagType_TAG_TYPE_INT_ARRAY:
- if tagVal.GetIntArray() == nil {
- return dest
- }
- for i := range tagVal.GetIntArray().Value {
- f := index.NewIntField(fieldKey, tagVal.GetIntArray().Value[i])
- f.NoSort = noSort
- dest = append(dest, f)
- }
- case databasev1.TagType_TAG_TYPE_STRING_ARRAY:
- if tagVal.GetStrArray() == nil {
- return dest
- }
- for i := range tagVal.GetStrArray().Value {
- f := index.NewStringField(fieldKey, tagVal.GetStrArray().Value[i])
- f.NoSort = noSort
- dest = append(dest, f)
- }
- default:
- logger.Panicf("unsupported tag value type: %T", tagVal.GetValue())
- }
- return dest
-}
diff --git a/banyand/stream/write_standalone.go b/banyand/stream/write_data.go
similarity index 97%
copy from banyand/stream/write_standalone.go
copy to banyand/stream/write_data.go
index 6d927b7e..5d2d66d8 100644
--- a/banyand/stream/write_standalone.go
+++ b/banyand/stream/write_data.go
@@ -102,11 +102,13 @@ func (s *syncCallback) Rev(_ context.Context, message bus.Message) (resp bus.Mes
tsTable.mustAddMemPart(memPart)
tsdb.Tick(memPart.partMetadata.MaxTimestamp)
- s.l.Info().
- Str("group", group).
- Uint32("shardID", shardID).
- Uint64("partID", memPart.partMetadata.ID).
- Msg("successfully stored memPart to tsTable")
+ if e := s.l.Debug(); e.Enabled() {
+ e.
+ Str("group", group).
+ Uint32("shardID", shardID).
+ Uint64("partID", memPart.partMetadata.ID).
+ Msg("successfully stored memPart to tsTable")
+ }
return
}
diff --git a/banyand/stream/write_liaison.go b/banyand/stream/write_liaison.go
index 0007d15b..b804be99 100644
--- a/banyand/stream/write_liaison.go
+++ b/banyand/stream/write_liaison.go
@@ -20,7 +20,6 @@ package stream
import (
"context"
"fmt"
- "strings"
"time"
"google.golang.org/protobuf/proto"
@@ -70,9 +69,7 @@ func (w *writeQueueCallback) CheckHealth() *common.Error {
return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "disk usage is too high, stop writing")
}
-func (w *writeQueueCallback) handle(dst map[string]*elementsInQueue, writeEvent *streamv1.InternalWriteRequest,
- docIDBuilder *strings.Builder,
-) (map[string]*elementsInQueue, error) {
+func (w *writeQueueCallback) handle(dst map[string]*elementsInQueue, writeEvent *streamv1.InternalWriteRequest) (map[string]*elementsInQueue, error) {
t := writeEvent.Request.Element.Timestamp.AsTime().Local()
if err := timestamp.Check(t); err != nil {
return nil, fmt.Errorf("invalid timestamp: %w", err)
@@ -86,7 +83,7 @@ func (w *writeQueueCallback) handle(dst map[string]*elementsInQueue, writeEvent
if err != nil {
return nil, err
}
- err = processElements(w.schemaRepo, et.elements, writeEvent, docIDBuilder, ts, &et.docs, &et.seriesDocs)
+ err = processElements(w.schemaRepo, et.elements, writeEvent, ts, &et.docs, &et.seriesDocs)
if err != nil {
return nil, err
}
@@ -158,7 +155,6 @@ func (w *writeQueueCallback) Rev(ctx context.Context, message bus.Message) (resp
return
}
groups := make(map[string]*elementsInQueue)
- var builder strings.Builder
for i := range events {
var writeEvent *streamv1.InternalWriteRequest
switch e := events[i].(type) {
@@ -175,7 +171,7 @@ func (w *writeQueueCallback) Rev(ctx context.Context, message bus.Message) (resp
continue
}
var err error
- if groups, err = w.handle(groups, writeEvent, &builder); err != nil {
+ if groups, err = w.handle(groups, writeEvent); err != nil {
w.l.Error().Err(err).Msg("cannot handle write event")
groups = make(map[string]*elementsInQueue)
continue
diff --git a/banyand/stream/write_standalone.go b/banyand/stream/write_standalone.go
index 6d927b7e..a81ff45c 100644
--- a/banyand/stream/write_standalone.go
+++ b/banyand/stream/write_standalone.go
@@ -18,266 +18,406 @@
package stream
import (
+ "bytes"
"context"
+ "fmt"
"time"
+ "google.golang.org/protobuf/proto"
+
"github.com/apache/skywalking-banyandb/api/common"
+ databasev1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
+ modelv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/model/v1"
+ streamv1 "github.com/apache/skywalking-banyandb/api/proto/banyandb/stream/v1"
+ "github.com/apache/skywalking-banyandb/banyand/internal/storage"
+ "github.com/apache/skywalking-banyandb/banyand/observability"
"github.com/apache/skywalking-banyandb/pkg/bus"
- "github.com/apache/skywalking-banyandb/pkg/encoding"
+ "github.com/apache/skywalking-banyandb/pkg/convert"
"github.com/apache/skywalking-banyandb/pkg/index"
"github.com/apache/skywalking-banyandb/pkg/logger"
+ pbv1 "github.com/apache/skywalking-banyandb/pkg/pb/v1"
+ "github.com/apache/skywalking-banyandb/pkg/timestamp"
)
-type syncCallback struct {
- l *logger.Logger
- schemaRepo *schemaRepo
+type writeCallback struct {
+ l *logger.Logger
+ schemaRepo *schemaRepo
+ maxDiskUsagePercent int
}
-func setUpSyncCallback(l *logger.Logger, schemaRepo *schemaRepo) bus.MessageListener {
- return &syncCallback{
- l: l,
- schemaRepo: schemaRepo,
+func setUpWriteCallback(l *logger.Logger, schemaRepo *schemaRepo, maxDiskUsagePercent int) bus.MessageListener {
+ if maxDiskUsagePercent > 100 {
+ maxDiskUsagePercent = 100
+ }
+ return &writeCallback{
+ l: l,
+ schemaRepo: schemaRepo,
+ maxDiskUsagePercent: maxDiskUsagePercent,
}
}
-func (s *syncCallback) CheckHealth() *common.Error {
- return nil
-}
-
-func (s *syncCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- data, ok := message.Data().([]byte)
- if !ok {
- s.l.Warn().Msg("invalid sync message data type")
- return
+func (w *writeCallback) CheckHealth() *common.Error {
+ if w.maxDiskUsagePercent < 1 {
+ return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "stream is readonly because \"stream-max-disk-usage-percent\" is 0")
}
-
- // Decode group name
- tail, groupBytes, err := encoding.DecodeBytes(data)
- if err != nil {
- s.l.Error().Err(err).Msg("failed to decode group name from sync message")
- return
+ diskPercent := observability.GetPathUsedPercent(w.schemaRepo.path)
+ if diskPercent < w.maxDiskUsagePercent {
+ return nil
}
- group := string(groupBytes)
+ w.l.Warn().Int("maxPercent", w.maxDiskUsagePercent).Int("diskPercent", diskPercent).Msg("disk usage is too high, stop writing")
+ return common.NewErrorWithStatus(modelv1.Status_STATUS_DISK_FULL, "disk usage is too high, stop writing")
+}
- // Decode shardID (uint32)
- if len(tail) < 4 {
- s.l.Error().Msg("insufficient data for shardID in sync message")
- return
+func (w *writeCallback) handle(dst map[string]*elementsInGroup, writeEvent *streamv1.InternalWriteRequest,
+) (map[string]*elementsInGroup, error) {
+ t := writeEvent.Request.Element.Timestamp.AsTime().Local()
+ if err := timestamp.Check(t); err != nil {
+ return nil, fmt.Errorf("invalid timestamp: %w", err)
}
- shardID := encoding.BytesToUint32(tail[:4])
- tail = tail[4:]
-
- // Decode memPart
- memPart := generateMemPart()
-
- if err = memPart.Unmarshal(tail); err != nil {
- s.l.Error().Err(err).Msg("failed to unmarshal memPart from sync message")
- return
+ ts := t.UnixNano()
+ eg, err := w.prepareElementsInGroup(dst, writeEvent, ts)
+ if err != nil {
+ return nil, err
}
-
- // Get the group schema from schemaRepo by the string group
- tsdb, err := s.schemaRepo.loadTSDB(group)
+ et, err := w.prepareElementsInTable(eg, writeEvent, ts)
if err != nil {
- s.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
- return
+ return nil, err
}
-
- // Get the segment using memPart.partMetadata.MinTimestamp
- segmentTime := time.Unix(0, memPart.partMetadata.MinTimestamp)
- segment, err := tsdb.CreateSegmentIfNotExist(segmentTime)
+ err = processElements(w.schemaRepo, et.elements, writeEvent, ts, &et.docs, &et.seriesDocs)
if err != nil {
- s.l.Error().Err(err).Str("group", group).Time("segmentTime", segmentTime).Msg("failed to create segment")
- return
+ return nil, err
}
- defer segment.DecRef()
+ return dst, nil
+}
- // Get Shard from a segment using shardID
- tsTable, err := segment.CreateTSTableIfNotExist(common.ShardID(shardID))
+func (w *writeCallback) prepareElementsInGroup(dst map[string]*elementsInGroup, writeEvent *streamv1.InternalWriteRequest, ts int64) (*elementsInGroup, error) {
+ gn := writeEvent.Request.Metadata.Group
+ tsdb, err := w.schemaRepo.loadTSDB(gn)
if err != nil {
- s.l.Error().Err(err).Str("group", group).Uint32("shardID", shardID).Msg("failed to create ts table")
- return
+ return nil, fmt.Errorf("cannot load tsdb for group %s: %w", gn, err)
}
- // Use tsTable.mustAddMemPart to introduce memPart to tsTable
- tsTable.mustAddMemPart(memPart)
- tsdb.Tick(memPart.partMetadata.MaxTimestamp)
-
- s.l.Info().
- Str("group", group).
- Uint32("shardID", shardID).
- Uint64("partID", memPart.partMetadata.ID).
- Msg("successfully stored memPart to tsTable")
-
- return
+ eg, ok := dst[gn]
+ if !ok {
+ eg = &elementsInGroup{
+ tsdb: tsdb,
+ tables: make([]*elementsInTable, 0),
+ segments: make([]storage.Segment[*tsTable, option], 0),
+ }
+ dst[gn] = eg
+ }
+ if eg.latestTS < ts {
+ eg.latestTS = ts
+ }
+ return eg, nil
}
-type seriesIndexCallback struct {
- l *logger.Logger
- schemaRepo *schemaRepo
-}
+func (w *writeCallback) prepareElementsInTable(eg *elementsInGroup, writeEvent *streamv1.InternalWriteRequest, ts int64) (*elementsInTable, error) {
+ var et *elementsInTable
+ for i := range eg.tables {
+ if eg.tables[i].timeRange.Contains(ts) {
+ et = eg.tables[i]
+ break
+ }
+ }
-func setUpSeriesIndexCallback(l *logger.Logger, schemaRepo *schemaRepo) bus.MessageListener {
- return &seriesIndexCallback{
- l: l,
- schemaRepo: schemaRepo,
+ if et == nil {
+ var segment storage.Segment[*tsTable, option]
+ for _, seg := range eg.segments {
+ if seg.GetTimeRange().Contains(ts) {
+ segment = seg
+ break
+ }
+ }
+ if segment == nil {
+ var err error
+ segment, err = eg.tsdb.CreateSegmentIfNotExist(time.Unix(0, ts))
+ if err != nil {
+ return nil, fmt.Errorf("cannot create segment: %w", err)
+ }
+ eg.segments = append(eg.segments, segment)
+ }
+
+ shardID := common.ShardID(writeEvent.ShardId)
+ tstb, err := segment.CreateTSTableIfNotExist(shardID)
+ if err != nil {
+ return nil, fmt.Errorf("cannot create ts table: %w", err)
+ }
+
+ et = &elementsInTable{
+ timeRange: segment.GetTimeRange(),
+ tsTable: tstb,
+ elements: generateElements(),
+ segment: segment,
+ seriesDocs: seriesDoc{
+ docs: make(index.Documents, 0),
+ docIDsAdded: make(map[uint64]struct{}),
+ },
+ }
+ et.elements.reset()
+ eg.tables = append(eg.tables, et)
}
+ return et, nil
}
-func (s *seriesIndexCallback) CheckHealth() *common.Error {
- return nil
-}
+func processElements(schemaRepo *schemaRepo, elements *elements, writeEvent *streamv1.InternalWriteRequest,
+ ts int64, tableDocs *index.Documents, seriesDocs *seriesDoc,
+) error {
+ req := writeEvent.Request
+
+ elements.timestamps = append(elements.timestamps, ts)
+ eID := convert.HashStr(req.Metadata.Name + "|" + req.Element.ElementId)
+ elements.elementIDs = append(elements.elementIDs, eID)
-func (s *seriesIndexCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- data, ok := message.Data().([]byte)
+ stm, ok := schemaRepo.loadStream(writeEvent.GetRequest().GetMetadata())
if !ok {
- s.l.Warn().Msg("invalid series index message data type")
- return
+ return fmt.Errorf("cannot find stream definition: %s", writeEvent.GetRequest().GetMetadata())
}
- // Decode group name (first part of the message)
- tail, groupBytes, err := encoding.DecodeBytes(data)
- if err != nil {
- s.l.Error().Err(err).Msg("failed to decode group name from series index message")
- return
+ fLen := len(req.Element.GetTagFamilies())
+ if fLen < 1 {
+ return fmt.Errorf("%s has no tag family", req)
}
- group := string(groupBytes)
-
- // Decode timestamp (uint64) - second part of the message
- if len(tail) < 8 {
- s.l.Error().Msg("insufficient data for timestamp in series index message")
- return
+ if fLen > len(stm.schema.GetTagFamilies()) {
+ return fmt.Errorf("%s has more tag families than %s", req.Metadata, stm.schema)
}
- timestamp := encoding.BytesToInt64(tail[:8])
- tail = tail[8:]
- // Unmarshal Documents from the remaining data
- var documents index.Documents
- if err = documents.Unmarshal(tail); err != nil {
- s.l.Error().Err(err).Msg("failed to unmarshal documents from series index message")
- return
+ series := &pbv1.Series{
+ Subject: req.Metadata.Name,
+ EntityValues: writeEvent.EntityValues,
}
-
- if len(documents) == 0 {
- s.l.Warn().Msg("empty documents in series index message")
- return
+ if err := series.Marshal(); err != nil {
+ return fmt.Errorf("cannot marshal series: %w", err)
}
+ elements.seriesIDs = append(elements.seriesIDs, series.ID)
- // Get TSDB by group name using schemaRepo
- tsdb, err := s.schemaRepo.loadTSDB(group)
- if err != nil {
- s.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
- return
- }
+ is := stm.indexSchema.Load().(indexSchema)
+ tagFamilies := make([]tagValues, 0, len(stm.schema.TagFamilies))
+ indexedTags := make(map[string]map[string]struct{})
+ var fields []index.Field
- // Use the timestamp to get the segment directly
- segmentTime := time.Unix(0, timestamp)
- segment, err := tsdb.CreateSegmentIfNotExist(segmentTime)
- if err != nil {
- s.l.Error().Err(err).Str("group", group).Time("segmentTime", segmentTime).Msg("failed to create segment")
- return
+ if len(is.indexRuleLocators.TagFamilyTRule) != len(stm.GetSchema().GetTagFamilies()) {
+ return fmt.Errorf("metadata crashed, tag family rule length %d, tag family length %d",
+ len(is.indexRuleLocators.TagFamilyTRule), len(stm.GetSchema().GetTagFamilies()))
}
- defer segment.DecRef()
- // Insert all documents into the segment's index
- if err := segment.IndexDB().Insert(documents); err != nil {
- s.l.Error().Err(err).Str("group", group).Int("documentCount", len(documents)).Msg("failed to insert documents to index")
- return
+ for i := range stm.GetSchema().GetTagFamilies() {
+ var tagFamily *modelv1.TagFamilyForWrite
+ if len(req.Element.TagFamilies) <= i {
+ tagFamily = pbv1.NullTagFamily
+ } else {
+ tagFamily = req.Element.TagFamilies[i]
+ }
+ tfr := is.indexRuleLocators.TagFamilyTRule[i]
+ tagFamilySpec := stm.GetSchema().GetTagFamilies()[i]
+ tf := tagValues{
+ tag: tagFamilySpec.Name,
+ }
+ indexedTags[tagFamilySpec.Name] = make(map[string]struct{})
+
+ for j := range tagFamilySpec.Tags {
+ var tagValue *modelv1.TagValue
+ if tagFamily == pbv1.NullTagFamily || len(tagFamily.Tags) <= j {
+ tagValue = pbv1.NullTagValue
+ } else {
+ tagValue = tagFamily.Tags[j]
+ }
+
+ t := tagFamilySpec.Tags[j]
+ indexed := false
+ if r, ok := tfr[t.Name]; ok && tagValue != pbv1.NullTagValue {
+ if r.GetType() == databasev1.IndexRule_TYPE_INVERTED {
+ fields = appendField(fields, index.FieldKey{
+ IndexRuleID: r.GetMetadata().GetId(),
+ Analyzer: r.Analyzer,
+ SeriesID: series.ID,
+ }, t.Type, tagValue, r.GetNoSort())
+ } else if r.GetType() == databasev1.IndexRule_TYPE_SKIPPING {
+ indexed = true
+ }
+ }
+ _, isEntity := is.indexRuleLocators.EntitySet[t.Name]
+ if tagFamilySpec.Tags[j].IndexedOnly || isEntity {
+ continue
+ }
+ tv := encodeTagValue(t.Name, t.Type, tagValue)
+ tv.indexed = indexed
+ tf.values = append(tf.values, tv)
+ }
+ if len(tf.values) > 0 {
+ tagFamilies = append(tagFamilies, tf)
+ }
}
-
- s.l.Info().
- Str("group", group).
- Int("documentCount", len(documents)).
- Time("segmentTime", segmentTime).
- Msg("successfully inserted documents to series index")
-
- return
-}
-
-type localIndexCallback struct {
- l *logger.Logger
- schemaRepo *schemaRepo
-}
-
-func setUpLocalIndexCallback(l *logger.Logger, schemaRepo *schemaRepo) bus.MessageListener {
- return &localIndexCallback{
- l: l,
- schemaRepo: schemaRepo,
+ elements.tagFamilies = append(elements.tagFamilies, tagFamilies)
+
+ *tableDocs = append(*tableDocs, index.Document{
+ DocID: eID,
+ Fields: fields,
+ Timestamp: ts,
+ })
+
+ docID := uint64(series.ID)
+ if _, existed := seriesDocs.docIDsAdded[docID]; !existed {
+ seriesDocs.docs = append(seriesDocs.docs, index.Document{
+ DocID: docID,
+ EntityValues: series.Buffer,
+ })
+ seriesDocs.docIDsAdded[docID] = struct{}{}
}
-}
-func (l *localIndexCallback) CheckHealth() *common.Error {
return nil
}
-func (l *localIndexCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
- data, ok := message.Data().([]byte)
+func (w *writeCallback) Rev(_ context.Context, message bus.Message) (resp bus.Message) {
+ events, ok := message.Data().([]any)
if !ok {
- l.l.Warn().Msg("invalid local index message data type")
- return
- }
-
- // Decode group name (first part of the message)
- tail, groupBytes, err := encoding.DecodeBytes(data)
- if err != nil {
- l.l.Error().Err(err).Msg("failed to decode group name from local index message")
- return
- }
- group := string(groupBytes)
-
- // Decode shardID (uint32)
- if len(tail) < 4 {
- l.l.Error().Msg("insufficient data for shardID in local index message")
+ w.l.Warn().Msg("invalid event data type")
return
}
- shardID := encoding.BytesToUint32(tail[:4])
- tail = tail[4:]
-
- // Unmarshal Documents from the remaining data
- var documents index.Documents
- if err = documents.Unmarshal(tail); err != nil {
- l.l.Error().Err(err).Msg("failed to unmarshal documents from local index message")
- return
- }
-
- if len(documents) == 0 {
- l.l.Warn().Msg("empty documents in local index message")
+ if len(events) < 1 {
+ w.l.Warn().Msg("empty event")
return
}
-
- // Get TSDB by group name using schemaRepo
- tsdb, err := l.schemaRepo.loadTSDB(group)
- if err != nil {
- l.l.Error().Err(err).Str("group", group).Msg("failed to load TSDB for group")
- return
+ groups := make(map[string]*elementsInGroup)
+ for i := range events {
+ var writeEvent *streamv1.InternalWriteRequest
+ switch e := events[i].(type) {
+ case *streamv1.InternalWriteRequest:
+ writeEvent = e
+ case []byte:
+ writeEvent = &streamv1.InternalWriteRequest{}
+ if err := proto.Unmarshal(e, writeEvent); err != nil {
+ w.l.Error().Err(err).RawJSON("written", e).Msg("fail to unmarshal event")
+ continue
+ }
+ default:
+ w.l.Warn().Msg("invalid event data type")
+ continue
+ }
+ var err error
+ if groups, err = w.handle(groups, writeEvent); err != nil {
+ w.l.Error().Err(err).Msg("cannot handle write event")
+ groups = make(map[string]*elementsInGroup)
+ continue
+ }
}
-
- // Use the first document's timestamp to get the segment
- firstDocTime := time.Unix(0, documents[0].Timestamp)
- segment, err := tsdb.CreateSegmentIfNotExist(firstDocTime)
- if err != nil {
- l.l.Error().Err(err).Str("group", group).Time("docTime", firstDocTime).Msg("failed to create segment for documents")
- return
+ for i := range groups {
+ g := groups[i]
+ for j := range g.tables {
+ es := g.tables[j]
+ es.tsTable.mustAddElements(es.elements)
+ releaseElements(es.elements)
+ if len(es.docs) > 0 {
+ index := es.tsTable.Index()
+ if err := index.Write(es.docs); err != nil {
+ w.l.Error().Err(err).Msg("cannot write element index")
+ }
+ }
+ if len(es.seriesDocs.docs) > 0 {
+ if err := es.segment.IndexDB().Insert(es.seriesDocs.docs); err != nil {
+ w.l.Error().Err(err).Msg("cannot write series index")
+ }
+ }
+ }
+ if len(g.segments) > 0 {
+ for _, segment := range g.segments {
+ segment.DecRef()
+ }
+ }
+ g.tsdb.Tick(g.latestTS)
}
- defer segment.DecRef()
+ return
+}
- // Get the tsTable using shardID
- tsTable, err := segment.CreateTSTableIfNotExist(common.ShardID(shardID))
- if err != nil {
- l.l.Error().Err(err).Str("group", group).Uint32("shardID", shardID).Msg("failed to create ts table")
- return
+func encodeTagValue(name string, tagType databasev1.TagType, tagVal *modelv1.TagValue) *tagValue {
+ tv := generateTagValue()
+ tv.tag = name
+ switch tagType {
+ case databasev1.TagType_TAG_TYPE_INT:
+ tv.valueType = pbv1.ValueTypeInt64
+ if tagVal.GetInt() != nil {
+ tv.value = convert.Int64ToBytes(tagVal.GetInt().GetValue())
+ }
+ case databasev1.TagType_TAG_TYPE_STRING:
+ tv.valueType = pbv1.ValueTypeStr
+ if tagVal.GetStr() != nil {
+ tv.value = convert.StringToBytes(tagVal.GetStr().GetValue())
+ }
+ case databasev1.TagType_TAG_TYPE_DATA_BINARY:
+ tv.valueType = pbv1.ValueTypeBinaryData
+ if tagVal.GetBinaryData() != nil {
+ tv.value = bytes.Clone(tagVal.GetBinaryData())
+ }
+ case databasev1.TagType_TAG_TYPE_INT_ARRAY:
+ tv.valueType = pbv1.ValueTypeInt64Arr
+ if tagVal.GetIntArray() == nil {
+ return tv
+ }
+ tv.valueArr = make([][]byte, len(tagVal.GetIntArray().Value))
+ for i := range tagVal.GetIntArray().Value {
+ tv.valueArr[i] = convert.Int64ToBytes(tagVal.GetIntArray().Value[i])
+ }
+ case databasev1.TagType_TAG_TYPE_STRING_ARRAY:
+ tv.valueType = pbv1.ValueTypeStrArr
+ if tagVal.GetStrArray() == nil {
+ return tv
+ }
+ tv.valueArr = make([][]byte, len(tagVal.GetStrArray().Value))
+ for i := range tagVal.GetStrArray().Value {
+ tv.valueArr[i] = []byte(tagVal.GetStrArray().Value[i])
+ }
+ default:
+ logger.Panicf("unsupported tag value type: %T", tagVal.GetValue())
}
+ return tv
+}
- // Insert documents into tsTable.index
- if err := tsTable.Index().Write(documents); err != nil {
- l.l.Error().Err(err).Str("group", group).Uint32("shardID", shardID).Int("documentCount", len(documents)).Msg("failed to insert documents to tsTable index")
- return
+func appendField(dest []index.Field, fieldKey index.FieldKey, tagType databasev1.TagType, tagVal *modelv1.TagValue, noSort bool) []index.Field {
+ switch tagType {
+ case databasev1.TagType_TAG_TYPE_INT:
+ v := tagVal.GetInt()
+ if v == nil {
+ return dest
+ }
+ f := index.NewIntField(fieldKey, v.Value)
+ f.NoSort = noSort
+ dest = append(dest, f)
+ case databasev1.TagType_TAG_TYPE_STRING:
+ v := tagVal.GetStr()
+ if v == nil {
+ return dest
+ }
+ f := index.NewStringField(fieldKey, v.Value)
+ f.NoSort = noSort
+ dest = append(dest, f)
+ case databasev1.TagType_TAG_TYPE_DATA_BINARY:
+ v := tagVal.GetBinaryData()
+ if v == nil {
+ return dest
+ }
+ f := index.NewBytesField(fieldKey, v)
+ f.NoSort = noSort
+ dest = append(dest, f)
+ case databasev1.TagType_TAG_TYPE_INT_ARRAY:
+ if tagVal.GetIntArray() == nil {
+ return dest
+ }
+ for i := range tagVal.GetIntArray().Value {
+ f := index.NewIntField(fieldKey, tagVal.GetIntArray().Value[i])
+ f.NoSort = noSort
+ dest = append(dest, f)
+ }
+ case databasev1.TagType_TAG_TYPE_STRING_ARRAY:
+ if tagVal.GetStrArray() == nil {
+ return dest
+ }
+ for i := range tagVal.GetStrArray().Value {
+ f := index.NewStringField(fieldKey, tagVal.GetStrArray().Value[i])
+ f.NoSort = noSort
+ dest = append(dest, f)
+ }
+ default:
+ logger.Panicf("unsupported tag value type: %T", tagVal.GetValue())
}
-
- l.l.Info().
- Str("group", group).
- Uint32("shardID", shardID).
- Int("documentCount", len(documents)).
- Msg("successfully inserted documents to tsTable index")
-
- return
+ return dest
}
diff --git a/dist/LICENSE b/dist/LICENSE
deleted file mode 100644
index 8e318023..00000000
--- a/dist/LICENSE
+++ /dev/null
@@ -1,562 +0,0 @@
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
-
-========================================================================
-Apache-2.0 licenses
-========================================================================
-
- github.com/SkyAPM/bluge v0.0.0-20250619030236-3750bbbf63b9 Apache-2.0
- github.com/SkyAPM/ice v0.0.0-20250619023539-b5173603b0b3 Apache-2.0
- github.com/apache/skywalking-cli v0.0.0-20240227151024-ee371a210afe Apache-2.0
- github.com/aws/aws-sdk-go-v2 v1.36.3 Apache-2.0
- github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.10 Apache-2.0
- github.com/aws/aws-sdk-go-v2/config v1.29.9 Apache-2.0
- github.com/aws/aws-sdk-go-v2/credentials v1.17.62 Apache-2.0
- github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 Apache-2.0
- github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34 Apache-2.0
- github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.34 Apache-2.0
- github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 Apache-2.0
- github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.34 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.3 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.7.0 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.15 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.15 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/s3 v1.78.2 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/sso v1.25.1 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/ssooidc v1.29.1 Apache-2.0
- github.com/aws/aws-sdk-go-v2/service/sts v1.33.17 Apache-2.0
- github.com/aws/smithy-go v1.22.2 Apache-2.0
- github.com/blevesearch/segment v0.9.1 Apache-2.0
- github.com/blevesearch/vellum v1.1.0 Apache-2.0
- github.com/containerd/continuity v0.4.5 Apache-2.0
- github.com/coreos/go-semver v0.3.1 Apache-2.0
- github.com/coreos/go-systemd/v22 v22.5.0 Apache-2.0
- github.com/docker/cli v27.4.1+incompatible Apache-2.0
- github.com/docker/docker v27.1.1+incompatible Apache-2.0
- github.com/docker/go-connections v0.5.0 Apache-2.0
- github.com/docker/go-units v0.5.0 Apache-2.0
- github.com/envoyproxy/protoc-gen-validate v1.2.1 Apache-2.0
- github.com/go-ini/ini v1.67.0 Apache-2.0
- github.com/go-logr/logr v1.4.2 Apache-2.0
- github.com/go-logr/stdr v1.2.2 Apache-2.0
- github.com/google/btree v1.1.3 Apache-2.0
- github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e Apache-2.0
- github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 Apache-2.0
- github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 Apache-2.0
- github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1 Apache-2.0
- github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.1 Apache-2.0
- github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 Apache-2.0
- github.com/inconshreveable/mousetrap v1.1.0 Apache-2.0
- github.com/jonboulle/clockwork v0.5.0 Apache-2.0
- github.com/machinebox/graphql v0.2.2 Apache-2.0
- github.com/minio/crc64nvme v1.0.1 Apache-2.0
- github.com/minio/md5-simd v1.1.2 Apache-2.0
- github.com/minio/minio-go/v7 v7.0.90 Apache-2.0
- github.com/moby/docker-image-spec v1.3.1 Apache-2.0
- github.com/moby/sys/user v0.3.0 Apache-2.0
- github.com/moby/term v0.5.0 Apache-2.0
- github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd Apache-2.0
- github.com/modern-go/reflect2 v1.0.2 Apache-2.0
- github.com/mschoch/smat v0.2.0 Apache-2.0
- github.com/oklog/run v1.1.0 Apache-2.0
- github.com/opencontainers/go-digest v1.0.0 Apache-2.0
- github.com/opencontainers/image-spec v1.1.0 Apache-2.0
- github.com/opencontainers/runc v1.2.3 Apache-2.0
- github.com/ory/dockertest/v3 v3.12.0 Apache-2.0
- github.com/prometheus/client_golang v1.21.1 Apache-2.0
- github.com/prometheus/client_model v0.6.1 Apache-2.0
- github.com/prometheus/common v0.63.0 Apache-2.0
- github.com/prometheus/procfs v0.16.0 Apache-2.0
- github.com/soheilhy/cmux v0.1.5 Apache-2.0
- github.com/spf13/afero v1.14.0 Apache-2.0
- github.com/spf13/cobra v1.9.1 Apache-2.0
- github.com/tklauser/numcpus v0.10.0 Apache-2.0
- github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb Apache-2.0
- github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 Apache-2.0
- github.com/xeipuuv/gojsonschema v1.2.0 Apache-2.0
- github.com/zinclabs/bluge_segment_api v1.0.0 Apache-2.0
- go.etcd.io/etcd/api/v3 v3.5.21 Apache-2.0
- go.etcd.io/etcd/client/pkg/v3 v3.5.21 Apache-2.0
- go.etcd.io/etcd/client/v2 v2.305.21 Apache-2.0
- go.etcd.io/etcd/client/v3 v3.5.21 Apache-2.0
- go.etcd.io/etcd/pkg/v3 v3.5.21 Apache-2.0
- go.etcd.io/etcd/raft/v3 v3.5.21 Apache-2.0
- go.etcd.io/etcd/server/v3 v3.5.21 Apache-2.0
- go.opentelemetry.io/auto/sdk v1.1.0 Apache-2.0
- go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 Apache-2.0
- go.opentelemetry.io/otel v1.35.0 Apache-2.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 Apache-2.0
- go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.35.0 Apache-2.0
- go.opentelemetry.io/otel/metric v1.35.0 Apache-2.0
- go.opentelemetry.io/otel/sdk v1.35.0 Apache-2.0
- go.opentelemetry.io/otel/trace v1.35.0 Apache-2.0
- go.opentelemetry.io/proto/otlp v1.5.0 Apache-2.0
- go.uber.org/mock v0.5.0 Apache-2.0
- google.golang.org/genproto v0.0.0-20250324211829-b45e905df463 Apache-2.0
- google.golang.org/genproto/googleapis/api v0.0.0-20250324211829-b45e905df463 Apache-2.0
- google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 Apache-2.0
- google.golang.org/grpc v1.71.0 Apache-2.0
- gopkg.in/yaml.v2 v2.4.0 Apache-2.0
- skywalking.apache.org/repo/goapi v0.0.0-20250225130248-3916480eb467 Apache-2.0
-
-========================================================================
-Apache-2.0 and BSD-3-Clause licenses
-========================================================================
-
- github.com/RoaringBitmap/roaring v1.9.4 Apache-2.0 and BSD-3-Clause
-
-========================================================================
-BSD-2-Clause licenses
-========================================================================
-
- github.com/gorilla/websocket v1.5.3 BSD-2-Clause
- github.com/kamstrup/intmap v0.5.1 BSD-2-Clause
- github.com/pkg/errors v0.9.1 BSD-2-Clause
- github.com/russross/blackfriday/v2 v2.1.0 BSD-2-Clause
-
-========================================================================
-BSD-2-Clause and ISC licenses
-========================================================================
-
- github.com/emirpasic/gods v1.18.1 BSD-2-Clause and ISC
-
-========================================================================
-BSD-2-Clause-Views licenses
-========================================================================
-
- github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 BSD-2-Clause-Views
-
-========================================================================
-BSD-3-Clause licenses
-========================================================================
-
- dario.cat/mergo v1.0.0 BSD-3-Clause
- github.com/bits-and-blooms/bitset v1.22.0 BSD-3-Clause
- github.com/blevesearch/mmap-go v1.0.4 BSD-3-Clause
- github.com/blevesearch/snowballstem v0.9.0 BSD-3-Clause
- github.com/fsnotify/fsnotify v1.9.0 BSD-3-Clause
- github.com/gogo/protobuf v1.3.2 BSD-3-Clause
- github.com/golang/protobuf v1.5.4 BSD-3-Clause
- github.com/golang/snappy v1.0.0 BSD-3-Clause
- github.com/google/go-cmp v0.7.0 BSD-3-Clause
- github.com/google/uuid v1.6.0 BSD-3-Clause
- github.com/grpc-ecosystem/grpc-gateway v1.16.0 BSD-3-Clause
- github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 BSD-3-Clause
- github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 BSD-3-Clause
- github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 BSD-3-Clause
- github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 BSD-3-Clause
- github.com/shirou/gopsutil/v3 v3.24.5 BSD-3-Clause
- github.com/spf13/pflag v1.0.6 BSD-3-Clause
- github.com/tklauser/go-sysconf v0.3.15 BSD-3-Clause
- github.com/xhit/go-str2duration/v2 v2.1.0 BSD-3-Clause
- golang.org/x/crypto v0.36.0 BSD-3-Clause
- golang.org/x/exp v0.0.0-20250305212735-054e65f0b394 BSD-3-Clause
- golang.org/x/mod v0.24.0 BSD-3-Clause
- golang.org/x/net v0.38.0 BSD-3-Clause
- golang.org/x/sys v0.31.0 BSD-3-Clause
- golang.org/x/text v0.23.0 BSD-3-Clause
- golang.org/x/time v0.11.0 BSD-3-Clause
- golang.org/x/tools v0.31.0 BSD-3-Clause
- google.golang.org/protobuf v1.36.6 BSD-3-Clause
-
-========================================================================
-BSD-3-Clause and Apache-2.0 and MIT licenses
-========================================================================
-
- github.com/klauspost/compress v1.18.0 BSD-3-Clause and Apache-2.0 and MIT
-
-========================================================================
-ISC licenses
-========================================================================
-
- github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc ISC
-
-========================================================================
-MIT licenses
-========================================================================
-
- github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 MIT
- github.com/Microsoft/go-winio v0.6.2 MIT
- github.com/SkyAPM/clock v1.3.1-0.20220809233656-dc7607c94a97 MIT
- github.com/VictoriaMetrics/fastcache v1.12.2 MIT
- github.com/axiomhq/hyperloglog v0.2.5 MIT
- github.com/beorn7/perks v1.0.1 MIT
- github.com/blevesearch/go-porterstemmer v1.0.3 MIT
- github.com/caio/go-tdigest v3.1.0+incompatible MIT
- github.com/cenkalti/backoff/v4 v4.3.0 MIT
- github.com/cespare/xxhash/v2 v2.3.0 MIT
- github.com/cpuguy83/go-md2man/v2 v2.0.6 MIT
- github.com/dgryski/go-metro v0.0.0-20250106013310-edb8663e5e33 MIT
- github.com/dustin/go-humanize v1.0.1 MIT
- github.com/go-chi/chi/v5 v5.2.1 MIT
- github.com/go-ole/go-ole v1.3.0 MIT
- github.com/go-resty/resty/v2 v2.16.5 MIT
- github.com/go-task/slim-sprig/v3 v3.0.0 MIT
- github.com/go-viper/mapstructure/v2 v2.2.1 MIT
- github.com/goccy/go-json v0.10.5 MIT
- github.com/golang-jwt/jwt/v4 v4.5.2 MIT
- github.com/json-iterator/go v1.1.12 MIT
- github.com/klauspost/cpuid/v2 v2.2.10 MIT
- github.com/mattn/go-colorable v0.1.14 MIT
- github.com/mattn/go-isatty v0.0.20 MIT
- github.com/montanaflynn/stats v0.7.1 MIT
- github.com/onsi/ginkgo/v2 v2.23.3 MIT
- github.com/onsi/gomega v1.36.3 MIT
- github.com/pelletier/go-toml/v2 v2.2.3 MIT
- github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 MIT
- github.com/robfig/cron/v3 v3.0.1 MIT
- github.com/rs/xid v1.6.0 MIT
- github.com/rs/zerolog v1.34.0 MIT
- github.com/sagikazarmark/locafero v0.9.0 MIT
- github.com/sirupsen/logrus v1.9.3 MIT
- github.com/sourcegraph/conc v0.3.0 MIT
- github.com/spf13/cast v1.7.1 MIT
- github.com/spf13/viper v1.20.1 MIT
- github.com/stretchr/testify v1.10.0 MIT
- github.com/subosito/gotenv v1.6.0 MIT
- github.com/tmc/grpc-websocket-proxy v0.0.0-20220101234140-673ab2c3ae75 MIT
- github.com/urfave/cli/v2 v2.27.6 MIT
- github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510 MIT
- github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 MIT
- github.com/yusufpapurcu/wmi v1.2.4 MIT
- github.com/zenizh/go-capturer v0.0.0-20211219060012-52ea6c8fed04 MIT
- go.etcd.io/bbolt v1.4.0 MIT
- go.uber.org/automaxprocs v1.6.0 MIT
- go.uber.org/multierr v1.11.0 MIT
- go.uber.org/zap v1.27.0 MIT
- gopkg.in/natefinch/lumberjack.v2 v2.2.1 MIT
-
-========================================================================
-MIT and Apache-2.0 licenses
-========================================================================
-
- gopkg.in/yaml.v3 v3.0.1 MIT and Apache-2.0
-
-========================================================================
-MIT and BSD-3-Clause and Apache-2.0 licenses
-========================================================================
-
- sigs.k8s.io/yaml v1.4.0 MIT and BSD-3-Clause and Apache-2.0
-
-========================================================================
-MPL-2.0 licenses
-========================================================================
-
- github.com/hashicorp/golang-lru v1.0.2 MPL-2.0
- github.com/shoenig/go-m1cpu v0.1.6 MPL-2.0
-
-========================================================================
-UI related licenses
-========================================================================
-
-The following components are used in UI.See project link for details.
-The text of each license is also included at licenses/ui-licenses/LICENSE-[project].txt.
-
-
-========================================================================
-0BSD licenses
-========================================================================
-
- tslib 2.3.0 0BSD
-
-========================================================================
-Apache-2.0 licenses
-========================================================================
-
- detect-libc 1.0.3 Apache-2.0
- echarts 5.6.0 Apache-2.0
-
-========================================================================
-BSD-2-Clause licenses
-========================================================================
-
- entities 4.5.0 BSD-2-Clause
-
-========================================================================
-BSD-3-Clause licenses
-========================================================================
-
- normalize-wheel-es 1.2.0 BSD-3-Clause
- source-map-js 1.2.1 BSD-3-Clause
- zrender 5.6.1 BSD-3-Clause
-
-========================================================================
-ISC licenses
-========================================================================
-
- graceful-fs 4.2.11 ISC
- picocolors 1.1.1 ISC
- semver 7.7.1 ISC
- yaml 2.7.1 ISC
-
-========================================================================
-MIT licenses
-========================================================================
-
- @babel/helper-string-parser 7.25.9 MIT
- @babel/helper-validator-identifier 7.25.9 MIT
- @babel/parser 7.27.0 MIT
- @babel/types 7.27.0 MIT
- @ctrl/tinycolor 3.6.1 MIT
- @element-plus/icons-vue 2.3.1 MIT
- @floating-ui/core 1.6.9 MIT
- @floating-ui/dom 1.6.13 MIT
- @floating-ui/utils 0.2.9 MIT
- @jridgewell/sourcemap-codec 1.5.0 MIT
- @parcel/watcher 2.5.1 MIT
- @parcel/watcher-linux-x64-glibc 2.5.1 MIT
- @parcel/watcher-linux-x64-musl 2.5.1 MIT
- @popperjs/core 2.11.7 MIT
- @types/lodash 4.17.16 MIT
- @types/lodash-es 4.17.12 MIT
- @types/web-bluetooth 0.0.16 MIT
- @vue/compiler-core 3.5.13 MIT
- @vue/compiler-dom 3.5.13 MIT
- @vue/compiler-sfc 3.5.13 MIT
- @vue/compiler-ssr 3.5.13 MIT
- @vue/devtools-api 6.6.4 MIT
- @vue/reactivity 3.5.13 MIT
- @vue/runtime-core 3.5.13 MIT
- @vue/runtime-dom 3.5.13 MIT
- @vue/server-renderer 3.5.13 MIT
- @vue/shared 3.5.13 MIT
- @vueuse/core 9.13.0 MIT
- @vueuse/metadata 9.13.0 MIT
- @vueuse/shared 9.13.0 MIT
- async-validator 4.2.5 MIT
- asynckit 0.4.0 MIT
- axios 1.8.4 MIT
- braces 3.0.3 MIT
- call-bind-apply-helpers 1.0.2 MIT
- chokidar 4.0.3 MIT
- codemirror 5.65.19 MIT
- combined-stream 1.0.8 MIT
- csstype 3.1.3 MIT
- dayjs 1.11.13 MIT
- delayed-stream 1.0.0 MIT
- dunder-proto 1.0.1 MIT
- element-plus 2.9.7 MIT
- es-define-property 1.0.1 MIT
- es-errors 1.3.0 MIT
- es-object-atoms 1.1.1 MIT
- es-set-tostringtag 2.1.0 MIT
- escape-html 1.0.3 MIT
- estree-walker 2.0.2 MIT
- fill-range 7.1.1 MIT
- follow-redirects 1.15.9 MIT
- form-data 4.0.2 MIT
- function-bind 1.1.2 MIT
- get-intrinsic 1.3.0 MIT
- get-proto 1.0.1 MIT
- gopd 1.2.0 MIT
- has-symbols 1.1.0 MIT
- has-tostringtag 1.0.2 MIT
- hasown 2.0.2 MIT
- immutable 5.1.1 MIT
- is-extglob 2.1.1 MIT
- is-glob 4.0.3 MIT
- is-number 7.0.0 MIT
- js-yaml 4.1.0 MIT
- jsonfile 6.1.0 MIT
- lodash 4.17.21 MIT
- lodash-es 4.17.21 MIT
- lodash-unified 1.0.3 MIT
- magic-string 0.30.17 MIT
- math-intrinsics 1.1.0 MIT
- memoize-one 6.0.0 MIT
- micromatch 4.0.8 MIT
- mime-db 1.52.0 MIT
- mime-types 2.1.35 MIT
- mitt 3.0.1 MIT
- nanoid 3.3.11 MIT
- node-addon-api 7.1.1 MIT
- picomatch 2.3.1 MIT
- pinia 2.3.1 MIT
- postcss 8.5.3 MIT
- proxy-from-env 1.1.0 MIT
- readdirp 4.1.2 MIT
- sass 1.86.1 MIT
- to-regex-range 5.0.1 MIT
- universalify 2.0.1 MIT
- vue 3.5.13 MIT
- vue-demi 0.14.10 MIT
- vue-router 4.5.0 MIT
-
-========================================================================
-Python-2.0 licenses
-========================================================================
-
- argparse 2.0.1 Python-2.0
diff --git a/dist/LICENSE.tpl b/dist/LICENSE.tpl
deleted file mode 100644
index 2f1c6797..00000000
--- a/dist/LICENSE.tpl
+++ /dev/null
@@ -1,9 +0,0 @@
-{{.LicenseContent }}
-{{ range .Groups }}
-========================================================================
-{{.LicenseID}} licenses
-========================================================================
-{{range .Deps}}
- {{.Name}} {{.Version}} {{.LicenseID}}
-{{- end }}
-{{ end }}
diff --git a/dist/NOTICE b/dist/NOTICE
deleted file mode 100644
index 04d6ba5b..00000000
--- a/dist/NOTICE
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache SkyWalking
-Copyright 2017-2024 The Apache Software Foundation
-
-This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
diff --git a/dist/licenses/license-dario.cat-mergo.txt b/dist/licenses/license-dario.cat-mergo.txt
deleted file mode 100644
index 68668029..00000000
--- a/dist/licenses/license-dario.cat-mergo.txt
+++ /dev/null
@@ -1,28 +0,0 @@
-Copyright (c) 2013 Dario Castañé. All rights reserved.
-Copyright (c) 2012 The Go Authors. All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- * Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
- * Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/dist/licenses/license-github.com-Azure-go-ansiterm.txt b/dist/licenses/license-github.com-Azure-go-ansiterm.txt
deleted file mode 100644
index e3d9a64d..00000000
--- a/dist/licenses/license-github.com-Azure-go-ansiterm.txt
+++ /dev/null
@@ -1,21 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2015 Microsoft Corporation
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
diff --git a/dist/licenses/license-github.com-Microsoft-go-winio.txt b/dist/licenses/license-github.com-Microsoft-go-winio.txt
deleted file mode 100644
index b8b569d7..00000000
--- a/dist/licenses/license-github.com-Microsoft-go-winio.txt
+++ /dev/null
@@ -1,22 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2015 Microsoft
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
diff --git a/dist/licenses/license-github.com-Nvveen-Gotty.txt b/dist/licenses/license-github.com-Nvveen-Gotty.txt
deleted file mode 100644
index 0b71c973..00000000
--- a/dist/licenses/license-github.com-Nvveen-Gotty.txt
+++ /dev/null
@@ -1,26 +0,0 @@
-Copyright (c) 2012, Neal van Veen (nealvanveen@gmail.com)
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-
-1. Redistributions of source code must retain the above copyright notice, this
- list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice,
- this list of conditions and the following disclaimer in the documentation
- and/or other materials provided with the distribution.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
-ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-The views and conclusions contained in the software and documentation are those
-of the authors and should not be interpreted as representing official policies,
-either expressed or implied, of the FreeBSD Project.
diff --git a/dist/licenses/license-github.com-RoaringBitmap-roaring.txt b/dist/licenses/license-github.com-RoaringBitmap-roaring.txt
deleted file mode 100644
index 3ccdd000..00000000
--- a/dist/licenses/license-github.com-RoaringBitmap-roaring.txt
+++ /dev/null
@@ -1,235 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright 2016 by the authors
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-================================================================================
-
-Portions of runcontainer.go are from the Go standard library, which is licensed
-under:
-
-Copyright (c) 2009 The Go Authors. All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
- copyright notice, this list of conditions and the following disclaimer
- in the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Google Inc. nor the names of its
- contributors may be used to endorse or promote products derived from
- this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/dist/licenses/license-github.com-SkyAPM-bluge.txt b/dist/licenses/license-github.com-SkyAPM-bluge.txt
deleted file mode 100644
index 7a4a3ea2..00000000
--- a/dist/licenses/license-github.com-SkyAPM-bluge.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
\ No newline at end of file
diff --git a/dist/licenses/license-github.com-SkyAPM-clock.txt b/dist/licenses/license-github.com-SkyAPM-clock.txt
deleted file mode 100644
index ce212cb1..00000000
--- a/dist/licenses/license-github.com-SkyAPM-clock.txt
+++ /dev/null
@@ -1,21 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2014 Ben Johnson
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
diff --git a/dist/licenses/license-github.com-SkyAPM-ice.txt b/dist/licenses/license-github.com-SkyAPM-ice.txt
deleted file mode 100644
index 7a4a3ea2..00000000
--- a/dist/licenses/license-github.com-SkyAPM-ice.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
\ No newline at end of file
diff --git a/dist/licenses/license-github.com-VictoriaMetrics-fastcache.txt b/dist/licenses/license-github.com-VictoriaMetrics-fastcache.txt
deleted file mode 100644
index 9a8145e5..00000000
--- a/dist/licenses/license-github.com-VictoriaMetrics-fastcache.txt
+++ /dev/null
@@ -1,22 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2018 VictoriaMetrics
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
diff --git a/dist/licenses/license-github.com-apache-skywalking-cli.txt b/dist/licenses/license-github.com-apache-skywalking-cli.txt
deleted file mode 100644
index 5a80c2b5..00000000
--- a/dist/licenses/license-github.com-apache-skywalking-cli.txt
+++ /dev/null
@@ -1,210 +0,0 @@
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright {yyyy} {name of copyright owner}
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-========================================================================
-Apache 2.0 licenses
-========================================================================
-
-The following components are provided under the Apache License. See project link for details.
-The text of each license is the standard Apache 2.0 license.
-
- pkg/display/graph/flamegraph files from jvm-profiling-tools/async-profiler: https://github.com/jvm-profiling-tools/async-profiler Apache-2.0
\ No newline at end of file
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-aws-protocol-eventstream.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-aws-protocol-eventstream.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-aws-protocol-eventstream.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-config.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-config.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-config.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-credentials.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-credentials.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-credentials.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-feature-ec2-imds.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-feature-ec2-imds.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-feature-ec2-imds.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-configsources.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-configsources.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-configsources.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-endpoints-v2.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-endpoints-v2.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-endpoints-v2.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-ini.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-ini.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-ini.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-v4a.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-v4a.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-internal-v4a.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-accept-encoding.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-accept-encoding.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-accept-encoding.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-checksum.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-checksum.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-checksum.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-presigned-url.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-presigned-url.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-presigned-url.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-s3shared.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-s3shared.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-internal-s3shared.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-s3.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-s3.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-s3.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
diff --git a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-sso.txt b/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-sso.txt
deleted file mode 100644
index d6456956..00000000
--- a/dist/licenses/license-github.com-aws-aws-sdk-go-v2-service-sso.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
... 24542 lines suppressed ...