You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by ha...@apache.org on 2026/04/26 09:16:59 UTC
(skywalking-banyandb-client-proto) 01/02: Sync schema-consistency Phase 1 from BanyanDB v0.11.0
This is an automated email from the ASF dual-hosted git repository.
hanahmily pushed a commit to branch schema-consistency-phase-1
in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb-client-proto.git
commit 10c12e8218f49e5e4691b931b53e176f8927c90a
Author: Hongtao Gao <ha...@gmail.com>
AuthorDate: Sun Apr 26 08:14:05 2026 +0000
Sync schema-consistency Phase 1 from BanyanDB v0.11.0
Mirror the client-facing proto surface introduced by
apache/skywalking-banyandb#1091 so OAP and other clients can consume the
new revision/propagation primitives.
Field tag numbers and enum values match the server side exactly to
preserve wire compatibility; layout differences between this repo's
flat files and the server's nested tree do not affect serialization.
Response surface (banyandb-database.proto, banyandb-common.proto):
- created_at on Stream (=5), Measure (=9), Trace (=7), Property (=7),
IndexRule (=7), IndexRuleBinding (=7), TopNAggregation (=10), Group (=5).
Stamped on Create, preserved across Update.
- mod_revision on the four "thin" Create/Update responses
(IndexRule, IndexRuleBinding, Group, TopNAggregation; tag 1).
- delete_time (=2) and mod_revision (=3) on every *ServiceDeleteResponse
(Stream, Measure, Trace, IndexRule, IndexRuleBinding, Group,
TopNAggregation, Property).
Status code (banyandb-model.proto):
- STATUS_SCHEMA_NOT_APPLIED = 10. Returned when the client's
mod_revision is ahead of the server cache.
Query gate (banyandb-stream.proto, banyandb-measure.proto, banyandb-trace.proto):
- QueryRequest.group_mod_revisions (map<string,int64>, tag 50): per-group
three-way ModRevision split. Empty map preserves prior behavior.
- QueryResponse.group_statuses (map<string,model.v1.Status>, tag 50):
per-group gate outcome.
Schema barriers (NEW banyandb-schema.proto):
- SchemaBarrierService with AwaitRevisionApplied, AwaitSchemaApplied,
AwaitSchemaDeleted plus SchemaKey / NodeLaggard supporting messages.
10 000-key cap server-side; timeout-bounded; returns laggards on expiry.
Skipped (not client-facing):
- cluster/v1/node_schema_status.proto — internal liaison↔data-node RPC
for Phase 2 cluster fan-out only.
Migration note: Java OAP client should treat STATUS_SCHEMA_NOT_APPLIED
as STATUS_EXPIRED_SCHEMA until the client-side enum handling is updated.
Refs: apache/skywalking-banyandb#1091
via [HAPI](https://hapi.run)
---
proto/banyandb/v1/banyandb-common.proto | 2 +
proto/banyandb/v1/banyandb-database.proto | 86 ++++++++++++++++++++++---
proto/banyandb/v1/banyandb-measure.proto | 7 ++
proto/banyandb/v1/banyandb-model.proto | 1 +
proto/banyandb/v1/banyandb-schema.proto | 103 ++++++++++++++++++++++++++++++
proto/banyandb/v1/banyandb-stream.proto | 7 ++
proto/banyandb/v1/banyandb-trace.proto | 7 ++
7 files changed, 205 insertions(+), 8 deletions(-)
diff --git a/proto/banyandb/v1/banyandb-common.proto b/proto/banyandb/v1/banyandb-common.proto
index d4ace04..52ae8cc 100644
--- a/proto/banyandb/v1/banyandb-common.proto
+++ b/proto/banyandb/v1/banyandb-common.proto
@@ -118,6 +118,8 @@ message Group {
ResourceOpts resource_opts = 3;
// updated_at indicates when resources of the group are updated
google.protobuf.Timestamp updated_at = 4;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 5;
}
// APIVersion is the version of the API
diff --git a/proto/banyandb/v1/banyandb-database.proto b/proto/banyandb/v1/banyandb-database.proto
index bd8242a..7d88881 100644
--- a/proto/banyandb/v1/banyandb-database.proto
+++ b/proto/banyandb/v1/banyandb-database.proto
@@ -57,6 +57,8 @@ message Stream {
Entity entity = 3 [(validate.rules).message.required = true];
// updated_at indicates when the stream is updated
google.protobuf.Timestamp updated_at = 4;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 5;
}
message Entity {
@@ -117,6 +119,8 @@ message Measure {
bool index_mode = 7;
// sharding_key determines the distribution of TopN-related data.
ShardingKey sharding_key = 8;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 9;
}
// TopNAggregation generates offline TopN statistics for a measure's TopN approximation
@@ -143,6 +147,8 @@ message TopNAggregation {
int32 lru_size = 8;
// updated_at indicates when the measure is updated
google.protobuf.Timestamp updated_at = 9;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 10;
}
// IndexRule defines how to generate indices based on tags and the index type
@@ -178,6 +184,8 @@ message IndexRule {
string analyzer = 5;
// no_sort indicates whether the index is not for sorting.
bool no_sort = 6;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 7;
}
// Subject defines which stream or measure would generate indices
@@ -206,6 +214,8 @@ message IndexRuleBinding {
google.protobuf.Timestamp expire_at = 5 [(validate.rules).timestamp.required = true];
// updated_at indicates when the IndexRuleBinding is updated
google.protobuf.Timestamp updated_at = 6;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 7;
}
// Property stores the user defined data
@@ -216,6 +226,8 @@ message Property {
repeated TagSpec tags = 2 [(validate.rules).repeated.min_items = 1];
// updated_at indicates when the property is updated
google.protobuf.Timestamp updated_at = 6;
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 7;
}
// TraceTagSpec defines the specification of a tag in a trace.
@@ -243,6 +255,8 @@ message Trace {
google.protobuf.Timestamp updated_at = 5;
// span_id_tag_name is the name of the tag that stores the span ID.
string span_id_tag_name = 6 [(validate.rules).string.min_len = 1];
+ // created_at is the first-appearance timestamp; survives updates unchanged.
+ google.protobuf.Timestamp created_at = 7;
}
message StreamRegistryServiceCreateRequest {
@@ -267,6 +281,10 @@ message StreamRegistryServiceDeleteRequest {
message StreamRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message StreamRegistryServiceGetRequest {
@@ -311,13 +329,19 @@ message IndexRuleBindingRegistryServiceCreateRequest {
banyandb.database.v1.IndexRuleBinding index_rule_binding = 1;
}
-message IndexRuleBindingRegistryServiceCreateResponse {}
+message IndexRuleBindingRegistryServiceCreateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful create.
+ int64 mod_revision = 1;
+}
message IndexRuleBindingRegistryServiceUpdateRequest {
banyandb.database.v1.IndexRuleBinding index_rule_binding = 1;
}
-message IndexRuleBindingRegistryServiceUpdateResponse {}
+message IndexRuleBindingRegistryServiceUpdateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful update.
+ int64 mod_revision = 1;
+}
message IndexRuleBindingRegistryServiceDeleteRequest {
banyandb.common.v1.Metadata metadata = 1;
@@ -325,6 +349,10 @@ message IndexRuleBindingRegistryServiceDeleteRequest {
message IndexRuleBindingRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message IndexRuleBindingRegistryServiceGetRequest {
@@ -369,13 +397,19 @@ message IndexRuleRegistryServiceCreateRequest {
banyandb.database.v1.IndexRule index_rule = 1;
}
-message IndexRuleRegistryServiceCreateResponse {}
+message IndexRuleRegistryServiceCreateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful create.
+ int64 mod_revision = 1;
+}
message IndexRuleRegistryServiceUpdateRequest {
banyandb.database.v1.IndexRule index_rule = 1;
}
-message IndexRuleRegistryServiceUpdateResponse {}
+message IndexRuleRegistryServiceUpdateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful update.
+ int64 mod_revision = 1;
+}
message IndexRuleRegistryServiceDeleteRequest {
banyandb.common.v1.Metadata metadata = 1;
@@ -383,6 +417,10 @@ message IndexRuleRegistryServiceDeleteRequest {
message IndexRuleRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message IndexRuleRegistryServiceGetRequest {
@@ -445,6 +483,10 @@ message MeasureRegistryServiceDeleteRequest {
message MeasureRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message MeasureRegistryServiceGetRequest {
@@ -489,13 +531,19 @@ message GroupRegistryServiceCreateRequest {
banyandb.common.v1.Group group = 1;
}
-message GroupRegistryServiceCreateResponse {}
+message GroupRegistryServiceCreateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful create.
+ int64 mod_revision = 1;
+}
message GroupRegistryServiceUpdateRequest {
banyandb.common.v1.Group group = 1;
}
-message GroupRegistryServiceUpdateResponse {}
+message GroupRegistryServiceUpdateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful update.
+ int64 mod_revision = 1;
+}
message GroupRegistryServiceDeleteRequest {
string group = 1;
@@ -503,6 +551,10 @@ message GroupRegistryServiceDeleteRequest {
message GroupRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message GroupRegistryServiceGetRequest {
@@ -544,13 +596,19 @@ message TopNAggregationRegistryServiceCreateRequest {
banyandb.database.v1.TopNAggregation top_n_aggregation = 1;
}
-message TopNAggregationRegistryServiceCreateResponse {}
+message TopNAggregationRegistryServiceCreateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful create.
+ int64 mod_revision = 1;
+}
message TopNAggregationRegistryServiceUpdateRequest {
banyandb.database.v1.TopNAggregation top_n_aggregation = 1;
}
-message TopNAggregationRegistryServiceUpdateResponse {}
+message TopNAggregationRegistryServiceUpdateResponse {
+ // mod_revision is the etcd revision assigned by the server on successful update.
+ int64 mod_revision = 1;
+}
message TopNAggregationRegistryServiceDeleteRequest {
banyandb.common.v1.Metadata metadata = 1;
@@ -558,6 +616,10 @@ message TopNAggregationRegistryServiceDeleteRequest {
message TopNAggregationRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message TopNAggregationRegistryServiceGetRequest {
@@ -639,6 +701,10 @@ message PropertyRegistryServiceDeleteRequest {
message PropertyRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message PropertyRegistryServiceGetRequest {
@@ -701,6 +767,10 @@ message TraceRegistryServiceDeleteRequest {
message TraceRegistryServiceDeleteResponse {
bool deleted = 1;
+ // delete_time is the server-assigned tombstone timestamp in unix nanos.
+ int64 delete_time = 2;
+ // mod_revision is the etcd revision of the tombstone; zero if the server did not record one.
+ int64 mod_revision = 3;
}
message TraceRegistryServiceGetRequest {
diff --git a/proto/banyandb/v1/banyandb-measure.proto b/proto/banyandb/v1/banyandb-measure.proto
index f8c4f91..8d22713 100644
--- a/proto/banyandb/v1/banyandb-measure.proto
+++ b/proto/banyandb/v1/banyandb-measure.proto
@@ -51,6 +51,9 @@ message QueryResponse {
repeated DataPoint data_points = 1;
// trace contains the trace information of the query when trace is enabled
common.v1.Trace trace = 2;
+ // group_statuses reports the per-group gate outcome. Populated even when the query
+ // short-circuits because any group failed the gate.
+ map<string, model.v1.Status> group_statuses = 50;
}
// QueryRequest is the request contract for query.
@@ -113,6 +116,10 @@ message QueryRequest {
repeated string stages = 14;
// rewriteAggTopNResult will rewrite agg result to raw data
bool rewrite_agg_top_n_result = 15;
+ // group_mod_revisions gates the query per group. Keys match entries in `groups`;
+ // values are the client's known mod_revision for that group. Empty map or value 0
+ // means "don't gate". A group not listed in the map is not gated.
+ map<string, int64> group_mod_revisions = 50;
}
diff --git a/proto/banyandb/v1/banyandb-model.proto b/proto/banyandb/v1/banyandb-model.proto
index e6066fc..b7d6f32 100644
--- a/proto/banyandb/v1/banyandb-model.proto
+++ b/proto/banyandb/v1/banyandb-model.proto
@@ -195,4 +195,5 @@ enum Status {
STATUS_VERSION_UNSUPPORTED = 7; // Client version not supported
STATUS_VERSION_DEPRECATED = 8; // Client version deprecated but still supported
STATUS_METADATA_REQUIRED = 9; // Metadata is required for the first request
+ STATUS_SCHEMA_NOT_APPLIED = 10; // Client mod_revision is ahead of the server cache; the schema mutation has not yet propagated
}
diff --git a/proto/banyandb/v1/banyandb-schema.proto b/proto/banyandb/v1/banyandb-schema.proto
new file mode 100644
index 0000000..edd0fff
--- /dev/null
+++ b/proto/banyandb/v1/banyandb-schema.proto
@@ -0,0 +1,103 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+syntax = "proto3";
+
+option java_package = "org.apache.skywalking.banyandb.schema.v1";
+
+package banyandb.schema.v1;
+
+import "google/protobuf/duration.proto";
+
+// SchemaBarrierService lets clients block until the cluster's schema state has
+// caught up to a target. Standalone deployments answer from the single
+// in-process cache; distributed deployments fan out to every data node.
+service SchemaBarrierService {
+ // AwaitRevisionApplied blocks until every data node's local schema cache has
+ // observed mod_revision >= min_revision, or the timeout elapses.
+ rpc AwaitRevisionApplied(AwaitRevisionAppliedRequest) returns (AwaitRevisionAppliedResponse);
+ // AwaitSchemaApplied blocks until every data node reports all requested keys
+ // present at or above the per-key mod_revision.
+ rpc AwaitSchemaApplied(AwaitSchemaAppliedRequest) returns (AwaitSchemaAppliedResponse);
+ // AwaitSchemaDeleted blocks until every data node has removed all requested
+ // keys from its cache.
+ rpc AwaitSchemaDeleted(AwaitSchemaDeletedRequest) returns (AwaitSchemaDeletedResponse);
+}
+
+// SchemaKey identifies a schema resource by kind and name. Valid kind values:
+// "measure", "stream", "trace", "property", "index_rule",
+// "index_rule_binding", "group", "top_n_aggregation".
+message SchemaKey {
+ string kind = 1;
+ string group = 2;
+ string name = 3;
+}
+
+// NodeLaggard reports a single data node that has not caught up to the
+// requested schema state. missing_keys is populated by AwaitSchemaApplied
+// responses; still_present_keys is populated by AwaitSchemaDeleted responses.
+message NodeLaggard {
+ string node = 1;
+ int64 current_mod_revision = 2;
+ repeated SchemaKey missing_keys = 3;
+ repeated SchemaKey still_present_keys = 4;
+}
+
+// AwaitRevisionAppliedRequest carries the minimum mod_revision the caller is
+// waiting for and a wall-clock budget for the wait.
+message AwaitRevisionAppliedRequest {
+ int64 min_revision = 1;
+ google.protobuf.Duration timeout = 2;
+}
+
+// AwaitRevisionAppliedResponse reports whether every node reached the target
+// revision before the timeout, and lists laggards otherwise.
+message AwaitRevisionAppliedResponse {
+ bool applied = 1;
+ repeated NodeLaggard laggards = 2;
+}
+
+// AwaitSchemaAppliedRequest pairs each key with a per-key minimum revision.
+// keys is capped at 10000 server-side; exceeding the cap returns
+// InvalidArgument. A min_revisions entry of 0 means "just present, any
+// revision".
+message AwaitSchemaAppliedRequest {
+ repeated SchemaKey keys = 1;
+ repeated int64 min_revisions = 2;
+ google.protobuf.Duration timeout = 3;
+}
+
+// AwaitSchemaAppliedResponse reports whether every requested key is present at
+// or above its target revision on every node.
+message AwaitSchemaAppliedResponse {
+ bool applied = 1;
+ repeated NodeLaggard laggards = 2;
+}
+
+// AwaitSchemaDeletedRequest names the keys that must disappear from every
+// node's cache before the call returns.
+message AwaitSchemaDeletedRequest {
+ repeated SchemaKey keys = 1;
+ google.protobuf.Duration timeout = 2;
+}
+
+// AwaitSchemaDeletedResponse reports whether every node has removed the
+// requested keys.
+message AwaitSchemaDeletedResponse {
+ bool applied = 1;
+ repeated NodeLaggard laggards = 2;
+}
diff --git a/proto/banyandb/v1/banyandb-stream.proto b/proto/banyandb/v1/banyandb-stream.proto
index 642b189..f689ccd 100644
--- a/proto/banyandb/v1/banyandb-stream.proto
+++ b/proto/banyandb/v1/banyandb-stream.proto
@@ -51,6 +51,9 @@ message QueryResponse {
repeated Element elements = 1;
// trace contains the trace information of the query when trace is enabled
common.v1.Trace trace = 2;
+ // group_statuses reports the per-group gate outcome. Populated even when the query
+ // short-circuits because any group failed the gate.
+ map<string, model.v1.Status> group_statuses = 50;
}
// QueryRequest is the request contract for query.
@@ -78,6 +81,10 @@ message QueryRequest {
bool trace = 9;
// stage is used to specify the stage of the query in the lifecycle
repeated string stages = 10;
+ // group_mod_revisions gates the query per group. Keys match entries in `groups`;
+ // values are the client's known mod_revision for that group. Empty map or value 0
+ // means "don't gate". A group not listed in the map is not gated.
+ map<string, int64> group_mod_revisions = 50;
}
diff --git a/proto/banyandb/v1/banyandb-trace.proto b/proto/banyandb/v1/banyandb-trace.proto
index b822b59..a05fa28 100644
--- a/proto/banyandb/v1/banyandb-trace.proto
+++ b/proto/banyandb/v1/banyandb-trace.proto
@@ -59,6 +59,9 @@ message QueryResponse {
repeated Trace traces = 1;
// trace_query_result contains the trace of the query execution if tracing is enabled.
common.v1.Trace trace_query_result = 2;
+ // group_statuses reports the per-group gate outcome. Populated even when the query
+ // short-circuits because any group failed the gate.
+ map<string, model.v1.Status> group_statuses = 50;
}
// InternalQueryResponse is the response of an internal query.
@@ -93,6 +96,10 @@ message QueryRequest {
bool trace = 9;
// stage is used to specify the stage of the query in the lifecycle
repeated string stages = 10;
+ // group_mod_revisions gates the query per group. Keys match entries in `groups`;
+ // values are the client's known mod_revision for that group. Empty map or value 0
+ // means "don't gate". A group not listed in the map is not gated.
+ map<string, int64> group_mod_revisions = 50;
}