You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/11/09 06:59:20 UTC

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 opened a new pull request #4: Support metadata registry api

lujiajing1126 opened a new pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4


   This PR intends to add support for schema management api for the client.
   
   Measure related APIs are not included so far, still wait for the upcoming PR https://github.com/apache/skywalking-banyandb/pull/57
   
   @hanahmily @wu-sheng Request for CR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750025152



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       @lujiajing1126 Would you pls link this doc here. I will polish it in the future. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749047765



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,
+
+```java
+IndexRuleBindingMetadataRegistry indexRuleBindingRegistry = client.indexRuleBindingRegistry();
+// define the rule binding "sw-index-rule-binding" and bind it with Stream "sw"
+IndexRuleBinding indexRuleBinding = new IndexRuleBinding("sw-index-rule-binding", IndexRuleBinding.Subject.referToStream("sw"));
+indexRuleBinding.setBeginAt(ZonedDateTime.now().minusDays(15));
+indexRuleBinding.setExpireAt(ZonedDateTime.now().plusYears(100));
+indexRuleBinding.addRule("db.instance");
+// create the index rule binding
+this.client.create(indexRuleBinding);
+```
+
+### Measure
+
+`Measure` can also be created with `MeasureMetadataRegistry`,
+
+```java
+// create a measure registry
+MeasureMetadataRegistry measureRegistry = client.measureRegistry();
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));
+// add field spec
+// compressMethod and encodingMethod can be specified
+m.addFieldSpec(Measure.FieldSpec.newIntField("tps").compressWithZSTD().encodeWithGorilla().build());
+// send create request
+measureRegistry.create(m);

Review comment:
       The APIs are not consistent. You don't have `indexRuleRegistry#create`, but `measureRegistry#create`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r751281932



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.

Review comment:
       Fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746292142



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       > Wait a moment, I can see stream is created from client, what is the issue of using client group?
   
   I suppose @hanahmily was referring to the `GroupRegistry`. You can CRUD groups other than that one set for the client.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746266425



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       The group is specified while creating the client,
   
   ```java
   // use `default` group
   client = new BanyanDBClient("127.0.0.1", 17912, "default");
   ```
   
   So the following schema management operations are automatically bounded to the group "default", except group related APIs.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746436533



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       If we make java-client as a thin client, removing group APIs sounds good to me.
   
   @lujiajing1126 Please remove `GroupRegistry` from this, then add a `default` group in the prerun phase of the `metadata` module of banyand once Etcd server is ready to serve.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746297353



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Server usually will provide different username nd password for admin client for keeping security.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng merged pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng merged pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749045194



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,
+
+```java
+IndexRuleBindingMetadataRegistry indexRuleBindingRegistry = client.indexRuleBindingRegistry();
+// define the rule binding "sw-index-rule-binding" and bind it with Stream "sw"
+IndexRuleBinding indexRuleBinding = new IndexRuleBinding("sw-index-rule-binding", IndexRuleBinding.Subject.referToStream("sw"));
+indexRuleBinding.setBeginAt(ZonedDateTime.now().minusDays(15));
+indexRuleBinding.setExpireAt(ZonedDateTime.now().plusYears(100));

Review comment:
       Why does binding have expired period?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-968486468


   > > > @hanahmily Shall we include Measure registry APIs in this PR?
   > > 
   > > 
   > > Absolutely.
   > 
   > Measure related APIs are added
   
   I think we need document updated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-968467352


   > > @hanahmily Shall we include Measure registry APIs in this PR?
   > 
   > Absolutely.
   
   Measure related APIs are added


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750784668



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.

Review comment:
       What is this `where` referring?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750020969



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       there would be more than one binding for a rule under a rolling upgrade. but only one binding will be active at the same time. 
   
   If the java client doesn't intend to leverage a rolling upgrade, simply dropping the existing rule then adding a new rule instead, I'm free to use neat and shorthand APIs.
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750876870



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);

Review comment:
       As I mentioned above, they're for rolling upgrades. We don't intend to delete the previous rule, marked it as expired instead. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750843755



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.

Review comment:
       Please make sure these are complete sentences. It is very rare to read like this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746297030



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       I noticed that. That is why I added another comment about separating CRUD client and Admin client.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750026524



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       I think we don't have to update them for now. Let's add them back once the official document is ready.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r748685884



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Group related APIs are removed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-967787293


   @hanahmily Shall we include Measure registry APIs in this PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-963871393


   @lujiajing1126 Please update the document, then it would make review much easier.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-964132688


   > @lujiajing1126 Please update the document, then it would make review much easier.
   
   Docs added


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750815386



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,
+which will be active between `beginAt` and `expireAt`.
+
+### Measure
+
+`Measure` can also be defined directly with `BanyanDBClient`,
+
+```java
+// create a measure registry
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));

Review comment:
       @hanahmily How does this rule work with multiple-dimensionality metrics in one measure?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750926201



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);

Review comment:
       Could we provide `beginAt=now` and `expireAt=forever` at the server side in default? I can see the incoming feature no matter in which case, we could support rolling upgrade.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746290305



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Wait a moment, I can stream is created from client, what is the issue of using client group?

##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Wait a moment, I can see stream is created from client, what is the issue of using client group?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746279293



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       It's confused to CRUD groups based on another group's client. Supposing the user connects to an empty instance without any group, how to set the third flag?  
   
   I prefer to introduce a `setGroup` to `client` to manage and switch the current group.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749301074



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       > Is this binding `1:1`? If so, I think the client API doesn't have to follow in this way. It is better to provide API as simple as possible.
   
   The relation between Stream/Measure and IndexRuleBinding is 1:1 while IndexRuleBinding and IndexRule are 1:N
   
   > `this.client.define("sw", "sw-index-rule-binding", indexRule)`. This is super easier.
   
   What do you mean by `define(streamName/measureName, ruleBindingName, indexRule)`?
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749301737



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       I think so. We should document the design and glossary somewhere @hanahmily 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746400291



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       IIUC, we should align whether group CRUD is a part of the client first. 
   
   I thought there're two use cases banyandb have to support:
   
   ### Development/Demo
   In this case, OAP should play a full-featured client which contains workload resources and group APIs. OAP would take care of creating a group for ingesting and retrieving data. 
   
   ### Production
   OAP only CRUDs workload resources. A human-being administrator(DBA) will create a group where the OAP writes.
   
   If we should both of them, we have to unify the client to support all functions, introducing a role-based access control( 
    RBAC) to authorize resources to a user/role. This strategy would not affect this client, and RBAC APIs should not belong to this.
   
   Let me wrap this up, I prefer to keep group APIs in this client.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746574172



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       > If we make java-client as a thin client, removing group APIs sounds good to me.
   > 
   > @lujiajing1126 Please remove `GroupRegistry` from this, then add a `default` group in the prerun phase of the `metadata` module of banyand once Etcd server is ready to serve.
   
   Reasonable. I would remove this later




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746247239



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Which group this stream belongs to?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746283617



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Make sense to me. I will refactor this part.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746415267



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       For the group, a.k.a. database instance, in BanyanDB. I prefer the `group` could be created default in BanyanDB, like many other databases in quick start mode.
   Then other relative APIs could be BanyanDB-CLI's responsibility, which has lower priority for now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746292142



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       > Wait a moment, I can see stream is created from client, what is the issue of using client group?
   
   I suppose @hanahmily was referring to the `GroupRegistry`. You can CRUD groups other than that one you set for the client. So it may be confusing.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750026968



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       OK, I think we should skip rolling upgrade for now. Let's keep a simple version of APIs for now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-970321291


   Short hand APIs are provided for schema/index rule creation.
   
   Docs updated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r751281478



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749299667



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,
+
+```java
+IndexRuleBindingMetadataRegistry indexRuleBindingRegistry = client.indexRuleBindingRegistry();
+// define the rule binding "sw-index-rule-binding" and bind it with Stream "sw"
+IndexRuleBinding indexRuleBinding = new IndexRuleBinding("sw-index-rule-binding", IndexRuleBinding.Subject.referToStream("sw"));
+indexRuleBinding.setBeginAt(ZonedDateTime.now().minusDays(15));
+indexRuleBinding.setExpireAt(ZonedDateTime.now().plusYears(100));
+indexRuleBinding.addRule("db.instance");
+// create the index rule binding
+this.client.create(indexRuleBinding);
+```
+
+### Measure
+
+`Measure` can also be created with `MeasureMetadataRegistry`,
+
+```java
+// create a measure registry
+MeasureMetadataRegistry measureRegistry = client.measureRegistry();
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));
+// add field spec
+// compressMethod and encodingMethod can be specified
+m.addFieldSpec(Measure.FieldSpec.newIntField("tps").compressWithZSTD().encodeWithGorilla().build());
+// send create request
+measureRegistry.create(m);

Review comment:
       Typo. Fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749301737



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       I couldn't agree more. We should document the design and glossary somewhere @hanahmily 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746298300



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       Notice, generally, only manipulating group and other access grants are admin privilege. Creating stream or measure belong to nornal client.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-968176457


   > @hanahmily Shall we include Measure registry APIs in this PR?
   
   Absolutely. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749045008



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       Is this binding `1:1`? If so, I think the client API doesn't have to follow in this way. It is better to provide API as simple as possible. 
   
   `this.client.define("sw", "sw-index-rule-binding", indexRule)`. This is super easier.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750785743



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);

Review comment:
       `beginAt, expireAt` @hanahmily What are these 2 fields for?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750880119



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,
+which will be active between `beginAt` and `expireAt`.
+
+### Measure
+
+`Measure` can also be defined directly with `BanyanDBClient`,
+
+```java
+// create a measure registry
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));

Review comment:
       Supporting a label `scope` which indicates the dimension, its candidates are `day`, `hour`, and `minute`. The rule should be:
   
   ```java 
   m.addIntervalRule(Measure.IntervalRule.matchStringLabel("scope", "day", "1d"));
   m.addIntervalRule(Measure.IntervalRule.matchStringLabel("scope", "hour", "1h"));
   m.addIntervalRule(Measure.IntervalRule.matchStringLabel("scope", "minute", "1m"));
   ```
   
   The value expression supports a decimal number with a unit suffix.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750924917



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))

Review comment:
       Then, please rename them. This is very misleading, because searchable is a very important capability for end users.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749312918



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       > My point is, whether should we provide a simple API for user about exposing index creation. The binding and rule are for implementation mostly, I think. The end users usually just add one operation. @hanahmily Could you provide some suggestions?
   
   Probably we could provide some shorthand APIs to simplify schema creation, i.e. we make the schema POJOs as rich domain models.
   
   ```java
   Stream stream = client.define(Stream);
   // and Measure measure = client.define(Measure);
   IndexRuleBinding binding = stream.defineIndexRuleBinding(IndexRuleBinding);
   binding.define(List indexRules);
   // Or we may even skip indexRuleBinding creation
   stream.defineIndexRules(List indexRules);
   ```

##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       > My point is, whether should we provide a simple API for user about exposing index creation. The binding and rule are for implementation mostly, I think. The end users usually just add one operation. @hanahmily Could you provide some suggestions?
   
   I see. Probably we could provide some shorthand APIs to simplify schema creation, i.e. we make the schema POJOs as rich domain models.
   
   ```java
   Stream stream = client.define(Stream);
   // and Measure measure = client.define(Measure);
   IndexRuleBinding binding = stream.defineIndexRuleBinding(IndexRuleBinding);
   binding.define(List indexRules);
   // Or we may even skip indexRuleBinding creation
   stream.defineIndexRules(List indexRules);
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749321668



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       I prefer to skip `binding` creation. But let's wait for @hanahmily for a minute.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750842823



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))

Review comment:
       > What is the difference between these 2 TagFamily(s)? Especially one says `searchable`, and the other(`binary`) is clearly not searchable. How do these 2 APIs define searchable?
   
   This is only an example. `searchable` and `binary` are names of the tag family




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-968520481


   > > > > @hanahmily Shall we include Measure registry APIs in this PR?
   > > > 
   > > > 
   > > > Absolutely.
   > > 
   > > 
   > > Measure related APIs are added
   > 
   > I think we need document updated.
   
   An example of Measure API usage has been added to the README


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r751280911



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);

Review comment:
       Default values are given now




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r751281786



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))

Review comment:
       Use sample names now




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746245753



##########
File path: src/main/proto/banyandb/v1/banyandb-metadata.proto
##########
@@ -0,0 +1,401 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+syntax = "proto3";
+
+option java_package = "org.apache.skywalking.banyandb.database.v1.metadata";

Review comment:
       Yes, and they were merged into banyandb API repo, I think we discussed and reviewed them then.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r746291288



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Group
+
+To save and fetch data, group and schema must be defined first,
+
+```java
+GroupRegistry groupRegistry = client.groupRegistry();
+// create a group called "default"
+groupRegistry.create("default");
+// list group
+List<String> groups = groupRegistry.list();
+// check existence
+boolean existence = groupRegistry.exist("default");
+```
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));

Review comment:
       I think the key point is how to create a client, should it be group binding or sensitive?
   I think it it better to have an adminClient and general CRUD client, the latter one could bind group in the initial stage, to ease CRUD.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r745335771



##########
File path: src/main/proto/banyandb/v1/banyandb-metadata.proto
##########
@@ -0,0 +1,401 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+syntax = "proto3";
+
+option java_package = "org.apache.skywalking.banyandb.database.v1.metadata";

Review comment:
       @hanahmily Please make me know these metadata APIs are ready to review and discuss.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750926905



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,
+which will be active between `beginAt` and `expireAt`.
+
+### Measure
+
+`Measure` can also be defined directly with `BanyanDBClient`,
+
+```java
+// create a measure registry
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));

Review comment:
       Got it. @lujiajing1126 Please use @hanahmily 's case to replace yours. It is much better.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750842561



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.

Review comment:
       the above code block




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749304169



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,

Review comment:
       My point is, whether should we provide a simple API for user about exposing index creation. The binding and rule are for implementation mostly, I think. The end users usually just add one operation. @hanahmily Could you provide some suggestions?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750024590



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       I would like to show more details in https://docs.google.com/document/d/1XFzHWXSWyIC7AVNIx0ifQ0eJAJYfoPO-2j_lgbl9zu8/edit after the measure feat is ready recently. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750021339



##########
File path: README.md
##########
@@ -19,10 +19,94 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+StreamMetadataRegistry streamRegistry = client.streamRegistry();
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema
+streamRegistry.create(s);
+```
+
+with `StreamMetadataRegistry`, CRUD operations are supported for `Stream`, `IndexRuleBinding` and `IndexRule`.
+
+### IndexRuleBinding/IndexRule
+
+For better search performance, index rules are necessary for `Stream` while `IndexRuleBinding` helps 
+bind the `IndexRule` to the `Stream`,
+
+```java
+IndexRuleMetadataRegistry indexRuleRegistry = client.indexRuleRegistry();
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+this.client.create(indexRule);
+```
+
+and then create an `IndexRuleBinding` to bind it/them to the Stream,
+
+```java
+IndexRuleBindingMetadataRegistry indexRuleBindingRegistry = client.indexRuleBindingRegistry();
+// define the rule binding "sw-index-rule-binding" and bind it with Stream "sw"
+IndexRuleBinding indexRuleBinding = new IndexRuleBinding("sw-index-rule-binding", IndexRuleBinding.Subject.referToStream("sw"));
+indexRuleBinding.setBeginAt(ZonedDateTime.now().minusDays(15));
+indexRuleBinding.setExpireAt(ZonedDateTime.now().plusYears(100));

Review comment:
       For the rolling upgrade as I mentioned above




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750785571



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,

Review comment:
       Same confused reading about this `where`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r750785174



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))

Review comment:
       What is the difference between these 2 TagFamily(s)? Especially one says `searchable`, and the other(`binary`) is clearly not searchable. How do these 2 APIs define searchable?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] hanahmily commented on pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
hanahmily commented on pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#issuecomment-970886759


   @wu-sheng please recheck this, thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] wu-sheng commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r749050943



##########
File path: src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Measure.java
##########
@@ -0,0 +1,438 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.banyandb.v1.client.metadata;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.RequiredArgsConstructor;
+import lombok.Setter;
+import org.apache.skywalking.banyandb.database.v1.metadata.BanyandbMetadata;
+import org.apache.skywalking.banyandb.v1.client.util.TimeUtils;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+@Setter
+@Getter
+@EqualsAndHashCode(callSuper = true)
+public class Measure extends NamedSchema<BanyandbMetadata.Measure> {
+    /**
+     * specs of tag families
+     */
+    private List<TagFamilySpec> tagFamilySpecs;
+
+    /**
+     * fieldSpecs denote measure values
+     */
+    private List<FieldSpec> fieldSpecs;
+
+    /**
+     * tag names used to generate an entity
+     */
+    private List<String> entityTagNames;
+
+    /**
+     * intervalRules define data points writing interval
+     */
+    private List<IntervalRule<?>> intervalRules;
+
+    /**
+     * number of shards
+     */
+    private int shardNum;
+
+    /**
+     * duration determines how long a Stream keeps its data
+     */
+    private Duration ttl;
+
+    public Measure(String name, int shardNum, Duration ttl) {
+        this(name, shardNum, ttl, null);
+    }
+
+    private Measure(String name, int shardNum, Duration ttl, ZonedDateTime updatedAt) {
+        super(name, updatedAt);
+        this.tagFamilySpecs = new ArrayList<>();
+        this.entityTagNames = new ArrayList<>();
+        this.fieldSpecs = new ArrayList<>();
+        this.intervalRules = new ArrayList<>();
+        this.shardNum = shardNum;
+        this.ttl = ttl;
+    }
+
+    /**
+     * Add a tag name as a part of the entity
+     *
+     * @param name the name of the tag
+     */
+    public Measure addTagNameAsEntity(String name) {
+        this.entityTagNames.add(name);
+        return this;
+    }
+
+    /**
+     * Add a tag family spec to the schema
+     *
+     * @param tagFamilySpec a tag family containing tag specs
+     */
+    public Measure addTagFamilySpec(TagFamilySpec tagFamilySpec) {
+        this.tagFamilySpecs.add(tagFamilySpec);
+        return this;
+    }
+
+    /**
+     * Add an interval rule to the schema
+     *
+     * @param intervalRule an interval rule to match tag name and value
+     */
+    public Measure addIntervalRule(IntervalRule<?> intervalRule) {

Review comment:
       FYI @hanahmily We need a very specific document about all concepts. Such as what is internal rule, and more about index and index binding. We should have links(from website) could be linked from here.
   Currently, this comment actually doesn't have real meaning.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-banyandb-java-client] lujiajing1126 commented on a change in pull request #4: Support metadata registry api

Posted by GitBox <gi...@apache.org>.
lujiajing1126 commented on a change in pull request #4:
URL: https://github.com/apache/skywalking-banyandb-java-client/pull/4#discussion_r751281116



##########
File path: README.md
##########
@@ -19,10 +19,81 @@ Create a `BanyanDBClient` with host, port and a user-specified group and then es
 ```java
 // use `default` group
 client = new BanyanDBClient("127.0.0.1", 17912, "default");
-// establish a connection
+// to send any request, a connection to the server must be estabilished
 client.connect(channel);
 ```
 
+## Schema Management
+
+### Stream
+
+Then we may define a stream with customized configurations,
+
+```java
+// build a stream "sw" with 2 shards and ttl equals to 30 days
+Stream s = new Stream("sw", 2, Duration.ofDays(30));
+s.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// TagFamily - data
+TagFamilySpec dataFamily = new TagFamilySpec("data");
+dataFamily.addTagSpec(TagFamilySpec.TagSpec.newBinaryTag("data_binary"));
+s.addTagFamilySpec(dataFamily);
+// TagFamily - searchable
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+        .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+        .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+s.addTagFamilySpec(searchableFamily);
+// create with the stream schema, client is the BanyanDBClient created above
+s = client.define(s);
+```
+
+where simple APIs (i.e. `BanyanDBClient.define`) are provided to define the schema of `Measure` or `Stream`.
+
+### IndexRules
+
+For better search performance, index rules are necessary for `Stream` and `Measure`. You have to
+specify a full list of index rules that would be bounded to the target `Stream` and `Measure`.
+
+```java
+// create IndexRule with inverted index type and save it to series store
+IndexRule indexRule = new IndexRule("db.instance", IndexRule.IndexType.INVERTED, IndexRule.IndexLocation.SERIES);
+// tag name specifies the indexed tag
+indexRule.addTag("db.instance");
+// create the index rule "db.instance"
+client.defineIndexRules(stream, beginAt, expireAt, indexRule);
+```
+
+where multiple index rules are supported. Internally, an `IndexRuleBinding` is created automatically for users,
+which will be active between `beginAt` and `expireAt`.
+
+### Measure
+
+`Measure` can also be defined directly with `BanyanDBClient`,
+
+```java
+// create a measure registry
+// create a new measure schema with 2 shards and ttl 30 days.
+Measure m = new Measure("measure-example", 2, Duration.ofDays(30));
+// set entity
+m.addTagNameAsEntity("service_id").addTagNameAsEntity("service_instance_id").addTagNameAsEntity("state");
+// add tag family: "searchable"
+TagFamilySpec searchableFamily = new TagFamilySpec("searchable");
+searchableFamily.addTagSpec(TagFamilySpec.TagSpec.newStringTag("trace_id"))
+    .addTagSpec(TagFamilySpec.TagSpec.newIntTag("state"))
+    .addTagSpec(TagFamilySpec.TagSpec.newStringTag("service_id"));
+m.addTagFamilySpec(searchableFamily);
+// set interval rules
+m.addIntervalRule(Measure.IntervalRule.matchStringLabel("interval", "day", "1d"));
+m.addIntervalRule(Measure.IntervalRule.matchNumericLabel("interval", 3600L, "1h"));

Review comment:
       Suggestion applied




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org