You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2021/04/17 06:44:53 UTC
[GitHub] [servicecomb-service-center] fuziye01 opened a new pull request #953: SCB-2094 Fast register with mongo
fuziye01 opened a new pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953
【issue】: #948
【特性/模块名称】：实例快速注册特性
【修改内容】：
默认使用实例注册流程，添加快速注册流程，并加上开关
如果开启快速注册开关，注册实例时候直接把实例放到队列然后返回
通过定时任务进行批量注册到mongodb
【自测情况】：
ut全部通过
基本流程测试通过
性能测试数据：
tps 9w (sc 8u16g*2 mongodb16u32g ),
tps 15w (sc 16u32g*2 mongodb16u32g )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620793162
##########
File path: etc/conf/app.yaml
##########
@@ -132,6 +132,11 @@ registry:
verifyPeer: false
certFile: /opt/ssl/client.crt
keyFile: /opt/ssl/client.key
+ fastRegister:
Review comment:
fastRegister并不是mongo层这个数据访问层的能力，而是sc整体的，不该放在mongo下。因为这并不是你操作mongodb时的一种能力，配置项设计要仔细思考
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r622014368
##########
File path: datasource/mongo/fast_register_timer.go
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "runtime"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/pkg/util"
+)
+
+const (
+ loopTime = 100 * time.Millisecond
+ batchLen = 10000
+ fuseMinCount = 3
+ fuseTime = 5 * time.Second
+ maxRegisterFailedTime = 500
+ ctxCancelTimeOut = 60 * time.Second
+)
+
+var fastRegisterTimeTask *FastRegisterTimeTask
+
+type FastRegisterTimeTask struct {
Review comment:
time是想表示定时任务
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] robotLJW commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
robotLJW commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617232839
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,25 @@
+package mongo
+
+import (
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/server/config"
+)
+
+var (
+ configOnce sync.Once
+ fastRegConfig FastRegConfig
+)
+
+type FastRegConfig struct {
+ enable bool
+ queueSize int
+}
+
+func Configuration() *FastRegConfig {
+ configOnce.Do(func() {
+ fastRegConfig.enable = config.GetBool("registry.mongo.fastRegister.enable", false)
+ fastRegConfig.queueSize = config.GetInt("registry.mongo.fastRegister.queueSize", 500000)
Review comment:
50000还是定义成常量会比较好，魔鬼数字
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620793955
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,79 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
Review comment:
为何要做这种判断，id有就是custom了，直接透传覆盖就行了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620793162
##########
File path: etc/conf/app.yaml
##########
@@ -132,6 +132,11 @@ registry:
verifyPeer: false
certFile: /opt/ssl/client.crt
keyFile: /opt/ssl/client.key
+ fastRegister:
Review comment:
fastRegister并不是mongo层这个数据访问层的能力，而是sc整体的，不该放在mongo下。因为这并不是你操作mongodb时的一种能力
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang merged pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang merged pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] little-cui commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
little-cui commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615510635
##########
File path: datasource/mongo/ms.go
##########
@@ -1852,7 +1916,7 @@ func (ds *DataSource) UnregisterInstance(ctx context.Context, request *discovery
func (ds *DataSource) Heartbeat(ctx context.Context, request *discovery.HeartbeatRequest) (*discovery.HeartbeatResponse, error) {
remoteIP := util.GetIPFromContext(ctx)
instanceFlag := util.StringJoin([]string{request.ServiceId, request.InstanceId}, "/")
- err := KeepAliveLease(ctx, request)
+ err := KeepAliveLease(request)
Review comment:
不应该去掉ctx，否则上层请求无法主动终止流程
##########
File path: datasource/mongo/ms.go
##########
@@ -1964,7 +2028,7 @@ func registryInstance(ctx context.Context, request *discovery.RegisterInstanceRe
Instance: instance,
}
- insertRes, err := client.GetMongoClient().Insert(ctx, model.CollectionInstance, data)
+ insertRes, err := client.GetMongoClient().Insert(context.TODO(), model.CollectionInstance, data)
Review comment:
同上，这种会导致阻塞gorouting
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,80 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
+
+ if len(request.Instance.InstanceId) == 0 {
+ isCustomID = false
+ request.Instance.InstanceId = uuid.Generator().GetInstanceID(ctx)
+ }
+
+ if isFastRegisterEnabled {
+ // fast register, just add instance to channel and batch register them later
+ event := &InstanceRegisterEvent{ctx, request, isCustomID, 0}
+ GetFastRegisterInstanceService().AddEvent(event)
Review comment:
这种实现牺牲了一致性，业务都会出现问题，设计不合理
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620802366
##########
File path: datasource/mongo/mongo.go
##########
@@ -234,3 +237,14 @@ func (ds *DataSource) initStore() {
sd.Store().Run()
<-sd.Store().Ready()
}
+
+func initFastRegister() {
+ fastRegConfig := FastRegConfiguration()
+
+ if fastRegConfig.enable {
+ fastRegisterService := NewFastRegisterInstanceService()
+ SetFastRegisterInstanceService(fastRegisterService)
+
+ NewRegisterTimer().Run()
Review comment:
计时器.run，这个名字起的不易理解, 我理解是计时器在空跑
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615489006
##########
File path: etc/conf/app.yaml
##########
@@ -132,6 +132,11 @@ registry:
verifyPeer: false
certFile: /opt/ssl/client.crt
keyFile: /opt/ssl/client.key
+ fastRegister:
Review comment:
用名字
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620797377
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,79 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
Review comment:
因为有异步注册逻辑，会先生成UUID返回。所以把判断流程提前了，后面的流程都会有id, 在生成UUID的时候会判断是不是custom，然后传下去
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620793162
##########
File path: etc/conf/app.yaml
##########
@@ -132,6 +132,11 @@ registry:
verifyPeer: false
certFile: /opt/ssl/client.crt
keyFile: /opt/ssl/client.key
+ fastRegister:
Review comment:
fastRegister并不是mongo层这个数据访问层的能力，而是sc整体的，不该放在mongo下
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] robotLJW commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
robotLJW commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617225510
##########
File path: datasource/mongo/event/instance_event_handler.go
##########
@@ -75,6 +75,7 @@ func (h InstanceEventHandler) OnEvent(evt sd.MongoEvent) {
metrics.ReportInstances(instance.Domain, increaseOne)
frameworkName, frameworkVersion := getFramework(microService)
metrics.ReportFramework(instance.Domain, instance.Project, frameworkName, frameworkVersion, increaseOne)
+ return
Review comment:
这个为什么加个return？insert不需要推？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617225892
##########
File path: datasource/mongo/event/instance_event_handler.go
##########
@@ -75,6 +75,7 @@ func (h InstanceEventHandler) OnEvent(evt sd.MongoEvent) {
metrics.ReportInstances(instance.Domain, increaseOne)
frameworkName, frameworkVersion := getFramework(microService)
metrics.ReportFramework(instance.Domain, instance.Project, frameworkName, frameworkVersion, increaseOne)
+ return
Review comment:
初始化不需要推，创建修改删除才需要推，参考etcd的逻辑EVT_INIT也是直接返回的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r621947481
##########
File path: datasource/mongo/fast_register_timer.go
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "runtime"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/pkg/util"
+)
+
+const (
+ loopTime = 100 * time.Millisecond
+ batchLen = 10000
+ fuseMinCount = 3
+ fuseTime = 5 * time.Second
+ maxRegisterFailedTime = 500
+ ctxCancelTimeOut = 60 * time.Second
+)
+
+var fastRegisterTimeTask *FastRegisterTimeTask
+
+type FastRegisterTimeTask struct {
Review comment:
省略time是不是更好
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620788288
##########
File path: README.md
##########
@@ -31,6 +31,7 @@ It provides out of box support for metrics and tracing. It has a web portal to m
- **[`Datacenters`](https://service-center.readthedocs.io/en/latest/dev-guides/multidcs.html)**: Additional layer of abstraction to clusters deployed in multiple datacenters
- **[`Aggregation`](https://service-center.readthedocs.io/en/latest/design-guides/design.html)**: Able to aggregate microservices from multiple registry platforms and
support platform registry and client side registry at the same time
+ - **[`FastRegister`](/docs/user-guides/fast-register.md)**: Fast register instance to service center
Review comment:
要用sphinx文档框架管理文档
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] robotLJW commented on pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
robotLJW commented on pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#issuecomment-823798781
commit可以合并一下
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620791711
##########
File path: datasource/mongo/fast_register_test.go
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo_test
+
+import (
+ "context"
+ "testing"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/datasource"
+ "github.com/apache/servicecomb-service-center/datasource/mongo"
+ pb "github.com/go-chassis/cari/discovery"
+ "github.com/stretchr/testify/assert"
+)
+
+func TestInstance_BatchCreate(t *testing.T) {
+ var serviceID string
+
+ mongo.FastRegConfiguration()
+ fastRegisterService := mongo.NewFastRegisterInstanceService()
+ mongo.SetFastRegisterInstanceService(fastRegisterService)
+
+ fastRegisterTimer := mongo.NewRegisterTimer()
+ fastRegisterTimer.Run()
+
+ t.Run("create service", func(t *testing.T) {
Review comment:
这些test case name需要再写清楚点
given， when，result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] little-cui commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
little-cui commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615510963
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,80 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
+
+ if len(request.Instance.InstanceId) == 0 {
+ isCustomID = false
+ request.Instance.InstanceId = uuid.Generator().GetInstanceID(ctx)
+ }
+
+ if isFastRegisterEnabled {
+ // fast register, just add instance to channel and batch register them later
+ event := &InstanceRegisterEvent{ctx, request, isCustomID, 0}
+ GetFastRegisterInstanceService().AddEvent(event)
Review comment:
1. 需要在配置项中说明功能实现机制
2. 需要提供一个readme，说明fast regsiter的正确使用方法
3. 在主Readme增加features，关联资料
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] little-cui commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
little-cui commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r618900304
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,27 @@
+package mongo
+
+import (
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/server/config"
+)
+
+const defaultQueueSize = 500000
+
+var (
+ configOnce sync.Once
+ fastRegConfig FastRegConfig
+)
+
+type FastRegConfig struct {
+ enable bool
+ queueSize int
+}
+
+func Configuration() *FastRegConfig {
Review comment:
不能叫Configuration，否则表示整个mongo实现的配置入口
##########
File path: datasource/mongo/FastRegister.md
##########
@@ -0,0 +1,82 @@
+# Fast Register
+
+Fast registration feature, can support millions of instance registration
+
+This feature is primarily used in scenarios where an ultra-high performance registry is required and is not recommended for scenarios where performance requirements are low or instance levels are small
+
+This feature is turn off by default, if need fast register you should open the fast registration switch
+
+When this feature is open, you can call register interface API, the service center will put instances in the queue, then direct return instanceId to users, at last, register registry asynchronously by timing task
+
+## QuickStart Guide
+1.Turn on the Fast Registration Switch
+
+Modify the service-center configuration file /conf/app.yaml
+
+The default configuration of app.yaml is as follows:
+```
+register:
+ mongo:
+ fastRegister:
+ # enable to fast register instance,
+ # if fastRegister.enable is true, instance will be registered asynchronously,
+ # just put instance in the queue and return, and then register through the timing task
+ enable: false
+ # if fastRegister.enable is true, should set the queueSize to indicate the max number of concurrent registrations
+ queueSize: 500000
+```
+
+Turn on the switch, setting as follows:
+```
+register.mongo.fasterRegister.enable=true
+```
+
+
+2.Configure the fast registration queue length, the default is 50W
+```
+register.mongo.fasterRegister.queueSize=500000
+```
+
+3.start service center
+```
+./service-center
+```
+
+4.Call the registry interface
+
+Call the registry interface, you will receive InstanceID soon, now a fast registration has been completed once
+
+* Registered instance APIs can be called concurrently
+* There is a slight delay between returning the instanceID and actually registering the instance to database, but 100w instance registration delays are within seconds
+* If more than 15 minutes did not discovery the instance ,there may be a problem with the environment. The client can register again with the InstanceID that has been generated and return to user
+
+## Process Design
+The flow chart is as follows:
+
+
+
+
+Normal Case:
+
+If the fast registration is enabled, it is put in the queue and eventually registered to MongoDB in batch by timed tasks(The time interval is 100 millimeters)
+
+
+Abnormal Case:
+
+1. If the connection between Mongo and service center is broken, and the registration fails, the instance will be put into the failure queue and registered again
+2. If the registration fails for 3 consecutive times, the fuse will be cut off for 5s and resume after successful registration
+3. If a single instance fails to register for more than 500 times, the instance will be discarded, and the SDK will register again when the heartbeat finds that the instance does not exist
+
+## Attention
+1.The database with ETCD scenario does not have this feature; only the Mongo database scenario does
+
+2.Because the registration is asynchronous, there will be a certain amount of delay in the registration, the delay is basically in the second level
+
+## Performance Test
Review comment:
用表格呈现
##########
File path: datasource/mongo/fast_register_test.go
##########
@@ -0,0 +1,129 @@
+package mongo_test
Review comment:
缺少license header
##########
File path: datasource/mongo/FastRegister.md
##########
@@ -0,0 +1,82 @@
+# Fast Register
Review comment:
文件放到docs/user-guides里面，并更新docs/user-guides.rst
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,27 @@
+package mongo
Review comment:
缺少license header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620802758
##########
File path: datasource/mongo/fast_register_timer.go
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "runtime"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/pkg/util"
+)
+
+const (
+ loopTime = 100 * time.Millisecond
+ batchLen = 10000
+ fuseMinCount = 3
+ fuseTime = 5 * time.Second
+ maxRegisterFailedTime = 500
+ ctxCancelTimeOut = 60 * time.Second
+)
+
+var fastRegisterTimer *FastRegisterTimer
+
+type FastRegisterTimer struct {
+ goroutine *gopool.Pool
+}
+
+func NewRegisterTimer() *FastRegisterTimer {
+ return &FastRegisterTimer{
+ goroutine: gopool.New(context.Background()),
+ }
+}
+
+func (rt *FastRegisterTimer) Run() {
+ gopool.Go(fastRegisterTimer.loopRegister)
+}
+
+func (rt *FastRegisterTimer) Stop() {
+ rt.goroutine.Close(true)
+}
+
+func (rt *FastRegisterTimer) loopRegister(ctx context.Context) {
+ defer rt.Stop()
+
+ blockCh := make(chan struct{}, runtime.NumCPU()-1)
+ failedCount := make(chan int, 1)
+ failedCount <- 0
+ ticker := time.NewTicker(loopTime)
+
+ for {
+ select {
+ case <-ctx.Done():
+ // server shutdown
+ return
+ case <-ticker.C:
+ length := len(GetFastRegisterInstanceService().InstEventCh)
+ if length == 0 {
+ continue
+ }
+
+ // fuse is triggered if failed registry counts more than fuseMinCount
Review comment:
这一大段，圈复杂度有点高，重构
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615489006
##########
File path: etc/conf/app.yaml
##########
@@ -132,6 +132,11 @@ registry:
verifyPeer: false
certFile: /opt/ssl/client.crt
keyFile: /opt/ssl/client.key
+ fastRegister:
Review comment:
用名词
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620797377
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,79 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
Review comment:
因为有异步注册逻辑，会先生成UUID返回。 在生成UUID的时候会判断是不是custom直接返回回来
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620802366
##########
File path: datasource/mongo/mongo.go
##########
@@ -234,3 +237,14 @@ func (ds *DataSource) initStore() {
sd.Store().Run()
<-sd.Store().Ready()
}
+
+func initFastRegister() {
+ fastRegConfig := FastRegConfiguration()
+
+ if fastRegConfig.enable {
+ fastRegisterService := NewFastRegisterInstanceService()
+ SetFastRegisterInstanceService(fastRegisterService)
+
+ NewRegisterTimer().Run()
Review comment:
计时器.run，这个名字起的不易理解, 我理解是计时器在空跑，结构体名 函数名要仔细思考和设计
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620793955
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,79 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
Review comment:
为何要做这种判断，id有就是custom了，直接传给后台就行了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] DFSOrange commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
DFSOrange commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615217747
##########
File path: datasource/mongo/ms.go
##########
@@ -1404,13 +1404,80 @@ func getServiceDetailUtil(ctx context.Context, mgs *model.Service, countOnly boo
}
// Instance management
-func (ds *DataSource) RegisterInstance(ctx context.Context, request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+func (ds *DataSource) RegisterInstance(ctx context.Context,
+ request *discovery.RegisterInstanceRequest) (*discovery.RegisterInstanceResponse, error) {
+
+ isCustomID := true
+
+ if len(request.Instance.InstanceId) == 0 {
+ isCustomID = false
+ request.Instance.InstanceId = uuid.Generator().GetInstanceID(ctx)
+ }
+
+ if isFastRegisterEnabled {
+ // fast register, just add instance to channel and batch register them later
+ event := &InstanceRegisterEvent{ctx, request, isCustomID, 0}
+ GetFastRegisterInstanceService().AddEvent(event)
+
+ return &discovery.RegisterInstanceResponse{
+ Response: discovery.CreateResponse(discovery.ResponseSuccess, "Register service instance successfully."),
+ InstanceId: request.Instance.InstanceId,
+ }, nil
+ }
+
+ return RegisterInstanceSingle(ctx, request, isCustomID)
+}
+
+func RegisterInstanceSingle(ctx context.Context, request *discovery.RegisterInstanceRequest,
+ isUserDefinedID bool) (*discovery.RegisterInstanceResponse, error) {
+
+ resp, needRegister, err := preProcessRegister(ctx, request.Instance, isUserDefinedID)
+
+ if err != nil || !needRegister {
+ log.Error("pre process instance err, or instance already existed", err)
+ return resp, err
+ }
+
+ return registryInstance(ctx, request)
+}
+
+func RegisterInstanceBatch(events []*InstanceRegisterEvent) (*discovery.RegisterInstanceResponse, error) {
+ instances := make([]interface{}, 0)
Review comment:
可以make([]interface{},0,len(events)),这样events多的时候slice不需要多次扩容
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] robotLJW commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
robotLJW commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r615507761
##########
File path: datasource/mongo/fast_register_inst_service.go
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "github.com/apache/servicecomb-service-center/server/config"
+
+ "github.com/go-chassis/cari/discovery"
+)
+
+var isFastRegisterEnabled bool
+
+var fastRegisterInstanceService *RegisterFastInstanceService
+
+type RegisterFastInstanceService struct {
+ InstEventCh chan *InstanceRegisterEvent
+ FailedInstCh chan *InstanceRegisterEvent
+}
+
+type InstanceRegisterEvent struct {
+ Ctx context.Context
+ Request *discovery.RegisterInstanceRequest
+ isCustomID bool
+ failedTime int
+}
+
+func NewFastRegisterInstanceService() *RegisterFastInstanceService {
Review comment:
https://github.com/apache/servicecomb-service-center/blob/master/datasource/etcd/config.go
可以参考下这个写法，其实这个配置按理应该初始化一次就行
##########
File path: datasource/mongo/fast_register_timer.go
##########
@@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "runtime"
+ "time"
+)
+
+const (
+ loopTime = 100
+ batchLen = 10000
+ fuseMinCount = 3
+ fuseTime = 1000
+ maxRegisterFailedTime = 500
+)
+
+var fastRegisterTimer *FastRegisterTimer
+
+func NewRegisterTimer() *FastRegisterTimer {
Review comment:
Newxxx 位置放到 type FastRegisterTimer struct { 后面更合适，见uber_go_guide_cn#函数名规范
https://github.com/xxjwxc/uber_go_guide_cn#%E5%87%BD%E6%95%B0%E5%90%8D
##########
File path: datasource/mongo/fast_register_inst_service.go
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "github.com/apache/servicecomb-service-center/server/config"
+
+ "github.com/go-chassis/cari/discovery"
+)
+
+var isFastRegisterEnabled bool
+
+var fastRegisterInstanceService *RegisterFastInstanceService
+
+type RegisterFastInstanceService struct {
Review comment:
我上次记得华哥说个一个.go文件最好只包含一个struct，其他的应该放到types里面会比较好，我之前心跳的一个struct也抽到types里面
https://github.com/robotLJW/servicecomb-service-center/blob/6a75041291c9b75f4c8d09a918c7b1efc158cf33/datasource/mongo/heartbeat/cache/types.go
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620795779
##########
File path: datasource/mongo/fast_register_timer.go
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "runtime"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/pkg/util"
+)
+
+const (
+ loopTime = 100 * time.Millisecond
+ batchLen = 10000
+ fuseMinCount = 3
+ fuseTime = 5 * time.Second
+ maxRegisterFailedTime = 500
+ ctxCancelTimeOut = 60 * time.Second
+)
+
+var fastRegisterTimer *FastRegisterTimer
+
+type FastRegisterTimer struct {
+ goroutine *gopool.Pool
+}
+
+func NewRegisterTimer() *FastRegisterTimer {
+ return &FastRegisterTimer{
+ goroutine: gopool.New(context.Background()),
+ }
+}
+
+func (rt *FastRegisterTimer) Run() {
+ gopool.Go(fastRegisterTimer.loopRegister)
+}
+
+func (rt *FastRegisterTimer) Stop() {
+ rt.goroutine.Close(true)
+}
+
+func (rt *FastRegisterTimer) loopRegister(ctx context.Context) {
+ defer rt.Stop()
+
+ blockCh := make(chan struct{}, runtime.NumCPU()-1)
+ failedCount := make(chan int, 1)
+ failedCount <- 0
+ ticker := time.NewTicker(loopTime)
+
+ for {
+ select {
+ case <-ctx.Done():
+ // server shutdown
+ return
+ case <-ticker.C:
+ length := len(GetFastRegisterInstanceService().InstEventCh)
+ if length == 0 {
+ continue
+ }
+
+ // fuse is triggered if failed registry counts more than fuseMinCount
+ count := <-failedCount
+ if count > fuseMinCount {
+ time.Sleep(fuseTime)
+ } else {
+ failedCount <- count
+ }
+
+ events := rt.generateEvents(length)
+ blockCh <- struct{}{}
+ go rt.BatchRegisterInstance(events, blockCh, failedCount)
+
+ case event, ok := <-GetFastRegisterInstanceService().FailedInstCh:
+ // if instance batch register failed, register it single
+ if !ok {
+ log.Error("failed instance channel is error", errors.New("channel closed"))
+ continue
+ }
+
+ if event.failedTime > maxRegisterFailedTime {
+ log.Error(fmt.Sprintf("instance register retry time is more than max register time:%d, "+
+ "the instance params maybe wrong, drop it", maxRegisterFailedTime),
+ errors.New("retry register instance failed"))
+ continue
+ }
+
+ blockCh <- struct{}{}
+
+ go rt.RegisterInstance(event, blockCh)
+ }
+ }
+}
+
+func (rt *FastRegisterTimer) generateEvents(length int) []*InstanceRegisterEvent {
+ // if channel len >= batch len, use batch len, otherwise use channel len
+ if length >= batchLen {
+ length = batchLen
+ }
+
+ events := make([]*InstanceRegisterEvent, 0)
+
+ for i := 0; i < length; i++ {
+ event, ok := <-GetFastRegisterInstanceService().InstEventCh
+
+ refreshCanceledCtx(event)
+
+ if !ok {
+ log.Error("instance event channel is error", errors.New("channel closed"))
+ continue
+ }
+ events = append(events, event)
+ }
+ return events
+}
+
+func (rt *FastRegisterTimer) BatchRegisterInstance(events []*InstanceRegisterEvent, blockCh chan struct{}, failedCount chan int) {
+ defer endBlock(blockCh)
+
+ ctx, cancel := context.WithTimeout(context.Background(), ctxCancelTimeOut)
+ defer cancel()
+
+ _, err := RegisterInstanceBatch(ctx, events)
Review comment:
RegisterInstanceBatch 和BatchRegisterInstance名字令人不易理解，容易混淆，一眼看不出职责
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620789363
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/server/config"
+)
+
+const defaultQueueSize = 500000
+
+var (
+ configOnce sync.Once
+ fastRegConfig FastRegConfig
+)
+
+type FastRegConfig struct {
+ enable bool
+ queueSize int
+}
+
+func FastRegConfiguration() *FastRegConfig {
+ configOnce.Do(func() {
+ fastRegConfig.enable = config.GetBool("registry.mongo.fastRegister.enable", false)
+ fastRegConfig.queueSize = config.GetInt("registry.mongo.fastRegister.queueSize", defaultQueueSize)
Review comment:
我不建议用2个参数，而是用quesize为0或大于0来控制，大于0说明需要启用功能，小于等于0关闭。不要让用户学习2个参数
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/server/config"
+)
+
+const defaultQueueSize = 500000
+
+var (
+ configOnce sync.Once
+ fastRegConfig FastRegConfig
+)
+
+type FastRegConfig struct {
+ enable bool
+ queueSize int
+}
+
+func FastRegConfiguration() *FastRegConfig {
+ configOnce.Do(func() {
+ fastRegConfig.enable = config.GetBool("registry.mongo.fastRegister.enable", false)
+ fastRegConfig.queueSize = config.GetInt("registry.mongo.fastRegister.queueSize", defaultQueueSize)
Review comment:
另外，这引出了另外一个问题，当queue满了的时候，应该走正常注册流程，而不是拒绝，这样才能持续服务，但是要充分测试下这个场景的表现
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617225892
##########
File path: datasource/mongo/event/instance_event_handler.go
##########
@@ -75,6 +75,7 @@ func (h InstanceEventHandler) OnEvent(evt sd.MongoEvent) {
metrics.ReportInstances(instance.Domain, increaseOne)
frameworkName, frameworkVersion := getFramework(microService)
metrics.ReportFramework(instance.Domain, instance.Project, frameworkName, frameworkVersion, increaseOne)
+ return
Review comment:
初始化不需要推，创建修改删除才需要推
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] robotLJW commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
robotLJW commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617234447
##########
File path: datasource/mongo/fast_register_inst_service.go
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+var fastRegisterInstanceService *RegisterFastInstanceService
+
+type RegisterFastInstanceService struct {
+ InstEventCh chan *InstanceRegisterEvent
+ FailedInstCh chan *InstanceRegisterEvent
+}
+
+func NewFastRegisterInstanceService() *RegisterFastInstanceService {
+ fastRegisterQueueSize := fastRegConfig.queueSize
Review comment:
这个会不会有空指针，如果没有执行Configuration()，还是我多虑的执行NEW方法前肯定会执行Configuration()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620789363
##########
File path: datasource/mongo/fast_register_config.go
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+import (
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/server/config"
+)
+
+const defaultQueueSize = 500000
+
+var (
+ configOnce sync.Once
+ fastRegConfig FastRegConfig
+)
+
+type FastRegConfig struct {
+ enable bool
+ queueSize int
+}
+
+func FastRegConfiguration() *FastRegConfig {
+ configOnce.Do(func() {
+ fastRegConfig.enable = config.GetBool("registry.mongo.fastRegister.enable", false)
+ fastRegConfig.queueSize = config.GetInt("registry.mongo.fastRegister.queueSize", defaultQueueSize)
Review comment:
我不建议用2个参数，而是用quesize为0或大于0来控制，大于0说明需要启用功能，小于等于0关闭
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] tianxiaoliang commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620795023
##########
File path: datasource/mongo/types.go
##########
@@ -0,0 +1,13 @@
+package mongo
+
+import (
+ "context"
+ "github.com/go-chassis/cari/discovery"
+)
+
+type InstanceRegisterEvent struct {
+ Ctx context.Context
+ Request *discovery.RegisterInstanceRequest
+ isCustomID bool
Review comment:
我觉得isCustomID这个参数没存在必要
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r620798148
##########
File path: datasource/mongo/types.go
##########
@@ -0,0 +1,13 @@
+package mongo
+
+import (
+ "context"
+ "github.com/go-chassis/cari/discovery"
+)
+
+type InstanceRegisterEvent struct {
+ Ctx context.Context
+ Request *discovery.RegisterInstanceRequest
+ isCustomID bool
Review comment:
同上，先生成UUID，再注册
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-service-center] fuziye01 commented on a change in pull request #953: SCB-2094 Fast register with mongo
Posted by GitBox <gi...@apache.org>.
fuziye01 commented on a change in pull request #953:
URL: https://github.com/apache/servicecomb-service-center/pull/953#discussion_r617248607
##########
File path: datasource/mongo/fast_register_inst_service.go
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except request compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to request writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package mongo
+
+var fastRegisterInstanceService *RegisterFastInstanceService
+
+type RegisterFastInstanceService struct {
+ InstEventCh chan *InstanceRegisterEvent
+ FailedInstCh chan *InstanceRegisterEvent
+}
+
+func NewFastRegisterInstanceService() *RegisterFastInstanceService {
+ fastRegisterQueueSize := fastRegConfig.queueSize
Review comment:
需要Configuration()执行过之后，fastRegister.enabled=true之后，才会调用NewFastRegisterInstanceService()方法
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org