You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2020/04/09 10:44:28 UTC
[GitHub] [servicecomb-kie] tianxiaoliang opened a new pull request #134:
feature: quota management
tianxiaoliang opened a new pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406139918
##########
File path: server/resource/v1/kv_resource.go
##########
@@ -55,6 +56,16 @@ func (r *KVResource) Post(rctx *restful.Context) {
WriteErrResponse(rctx, http.StatusBadRequest, err.Error())
return
}
+ err = quota.PreCreate("", kv.Domain, "", 1)
Review comment:
配额保证只是弱保证，超过一点没有关系是吧
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] tianxiaoliang commented on a change in pull
request #134: feature: quota management
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406136023
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
+)
+
+//BuildInManager read env config to max config item number, and db total usage
+// it is not a centralized QMS.
+type BuildInManager struct {
+}
+
+//GetQuotas get usage and quota
+func (m *BuildInManager) GetQuotas(serviceName, domain string) ([]*quota.Quota, error) {
+ max := archaius.GetInt64(QuotaConfigKey, DefaultQuota)
+ total, err := service.KVService.Total(context.TODO(), domain)
+ if err != nil {
+ openlogging.Error("find quotas failed: " + err.Error())
+ return nil, err
+ }
+ return []*quota.Quota{{
Review comment:
这个是按照公司内部的一个quota标准制定的，他给的接口返回list……
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] zhulijian1 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
zhulijian1 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406131514
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
+)
+
+//BuildInManager read env config to max config item number, and db total usage
+// it is not a centralized QMS.
+type BuildInManager struct {
+}
+
+//GetQuotas get usage and quota
+func (m *BuildInManager) GetQuotas(serviceName, domain string) ([]*quota.Quota, error) {
+ max := archaius.GetInt64(QuotaConfigKey, DefaultQuota)
+ total, err := service.KVService.Total(context.TODO(), domain)
+ if err != nil {
+ openlogging.Error("find quotas failed: " + err.Error())
+ return nil, err
+ }
+ return []*quota.Quota{{
Review comment:
这里为啥是个list，而不是map[resourcename]{Limit: max, Used: total,}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] zhulijian1 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
zhulijian1 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406124899
##########
File path: server/service/mongo/kv/kv_service.go
##########
@@ -257,3 +258,14 @@ func (s *Service) Get(ctx context.Context, domain, project, id string, options .
}
return findKVDocByID(ctx, domain, project, id)
}
+
+func (s *Service) Total(ctx context.Context, domain string) (int64, error) {
+ collection := session.GetDB().Collection(session.CollectionKV)
Review comment:
kv_service.go里不直接操作数据库，都在kv_dao.go里
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] tianxiaoliang commented on a change in pull
request #134: feature: quota management
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406137022
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
Review comment:
简单的build in配额管理，只要能够限制住配置数量即可，这个env名字可以改名叫quota_kv
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406128835
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
+)
+
+//BuildInManager read env config to max config item number, and db total usage
+// it is not a centralized QMS.
+type BuildInManager struct {
+}
+
+//GetQuotas get usage and quota
+func (m *BuildInManager) GetQuotas(serviceName, domain string) ([]*quota.Quota, error) {
+ max := archaius.GetInt64(QuotaConfigKey, DefaultQuota)
+ total, err := service.KVService.Total(context.TODO(), domain)
+ if err != nil {
+ openlogging.Error("find quotas failed: " + err.Error())
Review comment:
domain或者sevice name是不是打印出来，不然并发量大了，不知道是哪个请求发生的错误
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] zhulijian1 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
zhulijian1 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406567696
##########
File path: server/service/mongo/kv/kv_service.go
##########
@@ -257,3 +257,12 @@ func (s *Service) Get(ctx context.Context, domain, project, id string, options .
}
return findKVDocByID(ctx, domain, project, id)
}
+
+func (s *Service) Total(ctx context.Context, domain string) (int64, error) {
+ n, err := total(ctx, domain)
+ if err != nil {
+ openlogging.Error("find total number: " + err.Error())
Review comment:
日志重复了，直接返回total(ctx, domain)就可以。另外要不要像其它有几个方法一样加个ctx, _ = context.WithTimeout(ctx, session.Timeout)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406140547
##########
File path: server/handler/quota_handler.go
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package handler
+
+import (
Review comment:
当前这个handler什么都不做，后续功能待实现吗？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] tianxiaoliang merged pull request #134: feature:
quota management
Posted by GitBox <gi...@apache.org>.
tianxiaoliang merged pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406131648
##########
File path: server/service/mongo/kv/kv_service.go
##########
@@ -257,3 +258,14 @@ func (s *Service) Get(ctx context.Context, domain, project, id string, options .
}
return findKVDocByID(ctx, domain, project, id)
}
+
+func (s *Service) Total(ctx context.Context, domain string) (int64, error) {
+ collection := session.GetDB().Collection(session.CollectionKV)
+ filter := bson.M{"domain": domain}
+ total, err := collection.CountDocuments(ctx, filter)
+ if err != nil {
Review comment:
可以直接return collection.CountDocuments(ctx, filter)就行了
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] tianxiaoliang commented on a change in pull
request #134: feature: quota management
Posted by GitBox <gi...@apache.org>.
tianxiaoliang commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406563287
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
+)
+
+//BuildInManager read env config to max config item number, and db total usage
+// it is not a centralized QMS.
+type BuildInManager struct {
+}
+
+//GetQuotas get usage and quota
+func (m *BuildInManager) GetQuotas(serviceName, domain string) ([]*quota.Quota, error) {
+ max := archaius.GetInt64(QuotaConfigKey, DefaultQuota)
Review comment:
嗯 把持动态吧，这样随时可以动态变更
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406138161
##########
File path: server/resource/v1/kv_resource.go
##########
@@ -55,6 +56,16 @@ func (r *KVResource) Post(rctx *restful.Context) {
WriteErrResponse(rctx, http.StatusBadRequest, err.Error())
return
}
+ err = quota.PreCreate("", kv.Domain, "", 1)
+ if err != nil {
+ if err == quota.ErrReached {
+ WriteErrResponse(rctx, http.StatusUnprocessableEntity, err.Error())
Review comment:
是否可以记录下没有配额的日志
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] zhulijian1 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
zhulijian1 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406129128
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
Review comment:
配额可能不止一个，比如未来可能有view的quota什么的，这里不要只叫quota，指明是kv的quota
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-kie] aseTo2016 commented on a change in pull request
#134: feature: quota management
Posted by GitBox <gi...@apache.org>.
aseTo2016 commented on a change in pull request #134: feature: quota management
URL: https://github.com/apache/servicecomb-kie/pull/134#discussion_r406122783
##########
File path: server/plugin/qms/quota.go
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package qms
+
+import (
+ "context"
+ "github.com/apache/servicecomb-kie/server/service"
+ "github.com/go-chassis/go-archaius"
+ "github.com/go-chassis/go-chassis/pkg/backends/quota"
+ "github.com/go-mesh/openlogging"
+)
+
+const (
+ DefaultQuota = 10000
+ QuotaConfigKey = "QUOTA_CONFIG"
+)
+
+//BuildInManager read env config to max config item number, and db total usage
+// it is not a centralized QMS.
+type BuildInManager struct {
+}
+
+//GetQuotas get usage and quota
+func (m *BuildInManager) GetQuotas(serviceName, domain string) ([]*quota.Quota, error) {
+ max := archaius.GetInt64(QuotaConfigKey, DefaultQuota)
Review comment:
max是动态的吗？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services