You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by GitBox <gi...@apache.org> on 2022/11/09 08:28:56 UTC
[GitHub] [dubbo-go] CoolIceV opened a new pull request, #2114: feat: auto concurrency limiter of adaptive service
CoolIceV opened a new pull request, #2114:
URL: https://github.com/apache/dubbo-go/pull/2114
<!-- Thanks for sending a pull request!
Read https://github.com/apache/dubbo-go/blob/master/CONTRIBUTING.md before commit pull request.
-->
**What this PR does**:
Provides an auto concurrency limiter in unstable environments.
This is a component of adaptive service at the provider side.
The component at the consumer size is in #2067.
**Which issue(s) this PR fixes**:
Fixes #1834
**You should pay attention to items below to ensure your pr passes our ci test**
We do not merge pr with ci tests failed
- [ ] All ut passed (run 'go test ./...' in project root)
- [ ] After go-fmt ed , run 'go fmt project' using goland.
- [ ] Golangci-lint passed, run 'sudo golangci-lint run' in project root.
- [ ] Your new-created file needs to have [apache license](https://raw.githubusercontent.com/dubbogo/resources/master/tools/license/license.txt) at the top, like other existed file does.
- [ ] All integration test passed. You can run integration test locally (with docker env). Clone our [dubbo-go-samples](https://github.com/apache/dubbo-go-samples) project and replace the go.mod to your dubbo-go, and run 'sudo sh start_integration_test.sh' at root of samples project root. (M1 Slice is not Support)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on a diff in pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on code in PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#discussion_r1027079434
##########
filter/adaptivesvc/limiter/auto_concurrency_limiter.go:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package limiter
+
+import (
+ "math"
+ "math/rand"
+ "sync"
+ "time"
+)
+
+import (
+ "github.com/dubbogo/gost/log/logger"
+
+ "go.uber.org/atomic"
+)
+
+import (
+ "dubbo.apache.org/dubbo-go/v3/filter/adaptivesvc/limiter/cpu"
+)
+
+var (
+ _ Limiter = (*AutoConcurrency)(nil)
+ _ Updater = (*AutoConcurrencyUpdater)(nil)
+ cpuLoad *atomic.Uint64 = atomic.NewUint64(0) // range from 0 to 1000
+)
+
+// These parameters may need to be different between services
+const (
+ MaxExploreRatio = 0.3
+ MinExploreRatio = 0.06
+ SampleWindowSizeMs = 1000
+ MinSampleCount = 40
+ MaxSampleCount = 500
+ CpuDecay = 0.95
+)
+
+type AutoConcurrency struct {
+ sync.RWMutex
+
+ exploreRatio float64
+ emaFactor float64
+ noLoadLatency float64 //duration
+ maxQPS float64
+ halfSampleIntervalMS int64
+ maxConcurrency uint64
+
+ // metrics of the current round
+ startSampleTimeUs int64
+ lastSamplingTimeUs *atomic.Int64
+ resetLatencyUs int64 // time to reset noLoadLatency
+ remeasureStartUs int64 //time to reset req data (sampleCount, totalSampleUs, totalReqCount)
+ sampleCount int64
+ totalSampleUs int64
+ totalReqCount *atomic.Int64
+
+ inflight *atomic.Uint64
+}
+
+func init() {
+ go cpuproc()
+}
+
+// cpu = cpuᵗ⁻¹ * decay + cpuᵗ * (1 - decay)
+func cpuproc() {
+ ticker := time.NewTicker(time.Millisecond * 500) // same to cpu sample rate
+ defer func() {
+ ticker.Stop()
+ if err := recover(); err != nil {
+ go cpuproc()
+ }
+ }()
+
+ for range ticker.C {
+ usage := cpu.CpuUsage()
+ prevCPU := cpuLoad.Load()
+ curCPU := uint64(float64(prevCPU)*CpuDecay + float64(usage)*(1.0-CpuDecay))
+ logger.Debugf("current cpu usage: %d", curCPU)
+ cpuLoad.Store(curCPU)
+ }
+}
+
+func CpuUsage() uint64 {
Review Comment:
CpuUsage -> CPUUsage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on a diff in pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on code in PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#discussion_r1027079388
##########
filter/adaptivesvc/limiter/auto_concurrency_limiter.go:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package limiter
+
+import (
+ "math"
+ "math/rand"
+ "sync"
+ "time"
+)
+
+import (
+ "github.com/dubbogo/gost/log/logger"
+
+ "go.uber.org/atomic"
+)
+
+import (
+ "dubbo.apache.org/dubbo-go/v3/filter/adaptivesvc/limiter/cpu"
+)
+
+var (
+ _ Limiter = (*AutoConcurrency)(nil)
+ _ Updater = (*AutoConcurrencyUpdater)(nil)
+ cpuLoad *atomic.Uint64 = atomic.NewUint64(0) // range from 0 to 1000
+)
+
+// These parameters may need to be different between services
+const (
+ MaxExploreRatio = 0.3
+ MinExploreRatio = 0.06
+ SampleWindowSizeMs = 1000
+ MinSampleCount = 40
+ MaxSampleCount = 500
+ CpuDecay = 0.95
Review Comment:
nit: CpuDecay -> CPUDecay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] AlexStocks commented on pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
AlexStocks commented on PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#issuecomment-1354821142
@nanfeng1999 lizhenyuan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on a diff in pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on code in PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#discussion_r1027079127
##########
filter/adaptivesvc/limiter/auto_concurrency_limiter.go:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package limiter
+
+import (
+ "math"
+ "math/rand"
+ "sync"
+ "time"
+)
+
+import (
+ "github.com/dubbogo/gost/log/logger"
+
+ "go.uber.org/atomic"
+)
+
+import (
+ "dubbo.apache.org/dubbo-go/v3/filter/adaptivesvc/limiter/cpu"
+)
+
+var (
+ _ Limiter = (*AutoConcurrency)(nil)
+ _ Updater = (*AutoConcurrencyUpdater)(nil)
+ cpuLoad *atomic.Uint64 = atomic.NewUint64(0) // range from 0 to 1000
+)
+
+// These parameters may need to be different between services
+const (
+ MaxExploreRatio = 0.3
+ MinExploreRatio = 0.06
+ SampleWindowSizeMs = 1000
+ MinSampleCount = 40
+ MaxSampleCount = 500
+ CpuDecay = 0.95
+)
+
+type AutoConcurrency struct {
+ sync.RWMutex
+
+ exploreRatio float64
+ emaFactor float64
+ noLoadLatency float64 //duration
+ maxQPS float64
+ halfSampleIntervalMS int64
+ maxConcurrency uint64
+
+ // metrics of the current round
+ startSampleTimeUs int64
+ lastSamplingTimeUs *atomic.Int64
+ resetLatencyUs int64 // time to reset noLoadLatency
+ remeasureStartUs int64 //time to reset req data (sampleCount, totalSampleUs, totalReqCount)
Review Comment:
nit: //time -> // time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on a diff in pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on code in PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#discussion_r1027079065
##########
filter/adaptivesvc/limiter/auto_concurrency_limiter.go:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package limiter
+
+import (
+ "math"
+ "math/rand"
+ "sync"
+ "time"
+)
+
+import (
+ "github.com/dubbogo/gost/log/logger"
+
+ "go.uber.org/atomic"
+)
+
+import (
+ "dubbo.apache.org/dubbo-go/v3/filter/adaptivesvc/limiter/cpu"
+)
+
+var (
+ _ Limiter = (*AutoConcurrency)(nil)
+ _ Updater = (*AutoConcurrencyUpdater)(nil)
+ cpuLoad *atomic.Uint64 = atomic.NewUint64(0) // range from 0 to 1000
+)
+
+// These parameters may need to be different between services
+const (
+ MaxExploreRatio = 0.3
+ MinExploreRatio = 0.06
+ SampleWindowSizeMs = 1000
+ MinSampleCount = 40
+ MaxSampleCount = 500
+ CpuDecay = 0.95
+)
+
+type AutoConcurrency struct {
+ sync.RWMutex
+
+ exploreRatio float64
+ emaFactor float64
+ noLoadLatency float64 //duration
Review Comment:
nit: //duration -> // duration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#issuecomment-1357456396
Merged into `3.0-adaptive-stream` branch to make the running benchmark more convenient. Once the current version is in line with expectations, it will be merged into `3.0` branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] CoolIceV commented on pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
CoolIceV commented on PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#issuecomment-1308398441
This concurrency limiter uses CPU load as a limit switch to automatically calculate the capacity of the service when the load is too high, the effect is as follows.
QPS increases gradually over time. When the CPU load is high, flow limiting is automatically turned on. Then successful requests and CPU load are maintained at a relatively stable level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei merged pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei merged PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] justxuewei commented on a diff in pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
justxuewei commented on code in PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#discussion_r1027079327
##########
filter/adaptivesvc/limiter/auto_concurrency_limiter.go:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package limiter
+
+import (
+ "math"
+ "math/rand"
+ "sync"
+ "time"
+)
+
+import (
+ "github.com/dubbogo/gost/log/logger"
+
+ "go.uber.org/atomic"
+)
+
+import (
+ "dubbo.apache.org/dubbo-go/v3/filter/adaptivesvc/limiter/cpu"
+)
+
+var (
+ _ Limiter = (*AutoConcurrency)(nil)
+ _ Updater = (*AutoConcurrencyUpdater)(nil)
+ cpuLoad *atomic.Uint64 = atomic.NewUint64(0) // range from 0 to 1000
+)
+
+// These parameters may need to be different between services
+const (
+ MaxExploreRatio = 0.3
+ MinExploreRatio = 0.06
+ SampleWindowSizeMs = 1000
+ MinSampleCount = 40
+ MaxSampleCount = 500
+ CpuDecay = 0.95
+)
+
+type AutoConcurrency struct {
+ sync.RWMutex
+
+ exploreRatio float64
+ emaFactor float64
+ noLoadLatency float64 //duration
+ maxQPS float64
+ halfSampleIntervalMS int64
+ maxConcurrency uint64
+
+ // metrics of the current round
+ startSampleTimeUs int64
+ lastSamplingTimeUs *atomic.Int64
+ resetLatencyUs int64 // time to reset noLoadLatency
+ remeasureStartUs int64 //time to reset req data (sampleCount, totalSampleUs, totalReqCount)
+ sampleCount int64
+ totalSampleUs int64
+ totalReqCount *atomic.Int64
+
+ inflight *atomic.Uint64
+}
+
+func init() {
+ go cpuproc()
+}
+
+// cpu = cpuᵗ⁻¹ * decay + cpuᵗ * (1 - decay)
+func cpuproc() {
+ ticker := time.NewTicker(time.Millisecond * 500) // same to cpu sample rate
+ defer func() {
+ ticker.Stop()
+ if err := recover(); err != nil {
+ go cpuproc()
Review Comment:
Add a warning log to let users know what is happening here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo-go] sonarcloud[bot] commented on pull request #2114: feat: auto concurrency limiter of adaptive service
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #2114:
URL: https://github.com/apache/dubbo-go/pull/2114#issuecomment-1326352251
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_dubbo-go&pullRequest=2114)
[](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_dubbo-go&pullRequest=2114&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_dubbo-go&pullRequest=2114) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_dubbo-go&pullRequest=2114&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org