You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/12 02:01:33 UTC
[GitHub] [pulsar-client-go] zzzming opened a new pull request #684: Use critical section for batch counters in producer default_router
zzzming opened a new pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684
### Motivation
A group of batch counters are not properly critical section protected in the producer's default_router. Read and update of these counters should be protected as a whole, instead of individually synchronized using Atomic access. Multiple goroutines can access and update these counters at the same time. Basically it is not thread safe.
### Modifications
The change use mutex lock to protect the entire counters during the batch partition decision making process.
Minor improvement includes to evaluate message max size, byte size, and batch window on a needed basis.
### Verifying this change
- [ v] Make sure that the change passes the CI checks.
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): ( no)
- The public API: (no)
- The schema: ( no)
- The default values of configurations: (no)
- The wire protocol: (no)
### Documentation
- Does this pull request introduce a new feature? (/ no)
- If yes, how is the feature documented? (not applicable )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r776849103
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
Update: the implementation in #694 was modified slightly and now `DefaultRouterParallel-2` is no longer slower in the lock-free implementation. New bench comparison is below:
```
name old time/op new time/op delta
DefaultRouterParallel 27.4ns ± 3% 14.9ns ± 6% -45.51% (p=0.000 n=10+9)
DefaultRouterParallel-2 35.2ns ± 1% 33.7ns ± 8% ~ (p=0.161 n=9+9)
DefaultRouterParallel-4 49.4ns ± 6% 28.8ns ± 4% -41.66% (p=0.000 n=10+9)
DefaultRouterParallel-8 58.8ns ± 6% 36.3ns ± 1% -38.32% (p=0.000 n=10+8)
DefaultRouterParallel-16 69.7ns ± 3% 39.5ns ±21% -43.38% (p=0.000 n=10+10)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r775296098
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
Actually, I'm concerned that the use of a mutex will incur overhead and affect performance, especially under concurrent publishing. I've added a bench test to get quantitative numbers in https://github.com/apache/pulsar-client-go/pull/684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r775288008
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
I see that we only ever acquire write access; can this be a regular `sync.Mutex`
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
I see that we only ever acquire write access; can this be a regular `sync.Mutex`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r775298110
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
@zzzming , I've attempted to address the races when updating the default router state in a lock-free manner here: Please take a look and see if you are satisfied: https://github.com/apache/pulsar-client-go/pull/694
Comparing the results from the parallel default router bench test we can see that the use of a mutex has an impact on performance (old==this PR, new==https://github.com/apache/pulsar-client-go/pull/694):
```
name old time/op new time/op delta
DefaultRouterParallel 27.4ns ± 3% 14.8ns ± 2% -45.79% (p=0.000 n=10+8)
DefaultRouterParallel-2 35.2ns ± 1% 41.9ns ± 0% +18.96% (p=0.000 n=9+7)
DefaultRouterParallel-4 49.4ns ± 6% 44.1ns ± 8% -10.84% (p=0.000 n=10+9)
DefaultRouterParallel-8 58.8ns ± 6% 53.2ns ± 3% -9.53% (p=0.000 n=10+8)
DefaultRouterParallel-16 69.7ns ± 3% 51.3ns ± 0% -26.43% (p=0.000 n=10+8)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r775298110
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
@zzzming , I've attempted to address the races when updating the default router state in a lock-free manner here: Please take a look and see if you are satisfied: https://github.com/apache/pulsar-client-go/pull/694
Comparing the results from the parallel default router bench test we can see that the use of a mutex has an impact on performance (old==this PR, new==https://github.com/apache/pulsar-client-go/pull/694):
```
name old time/op new time/op delta
DefaultRouterParallel 27.4ns ± 3% 14.8ns ± 2% -45.79% (p=0.000 n=10+8)
DefaultRouterParallel-2 35.2ns ± 1% 41.9ns ± 0% +18.96% (p=0.000 n=9+7)
DefaultRouterParallel-4 49.4ns ± 6% 44.1ns ± 8% -10.84% (p=0.000 n=10+9)
DefaultRouterParallel-8 58.8ns ± 6% 53.2ns ± 3% -9.53% (p=0.000 n=10+8)
DefaultRouterParallel-16 69.7ns ± 3% 51.3ns ± 0% -26.43% (p=0.000 n=10+8)
```
EDIT: I can't explain why `DefaultRouterParallel-2` is slower on the branch that doesn't use the mutex.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #684: Use critical section for batch counters in producer default_router
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #684:
URL: https://github.com/apache/pulsar-client-go/pull/684#discussion_r775296098
##########
File path: pulsar/default_router.go
##########
@@ -30,6 +31,7 @@ type defaultRouter struct {
lastChangeTimestamp int64
msgCounter uint32
cumulativeBatchSize uint32
+ sync.RWMutex
Review comment:
Actually, I'm concerned that the use of a mutex will incur overhead and affect performance, especially under concurrent publishing. I've added a bench test to get quantitative numbers in https://github.com/apache/pulsar-client-go/pull/693
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org