You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/06/07 06:57:43 UTC
[GitHub] [pulsar-client-go] fishpenguin opened a new pull request #533: [issue516]Add lock for compressionProviders to fix data race problem
fishpenguin opened a new pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533
Fixes #516
*(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
Master Issue: #<xyz>
### Motivation
The map compressionProviders without mutex will cause data race, so I add a mutex here.
### Modifications
Add sync.mutex for compressionProviders
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
### Documentation
- Does this pull request introduce a new feature? (no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] cckellogg commented on a change in pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
cckellogg commented on a change in pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533#discussion_r647596143
##########
File path: pulsar/consumer_partition.go
##########
@@ -1062,16 +1065,19 @@ func getPreviousMessage(mid trackingMessageID) trackingMessageID {
}
func (pc *partitionConsumer) Decompress(msgMeta *pb.MessageMetadata, payload internal.Buffer) (internal.Buffer, error) {
+ pc.providersMutex.Lock()
Review comment:
Are there any overhead concerns for now having to grab a new lock each time we receive a message?
cc @merlimat
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] dferstay commented on a change in pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
dferstay commented on a change in pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533#discussion_r658280811
##########
File path: pulsar/consumer_partition.go
##########
@@ -1062,16 +1065,19 @@ func getPreviousMessage(mid trackingMessageID) trackingMessageID {
}
func (pc *partitionConsumer) Decompress(msgMeta *pb.MessageMetadata, payload internal.Buffer) (internal.Buffer, error) {
+ pc.providersMutex.Lock()
Review comment:
@fishpenguin , @cckellogg ,
I'm thinking an [RWMutex](https://golang.org/pkg/sync/#RWMutex) would generate less contention. The majority of accesses require the read lock; the write lock is only required when initialising the provider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] fishpenguin commented on pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
fishpenguin commented on pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533#issuecomment-870411938
@dferstay @merlimat please review my code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] fishpenguin commented on pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
fishpenguin commented on pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533#issuecomment-870411938
@dferstay @merlimat please review my code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] cckellogg commented on a change in pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
cckellogg commented on a change in pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533#discussion_r646916865
##########
File path: pulsar/consumer_partition.go
##########
@@ -1072,6 +1076,7 @@ func (pc *partitionConsumer) Decompress(msgMeta *pb.MessageMetadata, payload int
pc.compressionProviders[msgMeta.GetCompression()] = provider
}
+ pc.providersMutex.UnLock()
Review comment:
There is spot above where the code could return without releasing the lock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy merged pull request #533: [issue 516]Add lock for compressionProviders to fix data race problem
Posted by GitBox <gi...@apache.org>.
wolfstudy merged pull request #533:
URL: https://github.com/apache/pulsar-client-go/pull/533
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org