You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/27 18:56:51 UTC
[GitHub] [pulsar] lhotari opened a new pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
lhotari opened a new pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820
### Motivation
- since `java.util.HashMap` is used, there is a potential issue that the datastructure gets corrupted.
There is existing code that gives the impression that the producer cache is used by multiple threads.
For example:
https://github.com/apache/pulsar/blob/4046a6f9b73d635773c8696a8b9f28be1ce226cc/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java#L551-L559
### Modifications
use ConcurrentHashMap instead of HashMap for caching producers in ContextImpl class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari merged pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
lhotari merged pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-908672692
@lhotari - I took a look, and I think the current design is intentional to allow for the `PulsarClientException` to be thrown without special handling in the `computeIfAbsent` method, which does not allow for checked exceptions to be thrown from the `mappingFunction`. I think refactoring it will require more work than I first thought, so I won't be able to send a patch today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-908672692
@lhotari - I took a look, and I think the current design is intentional to allow for the `PulsarClientException` to be thrown without special handling in the `computeIfAbsent` method, which does not allow for checked exceptions to be thrown from the `mappingFunction`. I think refactoring it will require more work than I first thought, so I won't be able to send a patch today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-991628151
Removed the release/2.7.4 label due to in branch-2.7, no `publishProducers` for `ContextImpl.java`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-908629312
> LGTM. One question: wouldn't it also make sense to modify the `get` and `putIfAbsent` logic to instead use `computeIfAbsent` on this concurrent hashmap? That would prevent the unnecessary creation of the `newProducer` that currently gets closed if another thread created an equivalent producer.
yes, that's a valid point. It would be good to do that in a separate PR since the current logic should work as long as a ConcurrentHashMap is used. @michaeljmarshall Would you like to create a PR for that improvement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-991628151
Removed the release/2.7.4 label due to in branch-2.7, no `publishProducers` for `ContextImpl.java`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820#issuecomment-908629312
> LGTM. One question: wouldn't it also make sense to modify the `get` and `putIfAbsent` logic to instead use `computeIfAbsent` on this concurrent hashmap? That would prevent the unnecessary creation of the `newProducer` that currently gets closed if another thread created an equivalent producer.
yes, that's a valid point. It would be good to do that in a separate PR since the current logic should work as long as a ConcurrentHashMap is used. @michaeljmarshall Would you like to create a PR for that improvement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari merged pull request #11820: [Functions] ConcurrentHashMap should be used for caching producers
Posted by GitBox <gi...@apache.org>.
lhotari merged pull request #11820:
URL: https://github.com/apache/pulsar/pull/11820
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org