You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2024/12/10 16:05:46 UTC
[I] [Bug] Pulsar Functions async function handling is inefficient and could potentially get stuck [pulsar]
lhotari opened a new issue, #23706:
URL: https://github.com/apache/pulsar/issues/23706
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
all released versions
### Minimal reproduce step
When analyzing the Pulsar Functions async function handling code, there are performance concerns and concerns of the processing getting stuck
https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java#L135-L158
### What did you expect to see?
The Pulsar Functions async function handling code should be performant and there should be a reliable solution for back pressure when the `maxPendingAsyncRequests` limit is reached.
### What did you see instead?
Based on the code, it could lead into unnecessary CPU and potentially in the processing getting stuck.
### Anything else?
Using a `LinkedBlockingQueue` doesn't make sense for the concurrency limiting solution at https://github.com/apache/pulsar/blob/0a2ffe4743799dc253814d49fa3fd29b933444ac/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java#L59
This method is more or less pointless: https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstance.java#L135-L158 and causes the extra CPU consumption and the potential of processing getting stuck.
Instead, a simple java.util.concurrent.Semaphore based implementation should be used as a solution for limiting concurrency for implementing `maxPendingAsyncRequests`.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Pulsar Functions async function concurrency limit handling is not optimal when return type is CompletableFuture [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23706:
URL: https://github.com/apache/pulsar/issues/23706#issuecomment-2567356479
addressed by #23708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Pulsar Functions async function concurrency limit handling is not optimal when return type is CompletableFuture [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari closed issue #23706: [Bug] Pulsar Functions async function concurrency limit handling is not optimal when return type is CompletableFuture<Void>
URL: https://github.com/apache/pulsar/issues/23706
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org