You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "smbecker (via GitHub)" <gi...@apache.org> on 2024/05/22 20:07:59 UTC
[I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
smbecker opened a new issue, #217:
URL: https://github.com/apache/pulsar-dotpulsar/issues/217
### Is your feature request related to a problem? Please describe
I am looking to leverage [KEDA](https://keda.sh/) to support auto-scaling my Pulsar consumers based on topic volumes. One of the [best practices](https://keda.sh/docs/2.14/concepts/scaling-deployments/#leverage-the-container-lifecycle) with regards to properly handling requests to scale down would involve preventing the `MessageProcessor` from receiving new messages while allowing any messages that are currently in-flight (or have already been processed but not yet acknowledged) from continuing and sending the acknowledgement back to the broker. After reviewing the code, I can't seem to find a way to handle that.
### Describe the solution you'd like and alternatives you've considered
Add support for a mechanism to prevent the `MessageProcessor` from picking up new messages while allowing already pulled messages to continue. In the meantime, I can leverage [scale as job](https://keda.sh/docs/2.14/concepts/scaling-jobs/) to work-around this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
Posted by "entvex (via GitHub)" <gi...@apache.org>.
entvex closed issue #217: Cancel MessageProcessor while allowing in-flight messages to succeed
URL: https://github.com/apache/pulsar-dotpulsar/issues/217
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
Posted by "blankensteiner (via GitHub)" <gi...@apache.org>.
blankensteiner commented on issue #217:
URL: https://github.com/apache/pulsar-dotpulsar/issues/217#issuecomment-2144504836
Hi @smbecker
Just to make sure I understand, you want a "nice shutdown" functionality sort of like Complete+Completion on ISendChannel?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
Posted by "smbecker (via GitHub)" <gi...@apache.org>.
smbecker commented on issue #217:
URL: https://github.com/apache/pulsar-dotpulsar/issues/217#issuecomment-2146058143
> Hi @smbecker Just to make sure I understand, you want a "nice shutdown" functionality sort of like Complete+Completion on ISendChannel?
I don't think it really has anything to do with `ISendChannel`. What I was looking for was a way to provide a different `CancellationToken` used for [Process](https://github.com/apache/pulsar-dotpulsar/blob/master/src/DotPulsar/Internal/MessageProcessor.cs#L84) than the one used for [processing each message](https://github.com/apache/pulsar-dotpulsar/blob/master/src/DotPulsar/Internal/MessageProcessor.cs#L141) or acknowledging [https://github.com/apache/pulsar-dotpulsar/blob/master/src/DotPulsar/Internal/MessageProcessor.cs#L154-L162]. The idea being that when `MessageProcessor.Process` is cancelled, then any messages that have already been received (and processing started) could continue to process until completion and then be acknowledged before ultimately terminating.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
Posted by "entvex (via GitHub)" <gi...@apache.org>.
entvex commented on issue #217:
URL: https://github.com/apache/pulsar-dotpulsar/issues/217#issuecomment-2431344241
No feedback received.
3.4.0 with the ShutdownGracePeriod feature will release soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]
Posted by "entvex (via GitHub)" <gi...@apache.org>.
entvex commented on issue #217:
URL: https://github.com/apache/pulsar-dotpulsar/issues/217#issuecomment-2378689961
Hi @smbecker. If you want you can compile DotPulsar and try the new ShutdownGracePeriod feature.
https://github.com/apache/pulsar-dotpulsar/blob/master/CHANGELOG.md
If you do, please tell us how it works for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org