You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2025/02/28 08:52:52 UTC
[I] [Enhancement] Configurable dispatch rate limiter backoff to reduce the 1-second latency penalty when limits are reached [pulsar]
lhotari opened a new issue, #24036:
URL: https://github.com/apache/pulsar/issues/24036
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
The current dispatch rate limiter implementation introduces a fixed 1-second additional latency when the rate limit is reached. This delay is hardcoded as `MESSAGE_RATE_BACKOFF_MS = 1000` in the `PersistentTopic` class:
https://github.com/apache/pulsar/blob/e547beaa6dad72359bb5b8d30ce9fa80a6989f71/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L240
### Solution
Simply making `MESSAGE_RATE_BACKOFF_MS` configurable would be insufficient for several reasons:
1. **Token Replenishment Frequency**: Tokens are currently added to the rate limiter once per second. If the backoff time were reduced (e.g., to 100ms) without changing the token addition frequency, dispatchers would check for available tokens too frequently, wasting CPU resources.
2. **Implementation Differences**:
- **Classic RateLimiterImpl**: Uses a scheduled job to add permits periodically (controlled by `ratePeriod`).
- **PIP-322 AsyncTokenBucket**: Calculates tokens on-demand when the limiter is used (controlled by `addTokensResolutionNanos`), enabling better scaling to millions of rate limiter instances without any overhead of scheduled jobs.
The backoff mechanism is triggered in the `AbstractBaseDispatcher` class:
https://github.com/apache/pulsar/blob/e547beaa6dad72359bb5b8d30ce9fa80a6989f71/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractBaseDispatcher.java#L417-L428
Possible solution:
- Making `MESSAGE_RATE_BACKOFF_MS` configurable in dispatch rate limiter configurations.
- One possibility would be to internally configure it based on the rate period. However currently rate period's time unit is fixed to seconds for configuration:
- https://github.com/apache/pulsar/blob/master/pulsar-client-admin-api/src/main/java/org/apache/pulsar/common/policies/data/DispatchRate.java
- Another problem is that for users, it's more natural to configure the rate for 1 second or 1 minute. It could be better way to configure the backoff time.
Fairness (as defined in [fair queuing](https://en.wikipedia.org/wiki/Fair_queuing)) Considerations:
The current fixed 1-second backoff, which matches the 1-second token replenishment interval, may inadvertently provide some level of fairness in resource allocation. Changing this ratio could impact the fairness properties of the system.
Fairness is currently unaddressed in Pulsar's dispatch rate limiting. Addressing fairness is crucial to improving Pulsar's rate limiting and capacity management capabilities as described in the [Pulsar 4.0 blog post](https://pulsar.apache.org/blog/2024/10/24/announcing-apache-pulsar-4-0/#rate-limiting-and-capacity-management-in-modern-messaging-platforms).
To make Pulsar competitive with Confluent's Kora, which according to the [Kora paper](https://www.vldb.org/pvldb/vol16/p3822-povzner.pdf) includes features like "backpressure and auto-tuning" and "dynamic quota management," we need to enhance Pulsar's approach to fairness in resource allocation, including in dispatch rate limiting.
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] Configurable dispatch rate limiter backoff to reduce the 1-second latency penalty when limits are reached [pulsar]
Posted by "1991santhu (via GitHub)" <gi...@apache.org>.
1991santhu commented on issue #24036:
URL: https://github.com/apache/pulsar/issues/24036#issuecomment-5175046461
The fairness intuition in the description seems right to me, and I think it can be stated more concretely — which also suggests why decoupling the two intervals is harder than it looks.
With backoff fixed at exactly the replenishment period, every dispatcher that hits the limit parks for the same duration and wakes at the same moment tokens are added. So the "fairness" is not really fairness — it is a synchronised race at each replenishment tick, and the winners are decided by wake order and scheduler behaviour. It approximates fairness only because the race is re-run every second and, over enough ticks, the noise averages out. That is why it feels fair at 1s and would degrade if the ratio changed: shortening backoff to 100ms without touching replenishment gives ten races per token grant, nine of which find an empty bucket, which is the CPU waste noted in point 1 — but it also concentrates the winners among whichever dispatchers happen to poll first after the grant, so short backoff makes the allocation *less* even, not just noisier.
That points at a different fix from making the constant configurable. If the wait were spread across the replenishment window rather than aligned to it — each blocked dispatcher waiting a randomised fraction of the remaining period — arrivals would distribute across the window instead of arriving in a burst. Latency at the limit becomes lower on average (you no longer always pay the full second), CPU cost stays bounded because each dispatcher still wakes about once per period, and the allocation gets more even rather than less, because dispatchers are no longer competing in a single instant.
That framing also makes the token-bucket asymmetry easier to handle: for the classic `RateLimiterImpl` the window is the scheduled `ratePeriod`, and for the PIP-322 implementation it is however long until the bucket next has capacity, which is computable rather than fixed. Both give a bound to randomise within, without requiring the two intervals to stay equal.
One caveat on the fairness goal itself: spreading arrivals removes the synchronisation artefact but still allocates by arrival order, so a dispatcher with more consumers keeps taking a proportionally larger share. If the aim is the Kora-style fairness in the linked paper, that likely needs explicit per-entity accounting rather than an improved wait — worth separating the two so the backoff change is not expected to deliver fairness it cannot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org