[PR] [improve][client] Don't start a batch receive on an internal consumer without queued messages [pulsar]

["lhotari (via GitHub)" <gi...@apache.org>]

lhotari opened a new pull request, #26966:
URL: https://github.com/apache/pulsar/pull/26966

   ### Motivation
   
   `MultiTopicsConsumerImpl` receives from each internal consumer in a loop. After a receive that returned messages, it starts a batch receive (#16336). For internal consumers, the batch receive's policy is half the receiver queue or a 1 ms timeout.
   
   At low rates, the internal consumer's queue is usually empty after a message, so the batch receive waits for its timeout. That costs, for each received message:
   - a timer task;
   - a hand-off to the internal executor, which completes the batch receive with no messages;
   - a `receiveAsync()` for the next message.
   
   In a profile of the Pulsar Performance Testing Framework's IoT telemetry applications (2,000 multi-topic consumers at 10 msg/s each), the timeout task alone (`ConsumerBase.pendingBatchReceiveTask`) was 2.7 % of the process's CPU samples, besides the executor hand-offs.
   
   ### Modifications
   
   - The receive loop starts a batch receive only when the internal consumer has queued messages, and waits for the next message with `receiveAsync()` otherwise.
   - Under load the queue has messages, so the batch receive takes them as before.
   - The size check is only a hint: a message that arrives after it is received by `receiveAsync()`, which polls the queue on the internal consumer's executor.
   - The user's own batch receive policy isn't affected.
   
   ### Measurements
   
   Two interleaved pairs of unprofiled runs of the IoT telemetry scenario with the Pulsar Performance Testing Framework, without and with this change. The runs with the change also had the broker-only changes of #26948, which don't affect the applications' CPU:
   
   | | Without | With |
   |---|---|---|
   | Applications' CPUs (2,000 consumers, 20k msg/s) | 2.65, 2.62 | 2.25, 2.30 |
   | Publish to consume p99 (worst application) | 14, 14 ms | 12, 12 ms |
   
   In the applications' profile with the change, `pendingBatchReceiveTask` has no samples.
   
   ### Verifying this change
   
   This change is already covered by existing tests, such as `MultiTopicsConsumerTest`, `MultiTopicsReaderTest` and `PartitionedProducerConsumerTest`, which pass.
   
   ### Does this pull request potentially affect one of the following parts:
   
   - [ ] Dependencies (add or upgrade a dependency)
   - [ ] The public API
   - [ ] The schema
   - [ ] The default values of configurations
   - [ ] The threading model
   - [ ] The binary protocol
   - [ ] The REST endpoints
   - [ ] The admin CLI options
   - [ ] The metrics
   - [ ] Anything that affects deployment
   
   This change was developed with the assistance of Claude Code.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [improve][client] Don't start a batch receive on an internal consumer without queued messages [pulsar]

["merlimat (via GitHub)" <gi...@apache.org>]

merlimat merged PR #26966:
URL: https://github.com/apache/pulsar/pull/26966


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org