You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "void-ptr974 (via GitHub)" <gi...@apache.org> on 2026/06/13 04:00:41 UTC

[PR] [improve][broker] Avoid O(n) pending ack scans for size lookup [pulsar]

void-ptr974 opened a new pull request, #26019:
URL: https://github.com/apache/pulsar/pull/26019

   ### Motivation
   
   `PendingAcksMap.size()` currently computes the pending ack count by scanning all per-ledger pending ack maps. This makes a simple size lookup O(number of ledgers), even though callers only need the total number of pending ack entries.
   
   On consumers with many pending acknowledgments, this adds unnecessary CPU cost to paths that inspect or drain pending acks.
   
   ### Modifications
   
   Maintain a cached pending ack count in `PendingAcksMap`.
   
   The counter is updated when pending ack entries are added, removed, range-removed, or cleared. With this, `PendingAcksMap.size()` becomes an O(1) read instead of scanning nested maps.
   
   The change keeps the existing `TreeMap` structure and only optimizes size accounting.
   
   ### Verifying this change
   
   - `./gradlew :pulsar-broker:test --tests org.apache.pulsar.broker.service.PendingAcksMapTest`
   - `./gradlew :pulsar-broker:checkstyleMain`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker] Avoid O(n) pending ack scans for size lookup [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #26019:
URL: https://github.com/apache/pulsar/pull/26019#issuecomment-4697618404

   > Entries	Ledgers	Map scan	Cached counter	Speedup
   > 65,536	1,024	6,454.5 ns/op	23.8 ns/op	271.7x
   > 1,048,576	16,384	687,867.5 ns/op	23.7 ns/op	29,011.7x
   
   In practice a single consumer won't be handling very many ledgers at once so this isn't a bottleneck or major waste of CPU. Regardless, this is a useful improvement.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker] Avoid O(n) pending ack scans for size lookup [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on code in PR #26019:
URL: https://github.com/apache/pulsar/pull/26019#discussion_r3408765879


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PendingAcksMap.java:
##########
@@ -160,7 +164,7 @@ public boolean addPendingAckIfAllowed(long ledgerId, long entryId, int remaining
     public long size() {
         try {
             readLock.lock();
-            return pendingAcks.values().stream().mapToInt(TreeMap::size).sum();
+            return size;

Review Comment:
   We could remove the read lock here if we make the size as `volatile` (and still update it while holding the lock).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker] Avoid O(n) pending ack scans for size lookup [pulsar]

Posted by "void-ptr974 (via GitHub)" <gi...@apache.org>.
void-ptr974 commented on code in PR #26019:
URL: https://github.com/apache/pulsar/pull/26019#discussion_r3408871408


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PendingAcksMap.java:
##########
@@ -160,7 +164,7 @@ public boolean addPendingAckIfAllowed(long ledgerId, long entryId, int remaining
     public long size() {
         try {
             readLock.lock();
-            return pendingAcks.values().stream().mapToInt(TreeMap::size).sum();
+            return size;

Review Comment:
   Done, thanks for the suggestion. This makes `size()` even cheaper.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker] Avoid O(n) pending ack scans for size lookup [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat merged PR #26019:
URL: https://github.com/apache/pulsar/pull/26019


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org