You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/27 07:12:22 UTC
[GitHub] [pulsar] Technoboy- opened a new pull request, #15342: Use shrink map for message redelivery.
Technoboy- opened a new pull request, #15342:
URL: https://github.com/apache/pulsar/pull/15342
### Motivation
Sometimes the messages sent to consumers are delayed over 100ms and we find that the CPU is wasted on :
```
"BookKeeperClientWorker-OrderedExecutor-4-0" #64 prio=5 os_prio=0 cpu=2447920216.81ms elapsed=3636758.63s tid=0x00007f48bdc07290 nid=0xae runnable [0x00007f47ee2e2000]
java.lang.Thread.State: RUNNABLE
at org.apache.bookkeeper.util.collections.ConcurrentLongLongPairHashMap.forEach(ConcurrentLongLongPairHashMap.java:197)
at org.apache.pulsar.broker.service.persistent.MessageRedeliveryController.containsStickyKeyHashes(MessageRedeliveryController.java:97)
at org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers.getRestrictedMaxEntriesForConsumer(PersistentStickyKeyDispatcherMultipleConsumers.java:335)
at org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers.sendMessagesToConsumers(PersistentStickyKeyDispatcherMultipleConsumers.java:232)
at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:480)
- locked <0x0000100117401718> (a org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers)
at org.apache.bookkeeper.mledger.impl.OpReadEntry.lambda$checkReadCompletion$2(OpReadEntry.java:156)
```
Then we find there are many empty datas in the map from dump :
<img width="1056" alt="image" src="https://user-images.githubusercontent.com/6297296/165461606-9e803243-174b-4fc5-835a-5e9da9bca48a.png">
As #14515 has introduced shrink map, it's better to use it to avoid iterator the empty data.
### Documentation
- [x] `no-need-doc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #15342: [improve][broker] Use shrink map for message redelivery.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #15342:
URL: https://github.com/apache/pulsar/pull/15342#issuecomment-1110652739
I have added release labels, it's a serious problem that will cause the thread `BookKeeperClientWorker-OrderedExecutor` to 100% CPU usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #15342: [improve][broker] Use shrink map for message redelivery.
Posted by GitBox <gi...@apache.org>.
Technoboy- merged PR #15342:
URL: https://github.com/apache/pulsar/pull/15342
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org