You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/08/12 09:34:58 UTC
[GitHub] [pulsar] TianFeng111 opened a new issue, #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
TianFeng111 opened a new issue, #17076:
URL: https://github.com/apache/pulsar/issues/17076
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
pulsar-2.10.0（release）
### Minimal reproduce step
not clear
### What did you expect to see?
consumer got msgs correctly
### What did you see instead?
we got logs like this:
and monitor like this:
### Anything else?
Eagerly awaiting reply！
Sorry about mosaic in pics .
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1278382673
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] TianFeng111 commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
TianFeng111 commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1214303403
> @TianFeng111 Do you know why so many reset cursor requests?
It’s a delay queue actually，consumers do not consume msgs in a timely manner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] TianFeng111 commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
TianFeng111 commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1214303330
It’s a delay queue actually，consumers do not consume msgs in a timely manner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] TianFeng111 commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
TianFeng111 commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1214792756
>
@codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1213046793
@TianFeng111 Do you know why so many reset cursor requests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ronfarkash commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
ronfarkash commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1245062398
bumping
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] TianFeng111 commented on issue #17076: consumer got some acked msg cause by "Error resetting cursor for slowest consumer"[BUG]
Posted by GitBox <gi...@apache.org>.
TianFeng111 commented on issue #17076:
URL: https://github.com/apache/pulsar/issues/17076#issuecomment-1214555603
> @TianFeng111 Do you know why so many reset cursor requests?
This morning ,we observed that the problem recurs, some supplementary descriptions below
1、It's a delay queue, consumers will delay consumption by one minute.
2、This topics is a partitioned-topic, we only find this problem on partition-2.
3、Firlstly, we observed backlog sudden increaseand storage size reduction，at the same time we got so many reset cursor request in this broker's log.( Log is attached at the bottom)
4、Last week, as this issue described, we foud consumers on this partition got some acked history msg and at the same time, the backlog of the sudden increase returned to normal., which would affect downstream services.
5、But today ,after we foud the phenomenon described in point 3 (problems recurs probably)，the consumers on this partition seemed to be working fine, but the backlog still high .(monitor of this partition attached at the bottom)
6、We just observed the above phenomenon, but do not know it's cause and causality. What caused the initial backlog sudden increase?
Here are some log of this partition's broker and the monitor of this partition(today) below:
2022-08-15T08:57:44,581+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,586+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,586+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,588+0800 [pulsar-io-4-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/30.43.156.124:41976][persistent://ACE/0/0_949687666_20482_2014005_0-partition-0] Creating producer. producerId=274662
2022-08-15T08:57:44,591+0800 [Thread-69266] INFO org.apache.pulsar.broker.service.ServerCnx - [/30.43.156.124:41976] persistent://ACE/0/0_949687666_20482_2014005_0-partition-0 configured with schema false
2022-08-15T08:57:44,591+0800 [Thread-69266] INFO org.apache.pulsar.broker.service.ServerCnx - [/30.43.156.124:41976] Created new producer: Producer{topic=PersistentTopic{topic=persistent://ACE/0/0_949687666_20482_2014005_0-partition-0}, client=/30.43.156.124:41976, producerName=ACE_PRODUCER_1001_10821_5714_3813511, producerId=274662}
2022-08-15T08:57:44,592+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,592+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,597+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,597+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,600+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Creating a new ledger
2022-08-15T08:57:44,600+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[65, 67, 69, 47, 48, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 48, 95, 57, 52, 57, 54, 56, 55, 54, 54, 54, 95, 50, 48, 52, 56, 50, 95, 50, 48, 49, 52, 48, 48, 53, 95, 48, 45, 112, 97, 114, 116, 105, 116, 105, 111, 110, 45, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds
2022-08-15T08:57:44,600+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] WARN org.apache.bookkeeper.client.BookieWatcherImpl - New ensemble: [11.177.85.74:3181, 11.186.71.172:3181, 11.177.86.228:3181, 11.177.87.59:3181] is not adhering to Placement Policy. quarantinedBookies: []
2022-08-15T08:57:44,604+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,604+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,608+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,608+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,612+0800 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [11.177.85.74:3181, 11.186.71.172:3181, 11.177.86.228:3181, 11.177.87.59:3181] for ledger: 107006
2022-08-15T08:57:44,612+0800 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Created new ledger 107006
2022-08-15T08:57:44,613+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,613+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,618+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,618+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Initiate reset position to 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,618+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=ACE/0/persistent/0_949687666_20482_2014005_0-partition-0, name=ACE_SUBSCRIPTION_1001_10804_2515, ackPos=60734:0, readPos=60734:1} to 107006:-1 since ledger consumed completely
2022-08-15T08:57:44,618+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Ledger 60734 contains the current last confirmed entry 60734:0, and it is going to be deleted
2022-08-15T08:57:44,622+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] End TrimConsumedLedgers. ledgers=1 totalSize=0
2022-08-15T08:57:44,622+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] Removing ledger 60734 - size: 552
2022-08-15T08:57:44,622+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_949687666_20482_2014005_0-partition-0] reset position to 60734:1 before current read position 60734:1 on cursor ACE_SUBSCRIPTION_1001_10804_2515
2022-08-15T08:57:44,623+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_949687666_20482_2014005_0-partition-0] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10804_2515]
java.lang.NullPointerException: null
2022-08-15T08:57:44,623+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,623+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Initiate reset position to 88545:0 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,628+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] reset position to 88545:0 before current read position 105731:17659 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,628+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] WARN org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Failed to mark delete while trimming data ledgers: Reset cursor in progress - unable to mark delete position 88545:-1
2022-08-15T08:57:44,628+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Initiate reset position to 88545:0 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,632+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] End TrimConsumedLedgers. ledgers=11 totalSize=915274097
2022-08-15T08:57:44,632+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Removing ledger 87121 - size: 102888483
2022-08-15T08:57:44,632+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] reset position to 88545:0 before current read position 105731:17659 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,632+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Initiate reset position to 90184:0 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,637+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] WARN org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] Failed to mark delete while trimming data ledgers: Reset cursor in progress - unable to mark delete position 90184:-1
2022-08-15T08:57:44,637+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [ACE/0/persistent/0_0_20483_isasync_0-partition-2] reset position to 90184:0 skipping from current read position 88545:0 on cursor ACE_SUBSCRIPTION_1001_10597_2511
2022-08-15T08:57:44,638+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,638+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,639+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,639+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,639+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,640+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,640+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,640+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,640+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,641+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2022-08-15T08:57:44,642+0800 [pulsar-backlog-quota-checker-26-1] INFO org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type message_age exceeded for topic [persistent://ACE/0/0_0_20483_isasync_0-partition-2]. Applying [consumer_backlog_eviction] policy
2022-08-15T08:57:44,642+0800 [pulsar-backlog-quota-checker-26-1] ERROR org.apache.pulsar.broker.service.BacklogQuotaManager - [persistent://ACE/0/0_0_20483_isasync_0-partition-2] Error resetting cursor for slowest consumer [ACE_SUBSCRIPTION_1001_10597_2511]
java.lang.NullPointerException: null
at org.apache.pulsar.broker.service.BacklogQuotaManager.dropBacklogForTimeLimit(BacklogQuotaManager.java:218) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:98) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.lambda$null$79(BrokerService.java:1768) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_292]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$81(BrokerService.java:1766) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_292]
at org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$78(BrokerService.java:1750) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:413) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:185) ~[org.apache.pulsar-pulsar-common-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1748) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1759) ~[org.apache.pulsar-pulsar-broker-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.10.0.jar:2.10.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.4.jar:4.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_292]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org