You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/01 00:47:48 UTC
[GitHub] [pulsar] merlimat opened a new pull request #7715: Fixed race condition on deleting topic with active readers
merlimat opened a new pull request #7715:
URL: https://github.com/apache/pulsar/pull/7715
### Motivation
When forcifully deleting a topic that has active readers, there is a race condition that lead to a failure in deleting the topic.
The race is
1. Connected readers are closed and removed
2. Since there's no active consumer on a non-durable subscription (eg: the reader sub), then we trigger the removal of the subscription and its cursor. That happens in background, in order to avoid deadlocking. See https://github.com/apache/pulsar/blob/26c49a85dd89d4282a625f02d16959de595d282f/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java#L274
3. Once all consumers are closed, we then proceed in deleting the managed-ledger
4. To delete the managed-ledger, we get the current list of cursors and delete all of them in parallel
5. Since the cursors are removed in background, the list that we got might contain cursors that don't exist anymore when we try to delete them.
```
org.apache.bookkeeper.mledger.ManagedLedgerException: ManagedCursor not found: reader-7d300cd535
at org.apache.pulsar.broker.service.persistent.PersistentTopic$4.deleteLedgerFailed(PersistentTopic.java:884) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$16.deleteCursorFailed(ManagedLedgerImpl.java:2177) ~[org.apache.pulsar-managed-ledger-2.6.0.jar:2.6.0]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncDeleteCursor(ManagedLedgerImpl.java:766) ~[org.apache.pulsar-managed-ledger-2.6.0.jar:2.6.0]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncDelete(ManagedLedgerImpl.java:2156) ~[org.apache.pulsar-managed-ledger-2.6.0.jar:2.6.0]
at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$null$19(PersistentTopic.java:859) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?]
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?]
at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$delete$20(PersistentTopic.java:853) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
at java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:753) ~[?:?]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:731) ~[?:?]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2108) ~[?:?]
at org.apache.pulsar.broker.service.persistent.PersistentTopic.delete(PersistentTopic.java:843) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
at org.apache.pulsar.broker.service.persistent.PersistentTopic.deleteForcefully(PersistentTopic.java:791) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
at org.apache.pulsar.broker.service.BrokerService.deleteTopic(BrokerService.java:719) ~[org.apache.pulsar-pulsar-broker-2.6.0.jar:2.6.0]
Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: ManagedCursor not found: reader-7d300cd535
```
On topic deletion, we should be more tolerant of disappearing cursors and instead proceed as a success case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] srkukarni merged pull request #7715: Fixed race condition on deleting topic with active readers
Posted by GitBox <gi...@apache.org>.
srkukarni merged pull request #7715:
URL: https://github.com/apache/pulsar/pull/7715
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org