You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by pe...@apache.org on 2021/12/21 01:33:49 UTC
[pulsar] 13/18: Fix update ledger list to znode version mismatch failed, ledger not delete (#12015)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.9
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 71ba8ef2afda1a71663d5b2e7f415ba73dc3ea2c
Author: Hang Chen <ch...@apache.org>
AuthorDate: Mon Nov 29 12:22:55 2021 +0800
Fix update ledger list to znode version mismatch failed, ledger not delete (#12015)
### Motivation
When Zookeeper throws `Failed to update ledger list. z-node version mismatch. Closing managed ledger` exception when update ZNode list, it will not clean up the created ledger, which will lead to the new created ledger not be indexed to the topic managedLedger list, and can't be cleanup as topic retention. What's more, it will cause ZNode number increase in Zookeeper if the `z-node version mismatch` exception keeping throw out. The exception list as follow:
```
10:44:29.017 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [test/test/persistent/test_v1-partition-4] Created new ledger 67311140
10:44:29.018 [bookkeeper-ml-workers-OrderedExecutor-2-0] ERROR org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [test/test/persistent/test_v1-partition-4] Failed to update ledger list. z-node version mismatch. Closing managed ledger
10:44:29.018 [bookkeeper-ml-workers-OrderedExecutor-2-0] INFO org.apache.pulsar.broker.service.Producer - Disconnecting producer: Producer{topic=PersistentTopic{topic=persistent://test/test/test_v1-partition-4}, client=/10.1.2.3:38938, producerName=pulsar-101-1123, producerId=20}
```
### Modification
1. When updating ZNode list failed, delete the created ledger from broker cache and BookKeeper, regardless of whether the exception type is BadVersionException or not.
(cherry picked from commit e7b0e3dbc5b6a05e955dc2cad034de0487463150)
---
.../bookkeeper/mledger/impl/ManagedLedgerImpl.java | 30 +++++++++++-----------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
index caff296..7f4d291 100644
--- a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
+++ b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
@@ -1438,22 +1438,7 @@ public class ManagedLedgerImpl implements ManagedLedger, CreateCallback {
@Override
public void operationFailed(MetaStoreException e) {
- if (e instanceof BadVersionException) {
- synchronized (ManagedLedgerImpl.this) {
- log.error(
- "[{}] Failed to update ledger list. z-node version mismatch. Closing managed ledger",
- name);
- STATE_UPDATER.set(ManagedLedgerImpl.this, State.Fenced);
- // Return ManagedLedgerFencedException to addFailed callback
- // to indicate that the ledger is now fenced and topic needs to be closed
- clearPendingAddEntries(new ManagedLedgerFencedException(e));
- // Do not need to unlock ledgersListMutex here because we are going to close to topic anyways
- return;
- }
- }
-
log.warn("[{}] Error updating meta data with the new list of ledgers: {}", name, e.getMessage());
-
// Remove the ledger, since we failed to update the list
ledgers.remove(lh.getId());
mbean.startDataLedgerDeleteOp();
@@ -1465,6 +1450,21 @@ public class ManagedLedgerImpl implements ManagedLedger, CreateCallback {
}
}, null);
+ if (e instanceof BadVersionException) {
+ synchronized (ManagedLedgerImpl.this) {
+ log.error(
+ "[{}] Failed to update ledger list. z-node version mismatch. Closing managed ledger",
+ name);
+ lastLedgerCreationFailureTimestamp = clock.millis();
+ STATE_UPDATER.set(ManagedLedgerImpl.this, State.Fenced);
+ // Return ManagedLedgerFencedException to addFailed callback
+ // to indicate that the ledger is now fenced and topic needs to be closed
+ clearPendingAddEntries(new ManagedLedgerFencedException(e));
+ // Do not need to unlock ledgersListMutex here because we are going to close to topic anyways
+ return;
+ }
+ }
+
metadataMutex.unlock();
synchronized (ManagedLedgerImpl.this) {