You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by da...@apache.org on 2024/04/25 11:53:37 UTC
(doris) branch branch-2.0 updated: [fix](bdb) Write OP_TIMESTAMP operation until it successed (#33967) (#34061)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new ae9be518290 [fix](bdb) Write OP_TIMESTAMP operation until it successed (#33967) (#34061)
ae9be518290 is described below
commit ae9be518290e7f4715328e93435362c2f62e46c6
Author: walter <w4...@gmail.com>
AuthorDate: Thu Apr 25 19:53:29 2024 +0800
[fix](bdb) Write OP_TIMESTAMP operation until it successed (#33967) (#34061)
For now, it will reset the next journal id and return if the OP_TIMESTAMP
operation writes failed. Because BDBJE will replicate the committed txns (only
persisted in BDB log, but not replicated to other members) to FOLLOWERs after
the connection resumed, directly resetting the next journal id and returning
will cause subsequent txn written to the same journal ID not to be replayed by
the FOLLOWERS. So for OP_TIMESTAMP operation, try to write until it succeeds.
---
.../apache/doris/journal/bdbje/BDBJEJournal.java | 26 +++++++++++-----------
1 file changed, 13 insertions(+), 13 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java b/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
index 85ebfe2814b..3746e769f34 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
@@ -145,10 +145,21 @@ public class BDBJEJournal implements Journal { // CHECKSTYLE IGNORE THIS LINE: B
MetricRepo.COUNTER_EDIT_LOG_SIZE_BYTES.increase((long) theData.getSize());
MetricRepo.COUNTER_CURRENT_EDIT_LOG_SIZE_BYTES.increase((long) theData.getSize());
}
- LOG.debug("opCode = {}, journal size = {}", op, theData.getSize());
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("opCode = {}, journal size = {}", op, theData.getSize());
+ }
+
// Write the key value pair to bdb.
boolean writeSucceed = false;
- for (int i = 0; i < RETRY_TIME; i++) {
+ // ATTN: If all the followers exit except master, master should continue provide
+ // query service, so do not exit if the write operation is OP_TIMESTAMP.
+ //
+ // Because BDBJE will replicate the committed txns to FOLLOWERs after the connection
+ // resumed, directly reseting the next journal id and returning will cause subsequent
+ // txn written to the same journal ID not to be replayed by the FOLLOWERS. So for
+ // OP_TIMESTAMP operation, try to write until it succeeds here.
+ int retryTimes = op == OperationType.OP_TIMESTAMP ? Integer.MAX_VALUE : RETRY_TIME;
+ for (int i = 0; i < retryTimes; i++) {
try {
// Parameter null means auto commit
if (currentJournalDB.put(null, theKey, theData) == OperationStatus.SUCCESS) {
@@ -190,17 +201,6 @@ public class BDBJEJournal implements Journal { // CHECKSTYLE IGNORE THIS LINE: B
}
if (!writeSucceed) {
- if (op == OperationType.OP_TIMESTAMP) {
- /*
- * Do not exit if the write operation is OP_TIMESTAMP.
- * If all the followers exit except master, master should continue provide query
- * service.
- * To prevent master exit, we should exempt OP_TIMESTAMP write
- */
- nextJournalId.set(id);
- LOG.warn("master can not achieve quorum. write timestamp fail. but will not exit.");
- return -1;
- }
String msg = "write bdb failed. will exit. journalId: " + id + ", bdb database Name: "
+ currentJournalDB.getDatabaseName();
LOG.error(msg);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org