You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by li...@apache.org on 2024/04/17 08:35:43 UTC
(doris) branch branch-2.0 updated: [fix](routine-load) fix data lost when FE leader change (#33678) (#33762)
This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new f72acbda3cf [fix](routine-load) fix data lost when FE leader change (#33678) (#33762)
f72acbda3cf is described below
commit f72acbda3cf4f802cd603651d0106427071b6105
Author: HHoflittlefish777 <77...@users.noreply.github.com>
AuthorDate: Wed Apr 17 16:35:35 2024 +0800
[fix](routine-load) fix data lost when FE leader change (#33678) (#33762)
---
.../java/org/apache/doris/load/routineload/RoutineLoadJob.java | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
index 15f3e8c1270..94ba7ac8e17 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
@@ -1197,7 +1197,13 @@ public abstract class RoutineLoadJob extends AbstractTxnStateChangeCallback impl
@Override
public void replayOnAborted(TransactionState txnState) {
// attachment may be null if this task is aborted by FE
- if (txnState.getTxnCommitAttachment() != null) {
+ // it need check commit info before update progress
+ // for follower FE node progress may exceed correct progress
+ // the data will lost if FE leader change at this moment
+ if (txnState.getTxnCommitAttachment() != null
+ && checkCommitInfo((RLTaskTxnCommitAttachment) txnState.getTxnCommitAttachment(),
+ txnState,
+ TransactionState.TxnStatusChangeReason.fromString(txnState.getReason()))) {
replayUpdateProgress((RLTaskTxnCommitAttachment) txnState.getTxnCommitAttachment());
}
this.jobStatistic.abortedTaskNum++;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org