You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by da...@apache.org on 2024/09/23 03:39:54 UTC
(doris) branch branch-3.0 updated: [fix](routine load) write edit log when rescheduled job (#40728)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 32136ca4e93 [fix](routine load) write edit log when rescheduled job (#40728)
32136ca4e93 is described below
commit 32136ca4e9366e18b947f055c507718a7001a73c
Author: hui lai <13...@qq.com>
AuthorDate: Sat Sep 14 16:28:33 2024 +0800
[fix](routine load) write edit log when rescheduled job (#40728)
```
2024-09-11 20:00:53,079 ERROR (replayer|105) [RoutineLoadManager.replayChangeRoutineLoadJob():836] should not happened
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform PAUSED to PAUSED
at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:855) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1407) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadJob.updateState(RoutineLoadJob.java:1394) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadManager.replayChangeRoutineLoadJob(RoutineLoadManager.java:834) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:717) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.replayJournal(Env.java:2913) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env$4.runOneCycle(Env.java:2675) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```
`unprotectNeedReschedule()` will change job state to
`JobState.NEED_SCHEDULE` without `logOpRoutineLoadJob`.If job is paused
then rescheduled and paused finally, the record of two consecutive edit
logs will be 'PAUSED', the correct
replay sequence should be: `PAUSED` -> `NEED_SCHEDULE` ->` PAUSED`.
Therefore, it is need to write edit log when rescheduled job.
---
.../src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java | 2 +-
.../test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java | 1 -
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
index 42032ebfc96..e12b215a0e4 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
@@ -1514,7 +1514,7 @@ public abstract class RoutineLoadJob
.add("msg", "Job need to be rescheduled")
.build());
unprotectUpdateProgress();
- executeNeedSchedule();
+ unprotectUpdateState(JobState.NEED_SCHEDULE, null, false);
}
} finally {
writeUnlock();
diff --git a/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java b/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
index 863cc6807c7..6f3dd2eaaa8 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
@@ -290,7 +290,6 @@ public class RoutineLoadJobTest {
};
RoutineLoadJob routineLoadJob = new KafkaRoutineLoadJob();
- Deencapsulation.setField(routineLoadJob, "state", RoutineLoadJob.JobState.RUNNING);
Deencapsulation.setField(routineLoadJob, "progress", kafkaProgress);
routineLoadJob.update();
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org