You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "JNSimba (via GitHub)" <gi...@apache.org> on 2026/04/09 07:22:36 UTC
[PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
JNSimba opened a new pull request, #62270:
URL: https://github.com/apache/doris/pull/62270
### What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
Currently, StreamingInsertJob uses `PAUSED` state for both manual pause and automatic retry scenarios. Users cannot distinguish between:
- A job that was manually paused (needs no action)
- A job that encountered a transient error and is auto-recovering
- A job that has been continuously failing and retrying
This PR introduces a new `RETRYING` state to purify the semantics:
- **PAUSED** = needs manual intervention (manual pause, non-resumable errors, max retries exceeded)
- **RETRYING** = system is auto-recovering with exponential backoff
#### Key Changes
**New RETRYING State:**
- `RUNNING → RETRYING`: when task fails with a resumable error
- `RETRYING → RUNNING`: when backoff expires and createTask succeeds
- `RETRYING → PAUSED`: when max retry count exceeded or non-resumable error
- RETRYING is a runtime-only state, **not persisted to editlog** for backward compatibility
**Max Retry Limit:**
- New config `streaming_job_max_auto_resume_count` (default 10, mutable)
- After exceeding the limit, job transitions to PAUSED with `CANNOT_RESUME_ERR`
- Error message: "Auto resume failed after N attempts. Last error: ..."
**Backoff Interval:**
- Base time increased from 10s to 30s for better observability
- Progression: 30s → 60s → 120s → 240s → 300s (cap)
**New TVF Columns:**
- `LastTaskSuccessTime`: timestamp of the last successful task completion
- `RetryInfo`: JSON containing `retryCount` (e.g. `{"retryCount": 3}`), empty when count is 0
**Compatibility:**
- RETRYING never appears in editlog, so old FE never encounters unknown enum during rolling upgrade
- `gsonPostProcess` adds null fallback: if an unknown enum deserializes as null, defaults to PENDING
- `replayOnUpdated` skips RETRYING (same as RUNNING) — FE restart goes through PENDING for proper initialization
### Release note
Add RETRYING state to streaming insert job to distinguish auto-recovering jobs from manually paused ones. New TVF columns `LastTaskSuccessTime` and `RetryInfo` provide retry visibility. New config `streaming_job_max_auto_resume_count` limits auto-resume attempts.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] Yes.
- Resumable task failures now transition to `RETRYING` instead of `PAUSED`
- `PAUSED` state no longer triggers auto-resume; only `RETRYING` does
- Backoff base time changed from 10s to 30s
- Two new columns added to insert job TVF: `LastTaskSuccessTime`, `RetryInfo`
- Does this need documentation?
- [x] Yes. <!-- TODO: add docs PR -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4212829055
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4212634864
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4214322751
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #62270:
URL: https://github.com/apache/doris/pull/62270#discussion_r3058045936
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -747,8 +778,9 @@ public void onReplayCreate() throws JobException {
* @param replayJob
*/
public void replayOnUpdated(StreamingInsertJob replayJob) {
- if (!JobStatus.RUNNING.equals(replayJob.getJobStatus())) {
- // No need to restore in the running state, as scheduling relies on pending states.
+ if (!JobStatus.RUNNING.equals(replayJob.getJobStatus())
+ && !JobStatus.RETRYING.equals(replayJob.getJobStatus())) {
+ // No need to restore in the running/retrying state, as scheduling relies on pending states.
// insert TVF does not persist the running state.
// streaming multi task persists the running state when commitOffset() is called.
setJobStatus(replayJob.getJobStatus());
Review Comment:
`JobManager.replayUpdateJob()` does not replace the whole `StreamingInsertJob`; it calls `replayOnUpdated()`. After this PR the new TVF fields depend on `lastTaskSuccessTime` and `autoResumeCount`, but neither field is copied here. After an FE restart or image replay, `jobs()` will lose `LastTaskSuccessTime`/`RetryInfo` even if the edit log contained newer values. Please replay the new fields here, or explicitly treat them as runtime-only and avoid presenting them as persisted job metadata.
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingJobSchedulerTask.java:
##########
@@ -74,36 +77,44 @@ private void handlePendingState() throws JobException {
}
streamingInsertJob.createStreamingTask();
streamingInsertJob.setSampleStartTime(System.currentTimeMillis());
+ // Only reset autoResumeCount on user-initiated start (PENDING),
+ // not on system auto-resume (RETRYING)
+ if (JobStatus.PENDING.equals(streamingInsertJob.getJobStatus())) {
+ streamingInsertJob.setAutoResumeCount(0);
+ }
streamingInsertJob.updateJobStatus(JobStatus.RUNNING);
- streamingInsertJob.setAutoResumeCount(0);
}
private void handleRunningState() throws JobException {
streamingInsertJob.processTimeoutTasks();
streamingInsertJob.fetchMeta();
}
- private void autoResumeHandler() throws JobException {
- final FailureReason failureReason = streamingInsertJob.getFailureReason();
+ private void handleRetryingState() throws JobException {
final long latestAutoResumeTimestamp = streamingInsertJob.getLatestAutoResumeTimestamp();
final long autoResumeCount = streamingInsertJob.getAutoResumeCount();
+ final long maxAutoResumeCount = streamingInsertJob.getMaxAutoResumeCount();
final long current = System.currentTimeMillis();
- if (failureReason != null
- && failureReason.getCode() != InternalErrorCode.MANUAL_PAUSE_ERR
- && failureReason.getCode() != InternalErrorCode.TOO_MANY_FAILURE_ROWS_ERR
- && failureReason.getCode() != InternalErrorCode.CANNOT_RESUME_ERR) {
- long autoResumeIntervalTimeSec = autoResumeCount < 5
- ? Math.min((long) Math.pow(2, autoResumeCount) * BACK_OFF_BASIC_TIME_SEC,
- MAX_BACK_OFF_TIME_SEC) : MAX_BACK_OFF_TIME_SEC;
- if (current - latestAutoResumeTimestamp > autoResumeIntervalTimeSec * 1000L) {
- streamingInsertJob.setLatestAutoResumeTimestamp(current);
- if (autoResumeCount < Long.MAX_VALUE) {
- streamingInsertJob.setAutoResumeCount(autoResumeCount + 1);
- }
- streamingInsertJob.updateJobStatus(JobStatus.PENDING);
- return;
- }
+ // Exceeded max retry count, give up and transition to PAUSED
+ if (autoResumeCount >= maxAutoResumeCount) {
+ String lastError = streamingInsertJob.getFailureReason() != null
+ ? streamingInsertJob.getFailureReason().getMsg() : "unknown";
+ streamingInsertJob.setFailureReason(new FailureReason(
+ InternalErrorCode.CANNOT_RESUME_ERR,
+ "Auto resume failed after " + autoResumeCount
+ + " attempts. Last error: " + lastError));
+ streamingInsertJob.updateJobStatus(JobStatus.PAUSED);
+ return;
+ }
+
+ long autoResumeIntervalTimeSec = Math.min(
+ (long) Math.pow(2, autoResumeCount) * BACK_OFF_BASIC_TIME_SEC,
+ MAX_BACK_OFF_TIME_SEC);
+ if (current - latestAutoResumeTimestamp > autoResumeIntervalTimeSec * 1000L) {
+ streamingInsertJob.setLatestAutoResumeTimestamp(current);
+ streamingInsertJob.setAutoResumeCount(autoResumeCount + 1);
+ handlePendingState();
}
}
Review Comment:
`latestAutoResumeTimestamp` is still `0` the first time a job transitions from `RUNNING` to `RETRYING`. On the next scheduler tick this condition is immediately true, so the first retry happens with no backoff at all. That breaks the new `RETRYING` semantics and also burns one auto-resume attempt earlier than users expect. Please stamp the retry start time when entering `RETRYING` (or otherwise initialize it before this comparison).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba closed pull request #62270: [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones
URL: https://github.com/apache/doris/pull/62270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4214987164
# FE UT Coverage Report
Increment line coverage ` 12.70% (8/63)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/62270_beaa751b1e63b32777805ffd245e9291cfd99e48/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/62270_beaa751b1e63b32777805ffd245e9291cfd99e48/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4212406169
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4213585065
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4214323389
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4212313846
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #62270:
URL: https://github.com/apache/doris/pull/62270#discussion_r3056248397
##########
regression-test/suites/job_p0/streaming_job/test_streaming_job_retry_info.groovy:
##########
@@ -0,0 +1,139 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+import org.awaitility.Awaitility
+
+import static java.util.concurrent.TimeUnit.SECONDS
+
+// Verify that RetryInfo and LastTaskSuccessTime TVF columns are populated correctly.
+// 1. After a successful task, LastTaskSuccessTime should be set.
+// 2. After failures and retries, RetryInfo should contain retryCount.
+// 3. After manual resume, retryCount should be reset to 0.
+suite("test_streaming_job_retry_info", "nonConcurrent") {
+ def tableName = "test_streaming_job_retry_info_tbl"
+ def jobName = "test_streaming_job_retry_info_job"
+
+ sql """drop table if exists `${tableName}` force"""
+ sql """
+ DROP JOB IF EXISTS where jobname = '${jobName}'
+ """
+
+ sql """
+ CREATE TABLE IF NOT EXISTS ${tableName} (
+ `c1` int NULL,
+ `c2` string NULL,
+ `c3` int NULL,
+ ) ENGINE=OLAP
+ DUPLICATE KEY(`c1`)
+ COMMENT 'OLAP'
+ DISTRIBUTED BY HASH(`c1`) BUCKETS 3
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1");
+ """
+
+ // case1: Verify LastTaskSuccessTime is set after successful task
+ sql """
+ CREATE JOB ${jobName}
+ PROPERTIES(
+ "s3.max_batch_files" = "1"
+ )
+ ON STREAMING DO INSERT INTO ${tableName}
+ SELECT * FROM S3
+ (
+ "uri" = "s3://${s3BucketName}/regression/load/data/example_[0-1].csv",
+ "format" = "csv",
+ "provider" = "${getS3Provider()}",
+ "column_separator" = ",",
+ "s3.endpoint" = "${getS3Endpoint()}",
+ "s3.region" = "${getS3Region()}",
+ "s3.access_key" = "${getS3AK()}",
+ "s3.secret_key" = "${getS3SK()}"
+ );
+ """
+
+ try {
+ // Wait for at least one successful task
+ Awaitility.await().atMost(120, SECONDS)
+ .pollInterval(2, SECONDS).until(
+ {
+ def jobRes = sql """ select SucceedTaskCount from jobs("type"="insert") where Name = '${jobName}' and ExecuteType='STREAMING' """
+ log.info("jobRes: " + jobRes)
+ jobRes.size() == 1 && '0' < jobRes.get(0).get(0)
Review Comment:
This compares a string literal `'0'` with a value that is likely numeric (or a numeric-form string) via `<`, which is brittle and can behave unexpectedly (lexicographic compare or type coercion issues). Recommended fix: cast/parse the value to an integer/long and compare numerically (e.g., `jobRes.get(0).get(0) as long`).
```suggestion
jobRes.size() == 1 && (jobRes.get(0).get(0) as long) > 0L
```
##########
fe/fe-core/src/main/java/org/apache/doris/job/scheduler/StreamingTaskScheduler.java:
##########
@@ -94,9 +94,13 @@ private void scheduleTasks(List<AbstractStreamingTask> tasks) {
(StreamingInsertJob) Env.getCurrentEnv().getJobManager().getJob(task.getJobId());
job.setFailureReason(new FailureReason(e.getMessage()));
try {
- job.updateJobStatus(JobStatus.PAUSED);
+ if (job.isResumableFailure(job.getFailureReason())) {
+ job.updateJobStatus(JobStatus.RETRYING);
+ } else {
+ job.updateJobStatus(JobStatus.PAUSED);
+ }
} catch (JobException ex) {
Review Comment:
`job.updateJobStatus(JobStatus.RETRYING)` can throw `IllegalArgumentException` (e.g., if the job is already `RETRYING`), but the code only catches `JobException`. This can escape the current handler and impact the scheduler thread. Recommended fix: avoid calling `updateJobStatus(RETRYING)` unless the current status is `RUNNING` (or make the transition idempotent), and/or broaden the catch to handle `RuntimeException` here with a safe fallback (e.g., log + keep current state or PAUSED depending on policy).
```suggestion
} catch (JobException | RuntimeException ex) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #62270:
URL: https://github.com/apache/doris/pull/62270#discussion_r3056387413
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -414,13 +417,18 @@ public void updateJobStatus(JobStatus status) throws JobException {
lock.writeLock().lock();
try {
super.updateJobStatus(status);
- if (JobStatus.PAUSED.equals(getJobStatus())) {
+ if (JobStatus.PAUSED.equals(getJobStatus()) || JobStatus.RETRYING.equals(getJobStatus())) {
clearRunningStreamTask(status);
}
if (isFinalStatus()) {
Env.getCurrentGlobalTransactionMgr().getCallbackFactory().removeCallback(getJobId());
}
log.info("Streaming insert job {} update status to {}", getJobId(), getJobStatus());
Review Comment:
`RETRYING` is still persisted through checkpoint images, so the backward-compatibility guarantee here is not actually true. `StreamingInsertJob.write()` still serializes the full object with `GsonUtils.GSON.toJson(this)`, and `JobManager.write()` writes every job into the image. If the master saves an image while a job is in `RETRYING`, an older FE that later loads that image during rollback / mixed-version operation will still see the unknown enum value in `jobStatus`, even though the editlog skips it.
That means we need to normalize `RETRYING` before image serialization as well (for example, serialize it as `PENDING`/`PAUSED` or strip the transient status from the checkpoint representation), and ideally add a restart/image compatibility test for this path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #62270:
URL: https://github.com/apache/doris/pull/62270#issuecomment-4212717212
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streaming-job) Add RETRYING state to distinguish normal jobs from retrying ones [doris]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on code in PR #62270:
URL: https://github.com/apache/doris/pull/62270#discussion_r3056669829
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -414,13 +417,18 @@ public void updateJobStatus(JobStatus status) throws JobException {
lock.writeLock().lock();
try {
super.updateJobStatus(status);
- if (JobStatus.PAUSED.equals(getJobStatus())) {
+ if (JobStatus.PAUSED.equals(getJobStatus()) || JobStatus.RETRYING.equals(getJobStatus())) {
clearRunningStreamTask(status);
}
if (isFinalStatus()) {
Env.getCurrentGlobalTransactionMgr().getCallbackFactory().removeCallback(getJobId());
}
log.info("Streaming insert job {} update status to {}", getJobId(), getJobStatus());
Review Comment:
Neither RETRYING nor RUNNING writes to the editlog, so these two states will not appear in the image generated by the checkpoint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org