You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "ghkang98 (via GitHub)" <gi...@apache.org> on 2025/12/22 08:39:14 UTC
[PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
ghkang98 opened a new pull request, #59255:
URL: https://github.com/apache/doris/pull/59255
…ssioned
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59255:
URL: https://github.com/apache/doris/pull/59255#discussion_r2639669592
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteJob.java:
##########
@@ -352,6 +352,11 @@ public void dispatch() throws Exception {
for (Replica replica : tablet.getReplicas()) {
long replicaId = replica.getId();
long backendId = replica.getBackendId();
+ //skip the task dispatch when replica backend has been dropped or decommissioned
+ if (!Env.getCurrentSystemInfo().checkBackendOnline(backendId)){
Review Comment:
what if all backends are not available? because we may not have any alive backend after sending "delete" job to a BE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #59255:
URL: https://github.com/apache/doris/pull/59255#discussion_r2639520854
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteJob.java:
##########
@@ -420,19 +425,6 @@ public void await() throws Exception {
throw new UserException(String.format("delete job timeout, timeout(ms):%s, msg:%s", timeoutMs, errMsg));
case QUORUM_FINISHED:
case FINISHED:
- long nowQuorumTimeMs = System.currentTimeMillis();
Review Comment:
我们正常的逻辑，是：
1. 情形1 ： quorum finished，如果还有backend 没有返回，那么直接到超时，也就是现在这种情况。我们不要改变这个逻辑，保持跟过去的兼容。
2. 情形2：quorum finished，并且现在健康的backend 都返回了。这个就是fix 我们当前遇到的bug。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "ghkang98 (via GitHub)" <gi...@apache.org>.
ghkang98 closed pull request #59255: delete job not dispatch task when replica backend is drop or decommi…
URL: https://github.com/apache/doris/pull/59255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #59255:
URL: https://github.com/apache/doris/pull/59255#discussion_r2639520854
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteJob.java:
##########
@@ -420,19 +425,6 @@ public void await() throws Exception {
throw new UserException(String.format("delete job timeout, timeout(ms):%s, msg:%s", timeoutMs, errMsg));
case QUORUM_FINISHED:
case FINISHED:
- long nowQuorumTimeMs = System.currentTimeMillis();
Review Comment:
我们正常的逻辑，是：
1. 情形1 ： 等待quorum finished，直接到超时，也就是现在这种情况。
2. 情形2：quorum finished，并且现在健康的backend 都返回了。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59255:
URL: https://github.com/apache/doris/pull/59255#issuecomment-3681040126
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #59255:
URL: https://github.com/apache/doris/pull/59255#discussion_r2639520854
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteJob.java:
##########
@@ -420,19 +425,6 @@ public void await() throws Exception {
throw new UserException(String.format("delete job timeout, timeout(ms):%s, msg:%s", timeoutMs, errMsg));
case QUORUM_FINISHED:
case FINISHED:
- long nowQuorumTimeMs = System.currentTimeMillis();
Review Comment:
我们正常的逻辑，是：
1. 情形1 ： quorum finished，如果还有backend 没有返回，那么直接到超时，也就是现在这种情况。
2. 情形2：quorum finished，并且现在健康的backend 都返回了。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "Hastyshell (via GitHub)" <gi...@apache.org>.
Hastyshell commented on code in PR #59255:
URL: https://github.com/apache/doris/pull/59255#discussion_r2645609292
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteJob.java:
##########
@@ -352,6 +352,11 @@ public void dispatch() throws Exception {
for (Replica replica : tablet.getReplicas()) {
long replicaId = replica.getId();
long backendId = replica.getBackendId();
+ //skip the task dispatch when replica backend has been dropped or decommissioned
+ if (!Env.getCurrentSystemInfo().checkBackendOnline(backendId)){
Review Comment:
TransactionMgr seems to be able to handle this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] delete job not dispatch task when replica backend is drop or decommi… [doris]
Posted by "Hastyshell (via GitHub)" <gi...@apache.org>.
Hastyshell commented on PR #59255:
URL: https://github.com/apache/doris/pull/59255#issuecomment-3689697140
> We can add a failure handler to PushTask, so that the latch in the push task will count down immediately when the corresponding BE is unavailable.
>
> ```
> @Override
> public void failedWithMsg(String errMsg) {
> super.failedWithMsg(errMsg);
>
> Status s = new Status(TStatusCode.CANCELLED, errMsg);
> countDownLatchWithStatus(getBackendId(), getTabletId(), s);
> }
> ```
I think this can replace `checkBackendOnline`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org