You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/01/05 06:11:39 UTC
[GitHub] [dolphinscheduler] caishunfeng opened a new pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
caishunfeng opened a new pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005417162
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.5% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005537503
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005417162
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.5% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005550163
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS removed a comment on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
SbloodyS removed a comment on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005413746
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005550163
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a change in pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#discussion_r779251963
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
##########
@@ -1171,11 +1172,33 @@ private DependResult getDependResultForTask(TaskInstance taskInstance) {
* @param taskInstance task instance
*/
private void addTaskToStandByList(TaskInstance taskInstance) {
- logger.info("add task to stand by list, task name: {} , task id:{}", taskInstance.getName(), taskInstance.getId());
try {
- if (!readyToSubmitTaskQueue.contains(taskInstance)) {
- readyToSubmitTaskQueue.put(taskInstance);
+ // need to check if the tasks with same task code is active
+ boolean exist = false;
+ Map<Integer, TaskInstance> taskInstanceMap = taskInstanceHashMap.column(taskInstance.getTaskCode());
+ if (taskInstanceMap != null && taskInstanceMap.size() > 0) {
+ for (Entry<Integer, TaskInstance> entry : taskInstanceMap.entrySet()) {
+ Integer taskInstanceId = entry.getKey();
+ if (activeTaskProcessorMaps.containsKey(taskInstanceId)) {
+ TaskInstance latestTaskInstance = processService.findTaskInstanceById(taskInstanceId);
+ if (latestTaskInstance != null && !latestTaskInstance.getState().typeIsFailure()) {
+ exist = true;
+ break;
+ }
+ }
+ }
+ }
+ if (exist) {
+ logger.warn("task was found in active task list, task code:{}", taskInstance.getTaskCode());
+ return;
+ }
+ if (readyToSubmitTaskQueue.contains(taskInstance)) {
+ logger.warn("task was found in ready submit queue, task code:{}", taskInstance.getTaskCode());
+ return;
}
+ logger.info("add task to stand by list, task name: {}, task id:{}, task code:{}",
Review comment:
```suggestion
logger.info("add task to stand by list, task name:{}, task id:{}, task code:{}",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1006593552
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on a change in pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
lenboo commented on a change in pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#discussion_r779323637
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
##########
@@ -1171,11 +1172,33 @@ private DependResult getDependResultForTask(TaskInstance taskInstance) {
* @param taskInstance task instance
*/
private void addTaskToStandByList(TaskInstance taskInstance) {
- logger.info("add task to stand by list, task name: {} , task id:{}", taskInstance.getName(), taskInstance.getId());
try {
- if (!readyToSubmitTaskQueue.contains(taskInstance)) {
- readyToSubmitTaskQueue.put(taskInstance);
+ // need to check if the tasks with same task code is active
+ boolean exist = false;
+ Map<Integer, TaskInstance> taskInstanceMap = taskInstanceHashMap.column(taskInstance.getTaskCode());
+ if (taskInstanceMap != null && taskInstanceMap.size() > 0) {
+ for (Entry<Integer, TaskInstance> entry : taskInstanceMap.entrySet()) {
+ Integer taskInstanceId = entry.getKey();
+ if (activeTaskProcessorMaps.containsKey(taskInstanceId)) {
+ TaskInstance latestTaskInstance = processService.findTaskInstanceById(taskInstanceId);
+ if (latestTaskInstance != null && !latestTaskInstance.getState().typeIsFailure()) {
Review comment:
Is it necessary to query task instance from db?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo merged pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
lenboo merged pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005413359
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> :exclamation: No coverage uploaded for pull request base (`2.0.2-release@1d3f77b`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#section-missing-base-commit).
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## 2.0.2-release #7808 +/- ##
================================================
Coverage ? 31.35%
Complexity ? 1556
================================================
Files ? 433
Lines ? 14785
Branches ? 1479
================================================
Hits ? 4636
Misses ? 9693
Partials ? 456
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1d3f77b...98a8377](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005537503
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG) [24 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT) [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL) [321 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7808&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list) [16.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list) [0.9% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7808&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005413746
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#discussion_r779506349
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
##########
@@ -1171,11 +1172,33 @@ private DependResult getDependResultForTask(TaskInstance taskInstance) {
* @param taskInstance task instance
*/
private void addTaskToStandByList(TaskInstance taskInstance) {
- logger.info("add task to stand by list, task name: {} , task id:{}", taskInstance.getName(), taskInstance.getId());
try {
- if (!readyToSubmitTaskQueue.contains(taskInstance)) {
- readyToSubmitTaskQueue.put(taskInstance);
+ // need to check if the tasks with same task code is active
+ boolean exist = false;
+ Map<Integer, TaskInstance> taskInstanceMap = taskInstanceHashMap.column(taskInstance.getTaskCode());
+ if (taskInstanceMap != null && taskInstanceMap.size() > 0) {
+ for (Entry<Integer, TaskInstance> entry : taskInstanceMap.entrySet()) {
+ Integer taskInstanceId = entry.getKey();
+ if (activeTaskProcessorMaps.containsKey(taskInstanceId)) {
+ TaskInstance latestTaskInstance = processService.findTaskInstanceById(taskInstanceId);
+ if (latestTaskInstance != null && !latestTaskInstance.getState().typeIsFailure()) {
Review comment:
> Is it necessary to query task instance from db?
Yes, because the task instance in memory was not up to date.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #7808: [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7808:
URL: https://github.com/apache/dolphinscheduler/pull/7808#issuecomment-1005413359
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> :exclamation: No coverage uploaded for pull request base (`2.0.2-release@1d3f77b`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#section-missing-base-commit).
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## 2.0.2-release #7808 +/- ##
================================================
Coverage ? 31.33%
Complexity ? 1557
================================================
Files ? 433
Lines ? 14802
Branches ? 1483
================================================
Hits ? 4638
Misses ? 9705
Partials ? 459
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1d3f77b...8796130](https://codecov.io/gh/apache/dolphinscheduler/pull/7808?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org