You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "shouwangyw (via GitHub)" <gi...@apache.org> on 2024/08/20 02:35:46 UTC
[PR] Improvement the dependent result for workflow dependency tasks [dolphinscheduler]
shouwangyw opened a new pull request, #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
## Purpose of the pull request
This pull request improve the dependent result for workflow dependency tasks
## Brief change log
- *Change the method getDependResultByState to getDependResultByStateAndRetry*
## Verify this pull request
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
(or)
## Pull Request Notice
[Pull Request Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#discussion_r1726492157
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -337,18 +337,27 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
}
/**
- * get dependent result by task/process instance state
+ * get dependent result by task/process instance
*
- * @param state state
+ * @param processInstance process instance
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultOfTask(ProcessInstance processInstance, TaskInstance taskInstance) {
+ TaskExecutionStatus state = taskInstance.getState();
if (!state.isFinished()) {
return DependResult.WAITING;
} else if (state.isSuccess()) {
return DependResult.SUCCESS;
} else {
+ log.info("Task code: {}, task name: {}, retryTimes: {}, maxRetryTimes: {}",
+ taskInstance.getTaskCode(), taskInstance.getName(), taskInstance.getRetryTimes(),
+ taskInstance.getMaxRetryTimes());
+ if (processInstance.getState().isRunning()
+ && taskInstance.getRetryTimes() < taskInstance.getMaxRetryTimes()) {
+ return DependResult.WAITING;
+ }
Review Comment:
```suggestion
if (processInstance.getState().isRunning()
&& taskInstance.getRetryTimes() < taskInstance.getMaxRetryTimes()) {
log.info("taskDefinitionCode: {}, taskDefinitionName: {}, retryTimes: {}, maxRetryTimes: {}",
taskInstance.getTaskCode(), taskInstance.getName(), taskInstance.getRetryTimes(),
taskInstance.getMaxRetryTimes());
return DependResult.WAITING;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2298368679
Please follow the pull request using the correct title and description next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS merged PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2300170333
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#discussion_r1722975987
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -339,16 +339,25 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
/**
* get dependent result by task/process instance state
*
- * @param state state
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultByStateAndRetry(TaskInstance taskInstance) {
+ TaskExecutionStatus state = taskInstance.getState();
if (!state.isFinished()) {
return DependResult.WAITING;
} else if (state.isSuccess()) {
return DependResult.SUCCESS;
} else {
+ // Improvement the dependent result for workflow dependency tasks
+ // The reason is that within the maximum number of retries, the dependent task may start a new instance and run successfully
+ // If a failure result is returned directly, then the subsequent DAG of that dependent task node will not execute, and manual intervention in operations will be required
+ log.info("Task code: {}, task name: {}, retryTimes: {}, maxRetryTimes: {}",
+ taskInstance.getTaskCode(), taskInstance.getName(), taskInstance.getRetryTimes(), taskInstance.getMaxRetryTimes());
+ if (taskInstance.getRetryTimes() < taskInstance.getMaxRetryTimes()) {
+ return DependResult.WAITING;
+ }
Review Comment:
You need to verify the status of the workflow at the same time to avoid invalid checks.
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -339,16 +339,25 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
/**
* get dependent result by task/process instance state
*
- * @param state state
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultByStateAndRetry(TaskInstance taskInstance) {
+ TaskExecutionStatus state = taskInstance.getState();
if (!state.isFinished()) {
return DependResult.WAITING;
} else if (state.isSuccess()) {
return DependResult.SUCCESS;
} else {
+ // Improvement the dependent result for workflow dependency tasks
+ // The reason is that within the maximum number of retries, the dependent task may start a new instance and run successfully
+ // If a failure result is returned directly, then the subsequent DAG of that dependent task node will not execute, and manual intervention in operations will be required
Review Comment:
We don't need this comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2304081154
Awesome work, congrats on your first merged pull request!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#discussion_r1726523387
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -337,18 +337,27 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
}
/**
- * get dependent result by task/process instance state
+ * get dependent result by task/process instance
*
- * @param state state
+ * @param processInstance process instance
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultOfTask(ProcessInstance processInstance, TaskInstance taskInstance) {
+ TaskExecutionStatus state = taskInstance.getState();
if (!state.isFinished()) {
return DependResult.WAITING;
} else if (state.isSuccess()) {
return DependResult.SUCCESS;
} else {
+ log.info("Task code: {}, task name: {}, retryTimes: {}, maxRetryTimes: {}",
+ taskInstance.getTaskCode(), taskInstance.getName(), taskInstance.getRetryTimes(),
+ taskInstance.getMaxRetryTimes());
+ if (processInstance.getState().isRunning()
+ && taskInstance.getRetryTimes() < taskInstance.getMaxRetryTimes()) {
+ return DependResult.WAITING;
+ }
Review Comment:
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2304062661
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2304062777
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "shouwangyw (via GitHub)" <gi...@apache.org>.
shouwangyw commented on code in PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#discussion_r1726515101
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -337,18 +337,27 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
}
/**
- * get dependent result by task/process instance state
+ * get dependent result by task/process instance
*
- * @param state state
+ * @param processInstance process instance
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultOfTask(ProcessInstance processInstance, TaskInstance taskInstance) {
+ TaskExecutionStatus state = taskInstance.getState();
if (!state.isFinished()) {
return DependResult.WAITING;
} else if (state.isSuccess()) {
return DependResult.SUCCESS;
} else {
+ log.info("Task code: {}, task name: {}, retryTimes: {}, maxRetryTimes: {}",
+ taskInstance.getTaskCode(), taskInstance.getName(), taskInstance.getRetryTimes(),
+ taskInstance.getMaxRetryTimes());
+ if (processInstance.getState().isRunning()
+ && taskInstance.getRetryTimes() < taskInstance.getMaxRetryTimes()) {
+ return DependResult.WAITING;
+ }
Review Comment:
Okay, this can prevent prolonged checks from printing a large amount of duplicate logs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#discussion_r1723566182
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java:
##########
@@ -339,16 +339,22 @@ private ProcessInstance findLastProcessInterval(Long definitionCode, Long taskCo
/**
* get dependent result by task/process instance state
*
- * @param state state
+ * @param taskInstance task instance
* @return DependResult
*/
- private DependResult getDependResultByState(TaskExecutionStatus state) {
+ private DependResult getDependResultByStateAndRetry(ProcessInstance processInstance, TaskInstance taskInstance) {
Review Comment:
```suggestion
private DependResult getDependResultOfTask(ProcessInstance processInstance, TaskInstance taskInstance) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improvement-16487][Master] Improve the dependent result for workflow dependency tasks [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16488:
URL: https://github.com/apache/dolphinscheduler/pull/16488#issuecomment-2300161467
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16488&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16488&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16488)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org