You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/11/07 02:35:00 UTC
[GitHub] [dolphinscheduler] wangxj3 opened a new pull request #6722: [Feature-#6422] task group queue
wangxj3 opened a new pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722
Task group queue (TGQ) can limit the concurrency of taskInstance.TGQ also supports priority-based control, which ensures that important tasks can be executed first. Users can also execute a task compulsively, ignoring the TGQ.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979650807
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r758143817
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java
##########
@@ -70,6 +70,18 @@ public boolean submit(TaskInstance task, ProcessInstance processInstance, int ma
if (this.taskInstance == null) {
return false;
}
+ int taskGroupId = task.getTaskGroupId();
Review comment:
Is it better to create a new task type for `TGQ`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979783346
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979783346
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e0a3064) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/1a88b1863b9000d7e220461ed44de7c6ab9a8169?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a88b18) will **decrease** coverage by `0.48%`.
> The diff coverage is `3.96%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.67% 32.18% -0.49%
+ Complexity 1607 1605 -2
============================================
Files 431 434 +3
Lines 14340 14539 +199
Branches 1437 1462 +25
============================================
- Hits 4686 4680 -6
- Misses 9203 9406 +203
- Partials 451 453 +2
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `5.18% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `5.47% <0.00%> (-0.98%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1a88b18...e0a3064](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r756686265
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
+ * @param taskName
+ * @param groupId
+ * @param processId
+ * @param priority
+ * @return
+ */
+ public boolean acquireTaskGroup(int taskId,
+ String taskName, int groupId,
+ int processId, int priority) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(groupId);
+ if (taskGroup == null) {
+ return true;
+ }
+ // if task group is not applicable
+ if (taskGroup.getStatus() == Flag.NO.getCode()) {
+ return true;
+ }
+ TaskGroupQueue taskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskId);
+ if (taskGroupQueue == null) {
+ taskGroupQueue = insertIntoTaskGroupQueue(taskId, taskName, groupId, processId, priority, TaskGroupQueueStatus.WAIT_QUEUE);
+ } else {
+ if (taskGroupQueue.getStatus() == TaskGroupQueueStatus.ACQUIRE_SUCCESS) {
+ return true;
+ }
+ taskGroupQueue.setInQueue(Flag.NO.getCode());
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.WAIT_QUEUE);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ }
+ //check priority
+ List<TaskGroupQueue> highPriorityTasks = taskGroupQueueMapper.queryHighPriorityTasks(groupId, priority, TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (CollectionUtils.isNotEmpty(highPriorityTasks)) {
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+ //try to get taskGroup
+ int count = taskGroupMapper.selectAvailableCountById(groupId);
+ if (count == 1 && robTaskGroupResouce(taskGroupQueue)) {
+ return true;
+ }
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+
+ public boolean robTaskGroupResouce(TaskGroupQueue taskGroupQueue) {
Review comment:
Get the resources of the task group
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979771560
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r758101249
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskGroupServiceImpl.java
##########
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.service.impl;
+
+import org.apache.dolphinscheduler.api.enums.Status;
+import org.apache.dolphinscheduler.api.service.TaskGroupQueueService;
+import org.apache.dolphinscheduler.api.service.TaskGroupService;
+import org.apache.dolphinscheduler.api.utils.PageInfo;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.dolphinscheduler.common.enums.Flag;
+import org.apache.dolphinscheduler.dao.entity.TaskGroup;
+import org.apache.dolphinscheduler.dao.entity.User;
+import org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper;
+import org.apache.dolphinscheduler.service.process.ProcessService;
+import org.apache.dolphinscheduler.spi.utils.StringUtils;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+
+import com.baomidou.mybatisplus.core.metadata.IPage;
+import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
+
+/**
+ * task Group Service
+ */
+@Service
+public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupService {
+
+ @Autowired
+ private TaskGroupMapper taskGroupMapper;
+
+ @Autowired
+ private TaskGroupQueueService taskGroupQueueService;
+
+ @Autowired
+ private ProcessService processService;
+
+ private static final Logger logger = LoggerFactory.getLogger(TaskGroupServiceImpl.class);
+
+ /**
+ * create a Task group
+ *
+ * @param loginUser login user
+ * @param name task group name
+ * @param description task group description
+ * @param groupSize task group total size
+ * @return the result code and msg
+ */
+ @Override
+ public Map<String, Object> createTaskGroup(User loginUser, String name, String description, int groupSize) {
+ Map<String, Object> result = new HashMap<>();
+ if (isNotAdmin(loginUser, result)) {
+ return result;
+ }
+ if (name == null) {
+ putMsg(result, Status.NAME_NULL);
+ return result;
+ }
+ if (groupSize <= 0) {
+ putMsg(result, Status.TASK_GROUP_SIZE_ERROR);
+ return result;
+
+ }
+ TaskGroup taskGroup1 = taskGroupMapper.queryByName(loginUser.getId(), name);
+ if (taskGroup1 != null) {
+ putMsg(result, Status.TASK_GROUP_NAME_EXSIT);
+ return result;
+ }
+ TaskGroup taskGroup = new TaskGroup(name, description,
+ groupSize, loginUser.getId());
+ int insert = taskGroupMapper.insert(taskGroup);
+ logger.info("insert result:{}", insert);
+ putMsg(result, Status.SUCCESS);
+
+ return result;
+ }
+
+ /**
+ * update the task group
+ *
+ * @param loginUser login user
+ * @param name task group name
+ * @param description task group description
+ * @param groupSize task group total size
+ * @return the result code and msg
+ */
+ @Override
+ public Map<String, Object> updateTaskGroup(User loginUser, int id, String name, String description, int groupSize) {
+ Map<String, Object> result = new HashMap<>();
+ if (isNotAdmin(loginUser, result)) {
+ return result;
+ }
+ TaskGroup taskGroup = taskGroupMapper.selectById(id);
+ if (taskGroup.getStatus() != Flag.NO.getCode()) {
+ putMsg(result, Status.TASK_GROUP_STATUS_ERROR);
+ return result;
+ }
+ taskGroup.setGroupSize(groupSize);
+ taskGroup.setDescription(description);
+ if (StringUtils.isNotEmpty(name)) {
+ taskGroup.setName(name);
+ }
+ int i = taskGroupMapper.updateById(taskGroup);
+ logger.info("update result:{}", i);
+ putMsg(result, Status.SUCCESS);
+ return result;
+ }
+
+ /**
+ * get task group status
+ *
+ * @param id task group id
+ * @return is the task group available
+ */
+ @Override
+ public boolean isTheTaskGroupAvailable(int id) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(id);
Review comment:
It's better to use `select count(1) from task_group where id = {}` instead of selecting a full row.
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupMapper.xml
##########
@@ -0,0 +1,87 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroup" id="TaskGroupMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="name" column="name" jdbcType="VARCHAR"/>
+ <result property="description" column="description" jdbcType="VARCHAR"/>
+ <result property="groupSize" column="group_size" jdbcType="INTEGER"/>
+ <result property="useSize" column="use_size" jdbcType="INTEGER"/>
+ <result property="userId" column="user_id" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="VARCHAR"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id,name,description,group_size,use_size,create_time,update_time
+ </sql>
+
+ <select id="queryTaskGroupPaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroup">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group
+ <where>
+ <if test="userId != 0">
+ and user_id = #{userId}
+ </if>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ <if test="name != null and name != '' ">
+ and name like concat('%', #{name}, '%')
+ </if>
+ </where>
+ </select>
+
+ <!--modify data by id-->
+ <update id="updateTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size+1
+ where id = #{id} and use_size < group_size and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
+ </update>
+
+ <!--modify data by id-->
+ <update id="releaseTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size-1
+ where id = #{id} and use_size > 0 and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
Review comment:
the same
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupQueueMapper.xml
##########
@@ -0,0 +1,127 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue" id="TaskGroupQueueMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="taskId" column="task_id" jdbcType="INTEGER"/>
+ <result property="taskName" column="task_name" jdbcType="VARCHAR"/>
+ <result property="groupId" column="group_id" jdbcType="INTEGER"/>
+ <result property="processId" column="process_id" jdbcType="INTEGER"/>
+ <result property="priority" column="priority" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="INTEGER"/>
+ <result property="forceStart" column="force_start" jdbcType="INTEGER"/>
+ <result property="inQueue" column="in_queue" jdbcType="INTEGER"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id, task_id, task_name, group_id, process_id, priority, status , force_start , in_queue, create_time, update_time
+ </sql>
+
+ <select id="queryTaskGroupQueuePaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="groupId != 0">
+ and group_id = #{groupId}
+ </if>
+ <if test="processId != 0">
+ and process_id = #{processId}
+ </if>
+ </where>
+ </select>
+
+ <select id="queryByStatus" resultMap="TaskGroupQueueMap" resultType="map">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="status != 0">
Review comment:
Is `state=0` represents the initial state? it's a hidden logic for `queryByStatus`
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupQueueMapper.xml
##########
@@ -0,0 +1,127 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue" id="TaskGroupQueueMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="taskId" column="task_id" jdbcType="INTEGER"/>
+ <result property="taskName" column="task_name" jdbcType="VARCHAR"/>
+ <result property="groupId" column="group_id" jdbcType="INTEGER"/>
+ <result property="processId" column="process_id" jdbcType="INTEGER"/>
+ <result property="priority" column="priority" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="INTEGER"/>
+ <result property="forceStart" column="force_start" jdbcType="INTEGER"/>
+ <result property="inQueue" column="in_queue" jdbcType="INTEGER"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id, task_id, task_name, group_id, process_id, priority, status , force_start , in_queue, create_time, update_time
+ </sql>
+
+ <select id="queryTaskGroupQueuePaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="groupId != 0">
+ and group_id = #{groupId}
+ </if>
+ <if test="processId != 0">
+ and process_id = #{processId}
+ </if>
+ </where>
+ </select>
+
+ <select id="queryByStatus" resultMap="TaskGroupQueueMap" resultType="map">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ </where>
+ </select>
+
+ <delete id="deleteByTaskId">
+ delete from t_ds_task_group_queue
+ where task_id = #{taskId}
+ </delete>
+
+ <update id="updateStatusByTaskId">
+ update t_ds_task_group_queue
+ <set>
+ <if test="status != 0">
+ status = #{status},
+ </if>
+ </set>
+ where task_id = #{taskId}
+ </update>
+
+ <update id="updateInQueue">
+ update t_ds_task_group_queue
+ set in_queue = #{inQueue}
+ where id = #{id}
+ </update>
+
+ <update id="updateForceStart">
+ update t_ds_task_group_queue
+ set force_start = #{forceStart}
+ where task_id = #{taskId}
+ </update>
+
+ <update id="updateInQueueLimit1">
+ update t_ds_task_group_queue
+ set in_queue = #{newValue}
+ where group_id = #{groupId} and in_queue = #{oldValue} and status = #{status} order by priority desc limit 1
+ </update>
+
+ <update id="updateInQueueCAS">
+ update t_ds_task_group_queue
+ set in_queue = #{newValue}
+ where id = #{id} and in_queue = #{oldValue}
+ </update>
+
+ <select id="queryHighPriorityTasks" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql" />
+ from t_ds_task_group_queue
+ where group_id = #{groupId} and status = #{status} and priority > #{priority}
+ </select>
+ <select id="queryTheHighestPriorityTasks" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql" />
+ from t_ds_task_group_queue
+ where priority = (select max(priority) from t_ds_task_group_queue where group_id = #{groupId}
Review comment:
`select max(priority) from t_ds_task_group_queue where group_id = #{groupId} and status = #{status} and in_queue = #{inQueue} and force_start = #{forceStart}` Is it OK when this sub sql doesn't match anything?
##########
File path: dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
##########
@@ -1017,3 +1020,40 @@ CREATE TABLE `t_ds_environment_worker_group_relation` (
PRIMARY KEY (`id`),
UNIQUE KEY `environment_worker_group_unique` (`environment_code`,`worker_group`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
+
+-- ----------------------------
+-- Table structure for t_ds_task_group_queue
+-- ----------------------------
+DROP TABLE IF EXISTS `t_ds_task_group_queue`;
+CREATE TABLE `t_ds_task_group_queue` (
+ `id` int(11) NOT NULL AUTO_INCREMENT COMMENT'key',
+ `task_id` int(11) DEFAULT NULL COMMENT 'taskintanceid',
+ `task_name` varchar(100) DEFAULT NULL COMMENT 'TaskInstance name',
+ `group_id` int(11) DEFAULT NULL COMMENT 'taskGroup id',
+ `process_id` int(11) DEFAULT NULL COMMENT 'processInstace id',
+ `priority` int(8) DEFAULT '0' COMMENT 'priority',
+ `status` tinyint(4) DEFAULT '-1' COMMENT '-1: waiting 1: running 2: finished',
+ `force_start` tinyint(4) DEFAULT '0' COMMENT 'is force start 0 NO ,1 YES',
+ `in_queue` tinyint(4) DEFAULT '0' COMMENT 'ready to get the queue by other task finish 0 NO ,1 YES',
+ `create_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
+ `update_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+ PRIMARY KEY( `id` )
Review comment:
Does it need to add any index?
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupMapper.xml
##########
@@ -0,0 +1,87 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroup" id="TaskGroupMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="name" column="name" jdbcType="VARCHAR"/>
+ <result property="description" column="description" jdbcType="VARCHAR"/>
+ <result property="groupSize" column="group_size" jdbcType="INTEGER"/>
+ <result property="useSize" column="use_size" jdbcType="INTEGER"/>
+ <result property="userId" column="user_id" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="VARCHAR"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id,name,description,group_size,use_size,create_time,update_time
+ </sql>
+
+ <select id="queryTaskGroupPaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroup">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group
+ <where>
+ <if test="userId != 0">
+ and user_id = #{userId}
+ </if>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ <if test="name != null and name != '' ">
+ and name like concat('%', #{name}, '%')
+ </if>
+ </where>
+ </select>
+
+ <!--modify data by id-->
+ <update id="updateTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size+1
+ where id = #{id} and use_size < group_size and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
Review comment:
Why not split this sql and valid it before update? It can keep the sql logic more clearly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759084892
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java
##########
@@ -70,6 +70,18 @@ public boolean submit(TaskInstance task, ProcessInstance processInstance, int ma
if (this.taskInstance == null) {
return false;
}
+ int taskGroupId = task.getTaskGroupId();
Review comment:
Do you mean the status of task ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d9ba9b6) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/100a9ba3fc21764158593bd33e2b261b0ef39982?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (100a9ba) will **decrease** coverage by `0.43%`.
> The diff coverage is `3.96%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.86% 32.43% -0.44%
+ Complexity 1633 1632 -1
============================================
Files 439 442 +3
Lines 14459 14658 +199
Branches 1447 1472 +25
============================================
+ Hits 4752 4754 +2
- Misses 9248 9441 +193
- Partials 459 463 +4
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `4.92% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `4.10% <0.00%> (-0.73%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [3 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [100a9ba...d9ba9b6](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r749872573
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
+ * @param taskName
+ * @param groupId
+ * @param processId
+ * @param priority
+ * @return
+ */
+ public boolean acquireTaskGroup(int taskId,
+ String taskName, int groupId,
+ int processId, int priority) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(groupId);
+ if (taskGroup == null) {
+ return true;
+ }
+ // if task group is not applicable
+ if (taskGroup.getStatus() == Flag.NO.getCode()) {
+ return true;
+ }
+ TaskGroupQueue taskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskId);
+ if (taskGroupQueue == null) {
+ taskGroupQueue = insertIntoTaskGroupQueue(taskId, taskName, groupId, processId, priority, TaskGroupQueueStatus.WAIT_QUEUE);
+ } else {
+ if (taskGroupQueue.getStatus() == TaskGroupQueueStatus.ACQUIRE_SUCCESS) {
+ return true;
+ }
+ taskGroupQueue.setInQueue(Flag.NO.getCode());
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.WAIT_QUEUE);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ }
+ //check priority
+ List<TaskGroupQueue> highPriorityTasks = taskGroupQueueMapper.queryHighPriorityTasks(groupId, priority, TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (CollectionUtils.isNotEmpty(highPriorityTasks)) {
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+ //try to get taskGroup
+ int count = taskGroupMapper.selectAvailableCountById(groupId);
+ if (count == 1 && robTaskGroupResouce(taskGroupQueue)) {
+ return true;
+ }
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+
+ public boolean robTaskGroupResouce(TaskGroupQueue taskGroupQueue) {
Review comment:
what's the function name mean?
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TaskGroupController.java
##########
@@ -0,0 +1,298 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.controller;
+
+import static org.apache.dolphinscheduler.api.enums.Status.CLOSE_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.CREATE_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_GROUP_LIST_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_GROUP_QUEUE_LIST_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.START_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_TASK_GROUP_ERROR;
+
+import org.apache.dolphinscheduler.api.aspect.AccessLogAnnotation;
+import org.apache.dolphinscheduler.api.exceptions.ApiException;
+import org.apache.dolphinscheduler.api.service.TaskGroupQueueService;
+import org.apache.dolphinscheduler.api.service.TaskGroupService;
+import org.apache.dolphinscheduler.api.utils.Result;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.dolphinscheduler.dao.entity.User;
+
+import java.util.Map;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
+import org.springframework.web.bind.annotation.GetMapping;
+import org.springframework.web.bind.annotation.PostMapping;
+import org.springframework.web.bind.annotation.RequestAttribute;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestParam;
+import org.springframework.web.bind.annotation.ResponseStatus;
+import org.springframework.web.bind.annotation.RestController;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiImplicitParam;
+import io.swagger.annotations.ApiImplicitParams;
+import io.swagger.annotations.ApiOperation;
+import springfox.documentation.annotations.ApiIgnore;
+
+
+/**
+ * task group controller
+ */
+@Api(tags = "task group")
+@RestController
+@RequestMapping("/task-group")
+public class TaskGroupController extends BaseController {
+
+ @Autowired
+ private TaskGroupService taskGroupService;
+
+ /**
+ * query task group list
+ *
+ * @param loginUser login user
+ * @param name name
+ * @param description description
+ * @param groupSize group size
+ * @param name project id
+ * @return result and msg code
+ */
+ @ApiOperation(value = "createTaskGroup", notes = "CREATE_TAKS_GROUP_NOTE")
+ @ApiImplicitParams({
+ @ApiImplicitParam(name = "name", value = "NAME", dataType = "String"),
+ @ApiImplicitParam(name = "description", value = "DESCRIPTION", dataType = "String"),
+ @ApiImplicitParam(name = "groupSize", value = "GROUPSIZE", dataType = "Int"),
+
+ })
+ @PostMapping(value = "/create")
+ @ResponseStatus(HttpStatus.CREATED)
+ @ApiException(CREATE_TASK_GROUP_ERROR)
+ @AccessLogAnnotation(ignoreRequestArgs = "loginUser")
+ public Result createTaskGroup(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
+ @RequestParam("name") String name,
+ @RequestParam("description") String description,
+ @RequestParam("groupSize") Integer groupSize) {
+ User user = new User();
Review comment:
the user seems never use;
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskGroupQueueServiceImpl.java
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.service.impl;
+
+import org.apache.dolphinscheduler.api.enums.Status;
+import org.apache.dolphinscheduler.api.service.TaskGroupQueueService;
+import org.apache.dolphinscheduler.api.utils.PageInfo;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.dolphinscheduler.dao.entity.TaskGroupQueue;
+import org.apache.dolphinscheduler.dao.entity.User;
+import org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper;
+import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+
+import com.baomidou.mybatisplus.core.metadata.IPage;
+import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
+
+/**
+ * task group queue service
+ */
+@Service
+public class TaskGroupQueueServiceImpl extends BaseServiceImpl implements TaskGroupQueueService {
+
+ @Autowired
+ TaskGroupQueueMapper taskGroupQueueMapper;
+
+ @Autowired
+ private TaskInstanceMapper taskInstanceMapper;
+
+ private static final Logger logger = LoggerFactory.getLogger(TaskGroupQueueServiceImpl.class);
+
+ /**
+ * query tasks in task group queue by group id
+ *
+ * @param loginUser login user
+ * @param groupId group id
+ * @param pageNo page no
+ * @param pageSize page size
+ * @return tasks list
+ */
+ @Override
+ public Map<String, Object> queryTasksByGroupId(User loginUser, Integer groupId, Integer pageNo, Integer pageSize) {
+ return this.doQuery(loginUser, pageNo, pageSize, null, groupId);
+ }
+
+ /**
+ * query tasks in task group queue by project id
+ *
+ * @param loginUser login user
+ * @param pageNo page no
+ * @param pageSize page size
+ * @param processId process id
+ * @return tasks list
+ */
+ @Override
+ public Map<String, Object> queryTasksByProcessId(User loginUser, Integer pageNo, Integer pageSize, Integer processId) {
+ return this.doQuery(loginUser, pageNo, pageSize, null, processId);
+ }
+
+ /**
+ * query all tasks in task group queue
+ *
+ * @param loginUser login user
+ * @param pageNo page no
+ * @param pageSize page size
+ * @return tasks list
+ */
+ @Override
+ public Map<String, Object> queryAllTasks(User loginUser, Integer pageNo, Integer pageSize) {
+ return this.doQuery(loginUser, pageNo, pageSize, null, null);
+ }
+
+ public Map<String, Object> doQuery(User loginUser, Integer pageNo, Integer pageSize,
+ Integer groupId,
+ Integer processId) {
+ Map<String, Object> result = new HashMap<>();
+ if (isNotAdmin(loginUser, result)) {
+ return result;
+ }
+ Page<TaskGroupQueue> page = new Page<>(pageNo, pageSize);
+ IPage<TaskGroupQueue> taskGroupQueue = taskGroupQueueMapper.queryTaskGroupQueuePaging(page, groupId);
+
+ PageInfo<TaskGroupQueue> pageInfo = new PageInfo<>(pageNo, pageSize);
+ pageInfo.setTotal((int) taskGroupQueue.getTotal());
+ pageInfo.setTotalList(taskGroupQueue.getRecords());
+
+ result.put(Constants.DATA_LIST, pageInfo);
+ logger.info("select result:{}", taskGroupQueue);
Review comment:
Is this log for debug?
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
+ * @param taskName
+ * @param groupId
+ * @param processId
+ * @param priority
+ * @return
+ */
+ public boolean acquireTaskGroup(int taskId,
+ String taskName, int groupId,
+ int processId, int priority) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(groupId);
+ if (taskGroup == null) {
+ return true;
+ }
+ // if task group is not applicable
+ if (taskGroup.getStatus() == Flag.NO.getCode()) {
+ return true;
+ }
+ TaskGroupQueue taskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskId);
+ if (taskGroupQueue == null) {
+ taskGroupQueue = insertIntoTaskGroupQueue(taskId, taskName, groupId, processId, priority, TaskGroupQueueStatus.WAIT_QUEUE);
+ } else {
+ if (taskGroupQueue.getStatus() == TaskGroupQueueStatus.ACQUIRE_SUCCESS) {
+ return true;
+ }
+ taskGroupQueue.setInQueue(Flag.NO.getCode());
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.WAIT_QUEUE);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ }
+ //check priority
+ List<TaskGroupQueue> highPriorityTasks = taskGroupQueueMapper.queryHighPriorityTasks(groupId, priority, TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (CollectionUtils.isNotEmpty(highPriorityTasks)) {
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+ //try to get taskGroup
+ int count = taskGroupMapper.selectAvailableCountById(groupId);
+ if (count == 1 && robTaskGroupResouce(taskGroupQueue)) {
+ return true;
+ }
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+
+ public boolean robTaskGroupResouce(TaskGroupQueue taskGroupQueue) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(taskGroupQueue.getGroupId());
+ int affectedCount = taskGroupMapper.updateTaskGroupResource(taskGroup.getId(),taskGroupQueue.getId(),
+ TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (affectedCount > 0) {
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.ACQUIRE_SUCCESS);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return true;
+ }
+ return false;
+ }
+
+ public boolean acquireTaskGroupAgain(TaskGroupQueue taskGroupQueue) {
+ return robTaskGroupResouce(taskGroupQueue);
+ }
+
+ public void releaseAllTaskGroup(int processInstanceId) {
+ List<TaskInstance> taskInstances = this.taskInstanceMapper.loadAllInfosNoRelease(processInstanceId, TaskGroupQueueStatus.ACQUIRE_SUCCESS.getCode());
+ for (TaskInstance info : taskInstances) {
+ releaseTaskGroup(info);
+ }
+ }
+
+ /**
+ * release the TGQ resource when the corresponding task is finished.
+ *
+ * @return the result code and msg
+ */
+ public TaskInstance releaseTaskGroup(TaskInstance taskInstance) {
+
+ TaskGroup taskGroup = taskGroupMapper.selectById(taskInstance.getTaskGroupId());
+ if (taskGroup == null) {
+ return null;
+ }
+ TaskGroupQueue thisTaskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskInstance.getId());
+ if (thisTaskGroupQueue.getStatus() == TaskGroupQueueStatus.RELEASE) {
+ return null;
+ }
+ try {
+ while (taskGroupMapper.releaseTaskGroupResource(taskGroup.getId(), taskGroup.getUseSize()
+ , thisTaskGroupQueue.getId(), TaskGroupQueueStatus.ACQUIRE_SUCCESS.getCode()) != 1) {
+ thisTaskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskInstance.getId());
+ if (thisTaskGroupQueue.getStatus() == TaskGroupQueueStatus.RELEASE) {
+ return null;
+ }
+ taskGroup = taskGroupMapper.selectById(taskInstance.getTaskGroupId());
+ }
+ } catch (Exception e) {
+ logger.info("error:{}",e);
Review comment:
should be logger.error
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/processor/TaskEventProcessor.java
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.server.master.processor;
+
+import org.apache.dolphinscheduler.common.enums.StateEvent;
+import org.apache.dolphinscheduler.common.enums.StateEventType;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.remote.command.Command;
+import org.apache.dolphinscheduler.remote.command.CommandType;
+import org.apache.dolphinscheduler.remote.command.TaskEventChangeCommand;
+import org.apache.dolphinscheduler.remote.processor.NettyRequestProcessor;
+import org.apache.dolphinscheduler.server.master.processor.queue.StateEventResponseService;
+import org.apache.dolphinscheduler.service.bean.SpringApplicationContext;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+import io.netty.channel.Channel;
+
+/**
+ * handle state event received from master/api
+ */
+public class TaskEventProcessor implements NettyRequestProcessor {
+
+ private final Logger logger = LoggerFactory.getLogger(TaskEventProcessor.class);
+
+ private StateEventResponseService stateEventResponseService;
+
+ public TaskEventProcessor() {
+ stateEventResponseService = SpringApplicationContext.getBean(StateEventResponseService.class);
+ }
+
+ @Override
+ public void process(Channel channel, Command command) {
+ Preconditions.checkArgument(CommandType.TASK_FORCE_STATE_EVENT_REQUEST == command.getType()
+ || CommandType.TASK_WAKEUP_EVENT_REQUEST == command.getType()
+ , String.format("invalid command type: %s", command.getType()));
+
+ TaskEventChangeCommand taskEventChangeCommand = JSONUtils.parseObject(command.getBody(), TaskEventChangeCommand.class);
+ StateEvent stateEvent = new StateEvent();
+ stateEvent.setKey(taskEventChangeCommand.getKey());
+ stateEvent.setProcessInstanceId(taskEventChangeCommand.getProcessInstanceId());
+ stateEvent.setTaskInstanceId(taskEventChangeCommand.getTaskInstanceId());
+ stateEvent.setType(StateEventType.WAIT_TASK_GROUP);
+ logger.info("received command : {}", stateEvent);
+ stateEventResponseService.addEnent2WorkflowExecute(stateEvent);
Review comment:
should be `addEvent2WorkflowExecute`?
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
+ * @param taskName
+ * @param groupId
+ * @param processId
+ * @param priority
+ * @return
+ */
+ public boolean acquireTaskGroup(int taskId,
+ String taskName, int groupId,
+ int processId, int priority) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(groupId);
+ if (taskGroup == null) {
+ return true;
+ }
+ // if task group is not applicable
+ if (taskGroup.getStatus() == Flag.NO.getCode()) {
+ return true;
+ }
+ TaskGroupQueue taskGroupQueue = this.taskGroupQueueMapper.queryByTaskId(taskId);
+ if (taskGroupQueue == null) {
+ taskGroupQueue = insertIntoTaskGroupQueue(taskId, taskName, groupId, processId, priority, TaskGroupQueueStatus.WAIT_QUEUE);
+ } else {
+ if (taskGroupQueue.getStatus() == TaskGroupQueueStatus.ACQUIRE_SUCCESS) {
+ return true;
+ }
+ taskGroupQueue.setInQueue(Flag.NO.getCode());
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.WAIT_QUEUE);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ }
+ //check priority
+ List<TaskGroupQueue> highPriorityTasks = taskGroupQueueMapper.queryHighPriorityTasks(groupId, priority, TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (CollectionUtils.isNotEmpty(highPriorityTasks)) {
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+ //try to get taskGroup
+ int count = taskGroupMapper.selectAvailableCountById(groupId);
+ if (count == 1 && robTaskGroupResouce(taskGroupQueue)) {
+ return true;
+ }
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return false;
+ }
+
+ public boolean robTaskGroupResouce(TaskGroupQueue taskGroupQueue) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(taskGroupQueue.getGroupId());
+ int affectedCount = taskGroupMapper.updateTaskGroupResource(taskGroup.getId(),taskGroupQueue.getId(),
+ TaskGroupQueueStatus.WAIT_QUEUE.getCode());
+ if (affectedCount > 0) {
+ taskGroupQueue.setStatus(TaskGroupQueueStatus.ACQUIRE_SUCCESS);
+ this.taskGroupQueueMapper.updateById(taskGroupQueue);
+ this.taskGroupQueueMapper.updateInQueue(Flag.NO.getCode(), taskGroupQueue.getId());
+ return true;
+ }
+ return false;
+ }
+
+ public boolean acquireTaskGroupAgain(TaskGroupQueue taskGroupQueue) {
Review comment:
Is it better to call the `robTaskGroupResouce` directly?
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
Review comment:
Please add some comments for what this function to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-980980682
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979650807
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759172649
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java
##########
@@ -70,6 +70,18 @@ public boolean submit(TaskInstance task, ProcessInstance processInstance, int ma
if (this.taskInstance == null) {
return false;
}
+ int taskGroupId = task.getTaskGroupId();
Review comment:
Task group is effect on the task of shell、mr、spark and some type of task execute on worker。It's not a node type。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-975431459
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [32 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0f8b962) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/e6fe39ea47f3168201f5d7f8c8a5340d075d2379?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e6fe39e) will **decrease** coverage by `0.37%`.
> The diff coverage is `3.96%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.78% 32.40% -0.38%
- Complexity 1621 1623 +2
============================================
Files 432 435 +3
Lines 14385 14584 +199
Branches 1446 1471 +25
============================================
+ Hits 4716 4726 +10
- Misses 9217 9400 +183
- Partials 452 458 +6
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `5.18% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `5.47% <0.00%> (-0.98%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [e6fe39e...0f8b962](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979771560
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979803833
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759154412
##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java
##########
@@ -70,6 +70,18 @@ public boolean submit(TaskInstance task, ProcessInstance processInstance, int ma
if (this.taskInstance == null) {
return false;
}
+ int taskGroupId = task.getTaskGroupId();
Review comment:
No the task status, I mean the new task type, such as switchTask, sqlTask.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (24b9381) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/1a88b1863b9000d7e220461ed44de7c6ab9a8169?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a88b18) will **decrease** coverage by `0.40%`.
> The diff coverage is `3.96%`.
> :exclamation: Current head 24b9381 differs from pull request most recent head e8f8219. Consider uploading reports for the commit e8f8219 to get more accurate results
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.67% 32.27% -0.41%
- Complexity 1607 1610 +3
============================================
Files 431 434 +3
Lines 14340 14539 +199
Branches 1437 1462 +25
============================================
+ Hits 4686 4692 +6
- Misses 9203 9394 +191
- Partials 451 453 +2
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `5.18% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `5.47% <0.00%> (-0.98%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1a88b18...e8f8219](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979127758
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979803833
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo merged pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
lenboo merged pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759132546
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskGroupServiceImpl.java
##########
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.service.impl;
+
+import org.apache.dolphinscheduler.api.enums.Status;
+import org.apache.dolphinscheduler.api.service.TaskGroupQueueService;
+import org.apache.dolphinscheduler.api.service.TaskGroupService;
+import org.apache.dolphinscheduler.api.utils.PageInfo;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.dolphinscheduler.common.enums.Flag;
+import org.apache.dolphinscheduler.dao.entity.TaskGroup;
+import org.apache.dolphinscheduler.dao.entity.User;
+import org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper;
+import org.apache.dolphinscheduler.service.process.ProcessService;
+import org.apache.dolphinscheduler.spi.utils.StringUtils;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+
+import com.baomidou.mybatisplus.core.metadata.IPage;
+import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
+
+/**
+ * task Group Service
+ */
+@Service
+public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupService {
+
+ @Autowired
+ private TaskGroupMapper taskGroupMapper;
+
+ @Autowired
+ private TaskGroupQueueService taskGroupQueueService;
+
+ @Autowired
+ private ProcessService processService;
+
+ private static final Logger logger = LoggerFactory.getLogger(TaskGroupServiceImpl.class);
+
+ /**
+ * create a Task group
+ *
+ * @param loginUser login user
+ * @param name task group name
+ * @param description task group description
+ * @param groupSize task group total size
+ * @return the result code and msg
+ */
+ @Override
+ public Map<String, Object> createTaskGroup(User loginUser, String name, String description, int groupSize) {
+ Map<String, Object> result = new HashMap<>();
+ if (isNotAdmin(loginUser, result)) {
+ return result;
+ }
+ if (name == null) {
+ putMsg(result, Status.NAME_NULL);
+ return result;
+ }
+ if (groupSize <= 0) {
+ putMsg(result, Status.TASK_GROUP_SIZE_ERROR);
+ return result;
+
+ }
+ TaskGroup taskGroup1 = taskGroupMapper.queryByName(loginUser.getId(), name);
+ if (taskGroup1 != null) {
+ putMsg(result, Status.TASK_GROUP_NAME_EXSIT);
+ return result;
+ }
+ TaskGroup taskGroup = new TaskGroup(name, description,
+ groupSize, loginUser.getId());
+ int insert = taskGroupMapper.insert(taskGroup);
+ logger.info("insert result:{}", insert);
+ putMsg(result, Status.SUCCESS);
+
+ return result;
+ }
+
+ /**
+ * update the task group
+ *
+ * @param loginUser login user
+ * @param name task group name
+ * @param description task group description
+ * @param groupSize task group total size
+ * @return the result code and msg
+ */
+ @Override
+ public Map<String, Object> updateTaskGroup(User loginUser, int id, String name, String description, int groupSize) {
+ Map<String, Object> result = new HashMap<>();
+ if (isNotAdmin(loginUser, result)) {
+ return result;
+ }
+ TaskGroup taskGroup = taskGroupMapper.selectById(id);
+ if (taskGroup.getStatus() != Flag.NO.getCode()) {
+ putMsg(result, Status.TASK_GROUP_STATUS_ERROR);
+ return result;
+ }
+ taskGroup.setGroupSize(groupSize);
+ taskGroup.setDescription(description);
+ if (StringUtils.isNotEmpty(name)) {
+ taskGroup.setName(name);
+ }
+ int i = taskGroupMapper.updateById(taskGroup);
+ logger.info("update result:{}", i);
+ putMsg(result, Status.SUCCESS);
+ return result;
+ }
+
+ /**
+ * get task group status
+ *
+ * @param id task group id
+ * @return is the task group available
+ */
+ @Override
+ public boolean isTheTaskGroupAvailable(int id) {
+ TaskGroup taskGroup = taskGroupMapper.selectById(id);
Review comment:
i will fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-982516976
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r760107307
##########
File path: dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
##########
@@ -2568,6 +2581,174 @@ private void addAuthorizedResources(List<Resource> ownResources, int userId) {
return processTaskMap;
}
+ /**
+ * @param taskId task id
Review comment:
already fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b8a222e) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/4c4ebb89247af024ed4bf879fc90b280319bc488?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4c4ebb8) will **decrease** coverage by `0.44%`.
> The diff coverage is `3.96%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.66% 32.22% -0.45%
- Complexity 1604 1607 +3
============================================
Files 431 434 +3
Lines 14342 14541 +199
Branches 1437 1462 +25
============================================
+ Hits 4685 4686 +1
- Misses 9208 9404 +196
- Partials 449 451 +2
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `5.18% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `5.47% <0.00%> (-0.98%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4c4ebb8...b8a222e](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-980980682
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759067207
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupMapper.xml
##########
@@ -0,0 +1,87 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroup" id="TaskGroupMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="name" column="name" jdbcType="VARCHAR"/>
+ <result property="description" column="description" jdbcType="VARCHAR"/>
+ <result property="groupSize" column="group_size" jdbcType="INTEGER"/>
+ <result property="useSize" column="use_size" jdbcType="INTEGER"/>
+ <result property="userId" column="user_id" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="VARCHAR"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id,name,description,group_size,use_size,create_time,update_time
+ </sql>
+
+ <select id="queryTaskGroupPaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroup">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group
+ <where>
+ <if test="userId != 0">
+ and user_id = #{userId}
+ </if>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ <if test="name != null and name != '' ">
+ and name like concat('%', #{name}, '%')
+ </if>
+ </where>
+ </select>
+
+ <!--modify data by id-->
+ <update id="updateTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size+1
+ where id = #{id} and use_size < group_size and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
Review comment:
one SQL can ensure the atomicity.if split it , will increases the probability of multiple executions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759067640
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupMapper.xml
##########
@@ -0,0 +1,87 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroup" id="TaskGroupMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="name" column="name" jdbcType="VARCHAR"/>
+ <result property="description" column="description" jdbcType="VARCHAR"/>
+ <result property="groupSize" column="group_size" jdbcType="INTEGER"/>
+ <result property="useSize" column="use_size" jdbcType="INTEGER"/>
+ <result property="userId" column="user_id" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="VARCHAR"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id,name,description,group_size,use_size,create_time,update_time
+ </sql>
+
+ <select id="queryTaskGroupPaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroup">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group
+ <where>
+ <if test="userId != 0">
+ and user_id = #{userId}
+ </if>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ <if test="name != null and name != '' ">
+ and name like concat('%', #{name}, '%')
+ </if>
+ </where>
+ </select>
+
+ <!--modify data by id-->
+ <update id="updateTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size+1
+ where id = #{id} and use_size < group_size and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
+ </update>
+
+ <!--modify data by id-->
+ <update id="releaseTaskGroupResource">
+ update t_ds_task_group
+ set use_size = use_size-1
+ where id = #{id} and use_size > 0 and
+ (select count(1) FROM t_ds_task_group_queue where id = #{queueId} and status = #{queueStatus} ) = 1
Review comment:
the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759071644
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupQueueMapper.xml
##########
@@ -0,0 +1,127 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue" id="TaskGroupQueueMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="taskId" column="task_id" jdbcType="INTEGER"/>
+ <result property="taskName" column="task_name" jdbcType="VARCHAR"/>
+ <result property="groupId" column="group_id" jdbcType="INTEGER"/>
+ <result property="processId" column="process_id" jdbcType="INTEGER"/>
+ <result property="priority" column="priority" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="INTEGER"/>
+ <result property="forceStart" column="force_start" jdbcType="INTEGER"/>
+ <result property="inQueue" column="in_queue" jdbcType="INTEGER"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id, task_id, task_name, group_id, process_id, priority, status , force_start , in_queue, create_time, update_time
+ </sql>
+
+ <select id="queryTaskGroupQueuePaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="groupId != 0">
+ and group_id = #{groupId}
+ </if>
+ <if test="processId != 0">
+ and process_id = #{processId}
+ </if>
+ </where>
+ </select>
+
+ <select id="queryByStatus" resultMap="TaskGroupQueueMap" resultType="map">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="status != 0">
Review comment:
this is a unnecessary part ,I will delete it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-983556478
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [41.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-982516976
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-985233881
@wangxj3
Thank you for your contributions.
i think this feature is OK now, but still there have some works to do.
1. documents about this feature
2. upgrade sql-schema in directory sql/upgrade/2.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-975431459
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [32 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979127758
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL) [29 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6722&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list) [2.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6722&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a113b3b) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/1a88b1863b9000d7e220461ed44de7c6ab9a8169?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a88b18) will **decrease** coverage by `0.45%`.
> The diff coverage is `3.96%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 32.64% 32.19% -0.46%
- Complexity 1604 1605 +1
============================================
Files 431 434 +3
Lines 14340 14539 +199
Branches 1437 1462 +25
============================================
- Hits 4682 4681 -1
- Misses 9208 9409 +201
+ Partials 450 449 -1
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `5.18% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `5.47% <0.00%> (-0.98%)` | :arrow_down: |
| [...lphinscheduler/service/process/ProcessService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvcHJvY2Vzcy9Qcm9jZXNzU2VydmljZS5qYXZh) | `36.00% <6.09%> (-2.31%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1a88b18...a113b3b](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r760107071
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TaskGroupController.java
##########
@@ -0,0 +1,298 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.controller;
+
+import static org.apache.dolphinscheduler.api.enums.Status.CLOSE_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.CREATE_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_GROUP_LIST_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_GROUP_QUEUE_LIST_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.START_TASK_GROUP_ERROR;
+import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_TASK_GROUP_ERROR;
+
+import org.apache.dolphinscheduler.api.aspect.AccessLogAnnotation;
+import org.apache.dolphinscheduler.api.exceptions.ApiException;
+import org.apache.dolphinscheduler.api.service.TaskGroupQueueService;
+import org.apache.dolphinscheduler.api.service.TaskGroupService;
+import org.apache.dolphinscheduler.api.utils.Result;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.dolphinscheduler.dao.entity.User;
+
+import java.util.Map;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
+import org.springframework.web.bind.annotation.GetMapping;
+import org.springframework.web.bind.annotation.PostMapping;
+import org.springframework.web.bind.annotation.RequestAttribute;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestParam;
+import org.springframework.web.bind.annotation.ResponseStatus;
+import org.springframework.web.bind.annotation.RestController;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiImplicitParam;
+import io.swagger.annotations.ApiImplicitParams;
+import io.swagger.annotations.ApiOperation;
+import springfox.documentation.annotations.ApiIgnore;
+
+
+/**
+ * task group controller
+ */
+@Api(tags = "task group")
+@RestController
+@RequestMapping("/task-group")
+public class TaskGroupController extends BaseController {
+
+ @Autowired
+ private TaskGroupService taskGroupService;
+
+ /**
+ * query task group list
+ *
+ * @param loginUser login user
+ * @param name name
+ * @param description description
+ * @param groupSize group size
+ * @param name project id
+ * @return result and msg code
+ */
+ @ApiOperation(value = "createTaskGroup", notes = "CREATE_TAKS_GROUP_NOTE")
+ @ApiImplicitParams({
+ @ApiImplicitParam(name = "name", value = "NAME", dataType = "String"),
+ @ApiImplicitParam(name = "description", value = "DESCRIPTION", dataType = "String"),
+ @ApiImplicitParam(name = "groupSize", value = "GROUPSIZE", dataType = "Int"),
+
+ })
+ @PostMapping(value = "/create")
+ @ResponseStatus(HttpStatus.CREATED)
+ @ApiException(CREATE_TASK_GROUP_ERROR)
+ @AccessLogAnnotation(ignoreRequestArgs = "loginUser")
+ public Result createTaskGroup(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
+ @RequestParam("name") String name,
+ @RequestParam("description") String description,
+ @RequestParam("groupSize") Integer groupSize) {
+ User user = new User();
Review comment:
already fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter edited a comment on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-979648970
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6722](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cea203a) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/173a3856185abc0fa9be16715d2567ebbe054a6f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (173a385) will **decrease** coverage by `0.00%`.
> The diff coverage is `42.76%`.
> :exclamation: Current head cea203a differs from pull request most recent head 00fe466. Consider uploading reports for the commit 00fe466 to get more accurate results
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6722 +/- ##
============================================
- Coverage 40.44% 40.44% -0.01%
- Complexity 3594 3671 +77
============================================
Files 647 655 +8
Lines 26316 26757 +441
Branches 2926 2968 +42
============================================
+ Hits 10643 10821 +178
- Misses 14699 14947 +248
- Partials 974 989 +15
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cheduler/api/service/impl/ExecutorServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9FeGVjdXRvclNlcnZpY2VJbXBsLmphdmE=) | `47.13% <0.00%> (-2.02%)` | :arrow_down: |
| [.../dolphinscheduler/common/enums/StateEventType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1N0YXRlRXZlbnRUeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...inscheduler/common/enums/TaskGroupQueueStatus.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Rhc2tHcm91cFF1ZXVlU3RhdHVzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...heduler/remote/command/TaskEventChangeCommand.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2NvbW1hbmQvVGFza0V2ZW50Q2hhbmdlQ29tbWFuZC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/master/MasterServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9NYXN0ZXJTZXJ2ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/processor/TaskEventProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvVGFza0V2ZW50UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ter/processor/queue/StateEventResponseService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvU3RhdGVFdmVudFJlc3BvbnNlU2VydmljZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `10.08% <0.00%> (-0.58%)` | :arrow_down: |
| [...r/server/master/runner/task/BaseTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9CYXNlVGFza1Byb2Nlc3Nvci5qYXZh) | `4.92% <0.00%> (-0.04%)` | :arrow_down: |
| [...server/master/runner/task/CommonTaskProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvdGFzay9Db21tb25UYXNrUHJvY2Vzc29yLmphdmE=) | `4.10% <0.00%> (-0.73%)` | :arrow_down: |
| ... and [14 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6722/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [173a385...00fe466](https://codecov.io/gh/apache/dolphinscheduler/pull/6722?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759082935
##########
File path: dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
##########
@@ -1017,3 +1020,40 @@ CREATE TABLE `t_ds_environment_worker_group_relation` (
PRIMARY KEY (`id`),
UNIQUE KEY `environment_worker_group_unique` (`environment_code`,`worker_group`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
+
+-- ----------------------------
+-- Table structure for t_ds_task_group_queue
+-- ----------------------------
+DROP TABLE IF EXISTS `t_ds_task_group_queue`;
+CREATE TABLE `t_ds_task_group_queue` (
+ `id` int(11) NOT NULL AUTO_INCREMENT COMMENT'key',
+ `task_id` int(11) DEFAULT NULL COMMENT 'taskintanceid',
+ `task_name` varchar(100) DEFAULT NULL COMMENT 'TaskInstance name',
+ `group_id` int(11) DEFAULT NULL COMMENT 'taskGroup id',
+ `process_id` int(11) DEFAULT NULL COMMENT 'processInstace id',
+ `priority` int(8) DEFAULT '0' COMMENT 'priority',
+ `status` tinyint(4) DEFAULT '-1' COMMENT '-1: waiting 1: running 2: finished',
+ `force_start` tinyint(4) DEFAULT '0' COMMENT 'is force start 0 NO ,1 YES',
+ `in_queue` tinyint(4) DEFAULT '0' COMMENT 'ready to get the queue by other task finish 0 NO ,1 YES',
+ `create_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
+ `update_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+ PRIMARY KEY( `id` )
Review comment:
There are many different query scenarios, and the effect of adding indexes is not obvious. We will consider whether to add task_id and group_id later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangxj3 commented on a change in pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
wangxj3 commented on a change in pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#discussion_r759078247
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskGroupQueueMapper.xml
##########
@@ -0,0 +1,127 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
+<mapper namespace="org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper">
+
+ <resultMap type="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue" id="TaskGroupQueueMap">
+ <result property="id" column="id" jdbcType="INTEGER"/>
+ <result property="taskId" column="task_id" jdbcType="INTEGER"/>
+ <result property="taskName" column="task_name" jdbcType="VARCHAR"/>
+ <result property="groupId" column="group_id" jdbcType="INTEGER"/>
+ <result property="processId" column="process_id" jdbcType="INTEGER"/>
+ <result property="priority" column="priority" jdbcType="INTEGER"/>
+ <result property="status" column="status" jdbcType="INTEGER"/>
+ <result property="forceStart" column="force_start" jdbcType="INTEGER"/>
+ <result property="inQueue" column="in_queue" jdbcType="INTEGER"/>
+ <result property="createTime" column="create_time" jdbcType="TIMESTAMP"/>
+ <result property="updateTime" column="update_time" jdbcType="TIMESTAMP"/>
+ </resultMap>
+
+ <sql id = "baseSql">
+ id, task_id, task_name, group_id, process_id, priority, status , force_start , in_queue, create_time, update_time
+ </sql>
+
+ <select id="queryTaskGroupQueuePaging" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="groupId != 0">
+ and group_id = #{groupId}
+ </if>
+ <if test="processId != 0">
+ and process_id = #{processId}
+ </if>
+ </where>
+ </select>
+
+ <select id="queryByStatus" resultMap="TaskGroupQueueMap" resultType="map">
+ select
+ <include refid="baseSql">
+ </include>
+ from t_ds_task_group_queue
+ <where>
+ <if test="status != 0">
+ and status = #{status}
+ </if>
+ </where>
+ </select>
+
+ <delete id="deleteByTaskId">
+ delete from t_ds_task_group_queue
+ where task_id = #{taskId}
+ </delete>
+
+ <update id="updateStatusByTaskId">
+ update t_ds_task_group_queue
+ <set>
+ <if test="status != 0">
+ status = #{status},
+ </if>
+ </set>
+ where task_id = #{taskId}
+ </update>
+
+ <update id="updateInQueue">
+ update t_ds_task_group_queue
+ set in_queue = #{inQueue}
+ where id = #{id}
+ </update>
+
+ <update id="updateForceStart">
+ update t_ds_task_group_queue
+ set force_start = #{forceStart}
+ where task_id = #{taskId}
+ </update>
+
+ <update id="updateInQueueLimit1">
+ update t_ds_task_group_queue
+ set in_queue = #{newValue}
+ where group_id = #{groupId} and in_queue = #{oldValue} and status = #{status} order by priority desc limit 1
+ </update>
+
+ <update id="updateInQueueCAS">
+ update t_ds_task_group_queue
+ set in_queue = #{newValue}
+ where id = #{id} and in_queue = #{oldValue}
+ </update>
+
+ <select id="queryHighPriorityTasks" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql" />
+ from t_ds_task_group_queue
+ where group_id = #{groupId} and status = #{status} and priority > #{priority}
+ </select>
+ <select id="queryTheHighestPriorityTasks" resultType="org.apache.dolphinscheduler.dao.entity.TaskGroupQueue">
+ select
+ <include refid="baseSql" />
+ from t_ds_task_group_queue
+ where priority = (select max(priority) from t_ds_task_group_queue where group_id = #{groupId}
Review comment:
if the result of sub sql is null ,this sql will return null ,this is the result we want
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on pull request #6722: [Feature-#6422] task group queue
Posted by GitBox <gi...@apache.org>.
lenboo commented on pull request #6722:
URL: https://github.com/apache/dolphinscheduler/pull/6722#issuecomment-985233881
i think this feature is OK now, but still there have some works to do.
1. documents about this feature
2. upgrade sql-schema in directory sql/upgrade/2.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org