You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "eye-gu (via GitHub)" <gi...@apache.org> on 2026/06/12 09:11:55 UTC

[PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

eye-gu opened a new pull request, #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344

   <!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
   
   ## Was this PR generated or assisted by AI?
   
   Add unit tests.
   
   ## Purpose of the pull request
   
   fix #18338
   
   ## Brief change log
   
   <!--*(for example:)*
   - *Add maven-checkstyle-plugin to root pom.xml*
   -->
   
   ## Verify this pull request
   
   - Added TaskSubmittedStateActionTest
   - Added TaskGroupCoordinatorTest#isTaskWaitingForTaskGroupSlot
   
   ## Pull Request Notice
   [Pull Request Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
   
   If your pull request contains incompatible change, you should also add it to `docs/docs/en/guide/upgrade/incompatible.md`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344#issuecomment-4690187960

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=18344) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=18344&metric=new_coverage&view=list) (required ≥ 60%)  
     
   <!-- sqra:qg_revision=184a56e79b09572a8f677db63e8a15842bedec63279916924e37cd83e67fccc8 -->
   <!-- sqra-placement-anchor -->
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=18344)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344#issuecomment-4691640932

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=18344) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=18344&metric=new_coverage&view=list) (required ≥ 60%)  
     
   <!-- sqra:qg_revision=184a56e79b09572a8f677db63e8a15842bedec63279916924e37cd83e67fccc8 -->
   <!-- sqra-placement-anchor -->
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=18344)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "eye-gu (via GitHub)" <gi...@apache.org>.
eye-gu commented on code in PR #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344#discussion_r3409575448


##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/engine/task/statemachine/TaskSubmittedStateAction.java:
##########
@@ -143,6 +143,15 @@ public void onPauseEvent(final IWorkflowExecution workflowExecution,
             taskExecution.getWorkflowEventBus().publish(TaskPausedLifecycleEvent.of(taskExecution));
             return;
         }
+        // Task not in dispatch queue, check if it's waiting for TaskGroup slot
+        if (taskGroupCoordinator.isTaskWaitingForTaskGroupSlot(taskExecution.getTaskInstance())) {
+            // Release the TaskGroupQueue first to prevent TaskGroupCoordinator
+            // from concurrently acquiring the slot and racing with this pause
+            taskGroupCoordinator.releaseTaskGroupSlot(taskExecution.getTaskInstance());
+            log.info("Task: {} is waiting for TaskGroup slot, pause it directly", taskExecution.getName());
+            taskExecution.getWorkflowEventBus().publish(TaskPausedLifecycleEvent.of(taskExecution));
+            return;
+        }

Review Comment:
   You're right, there is a concurrency issue with the TaskGroupCoordinator's periodic task. However, the leaked slot will be fixed by amendTaskGroupUseSize eventually, and after the task enters PAUSE state, the dispatch event triggered by TGC's RPC will be safely ignored (just a warn log). So in practice there's no real impact. Is this acceptable? I haven't come up with a clean way to fully fix the concurrency yet.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed pull request #18344: [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill
URL: https://github.com/apache/dolphinscheduler/pull/18344


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344#discussion_r3409514231


##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/engine/task/statemachine/TaskSubmittedStateAction.java:
##########
@@ -143,6 +143,15 @@ public void onPauseEvent(final IWorkflowExecution workflowExecution,
             taskExecution.getWorkflowEventBus().publish(TaskPausedLifecycleEvent.of(taskExecution));
             return;
         }
+        // Task not in dispatch queue, check if it's waiting for TaskGroup slot
+        if (taskGroupCoordinator.isTaskWaitingForTaskGroupSlot(taskExecution.getTaskInstance())) {
+            // Release the TaskGroupQueue first to prevent TaskGroupCoordinator
+            // from concurrently acquiring the slot and racing with this pause
+            taskGroupCoordinator.releaseTaskGroupSlot(taskExecution.getTaskInstance());
+            log.info("Task: {} is waiting for TaskGroup slot, pause it directly", taskExecution.getName());
+            taskExecution.getWorkflowEventBus().publish(TaskPausedLifecycleEvent.of(taskExecution));
+            return;
+        }

Review Comment:
   This may result in a state error due to concurrency.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18338] check task if it's waiting for TaskGroup slot when pause/kill [dolphinscheduler]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #18344:
URL: https://github.com/apache/dolphinscheduler/pull/18344#discussion_r3402354013


##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/engine/TaskGroupCoordinator.java:
##########
@@ -317,6 +317,16 @@ private void dealWithWaitingTaskGroupQueue() {
         }
     }
 
+    @Override
+    public boolean isTaskWaitingForTaskGroupSlot(final TaskInstance taskInstance) {
+        if (!TaskGroupUtils.isUsingTaskGroup(taskInstance)) {
+            return false;
+        }
+        List<TaskGroupQueue> taskGroupQueues = taskGroupQueueDao.queryByTaskInstanceId(taskInstance.getId());
+        return taskGroupQueues.stream()
+                .anyMatch(q -> q.getStatus() == TaskGroupQueueStatus.WAIT_QUEUE);
+    }

Review Comment:
   `isTaskWaitingForTaskGroupSlot` can throw `NullPointerException` when `taskInstance` is null (since `TaskGroupUtils.isUsingTaskGroup` dereferences it) or when `taskInstance.getId()` is null (DB query + stream). This method is used from pause/kill handling, so a defensive null/empty check would prevent unexpected failures in edge cases (e.g., partially-initialized task executions).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org