You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "kittimzhe (via GitHub)" <gi...@apache.org> on 2026/10/15 09:29:44 UTC

[PR] [Fix-18698][Master] Reconcile stale serial commands whose workflow instance is already final [dolphinscheduler]

kittimzhe opened a new pull request, #18709:
URL: https://github.com/apache/dolphinscheduler/pull/18709

   ## Purpose of the Pull Request
   
   Reclaim serial-command rows whose workflow instance has already reached a final state (or is gone), so a serial queue whose cleanup transaction never ran can drain again. This implements the reconciliation suggested in #18698.
   
   ## Brief Change Log
   
   `WorkflowSerialCoordinator`'s poll loop gains a reconciliation pass, run before this round's groups are assembled:
   
   - a serial command whose workflow instance is **missing** is recycled;
   - a serial command whose workflow instance is **already in a final state** (`WorkflowExecutionStatus.isFinalState()`) is recycled;
   - recycled rows are removed from the in-memory list too, so no handler is handed a dead workflow instance in the same round;
   - everything else — `SERIAL_WAIT` while queued, `RUNNING_EXECUTION` while running — is untouched.
   
   ## Why this is safe for live entries
   
   - While an entry waits in the queue, its instance holds the non-final `SERIAL_WAIT` state, and that state is **persisted before the serial command row is inserted, in one transaction** (`AbstractWorkflowTrigger` / `AbstractWorkflowInstanceTrigger` both do `workflowInstanceDao.updateById/insert(SERIAL_WAIT)` then `serialCommandDao.insert` under `@Transactional`). So a visible row always implies a non-final instance state at insert time.
   - The pass is idempotent with the normal cleanup: `AbstractWorkflowStateAction#workflowFinish` deletes the same rows in its finish transaction; whichever runs first or last, both target the same rows, and final states are monotonic.
   
   ## Verification
   
   - `mvn test -pl dolphinscheduler-master -Dtest=WorkflowSerialCoordinatorTest` — 8/8 green, including 5 new cases covering: final-state instance → recycled; missing instance → recycled; null workflowInstanceId → recycled (no instance lookup); `SERIAL_WAIT`/`RUNNING_EXECUTION` → kept; empty input → no DAO calls.
   - `mvn spotless:check -pl dolphinscheduler-master` — clean.
   - `mvn compile -pl dolphinscheduler-master -am` — clean.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18698][Master] Reconcile stale serial commands whose workflow instance is already final [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on PR #18709:
URL: https://github.com/apache/dolphinscheduler/pull/18709#issuecomment-6189543254

   Closing this PR for now until reached a consensus.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix-18698][Master] Reconcile stale serial commands whose workflow instance is already final [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed pull request #18709: [Fix-18698][Master] Reconcile stale serial commands whose workflow instance is already final
URL: https://github.com/apache/dolphinscheduler/pull/18709


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org