You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "joeyutong (via GitHub)" <gi...@apache.org> on 2026/10/10 11:23:44 UTC
[I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
joeyutong opened a new issue, #1218:
URL: https://github.com/apache/flink-agents/issues/1218
## Description
Track lifecycle and metric bookkeeping gaps in the JDK < 21 parallel action execution engine.
- [ ] **Correct the listener threading contract.** `TaskLifecycleListener` and `ComponentExecutionListener` promise mailbox-thread callbacks, but the parallel engine also invokes callbacks on action workers. Document the execution-context contract without promising fixed mailbox-thread affinity. ([Source](https://github.com/apache/flink-agents/blob/bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138/runtime/src/main/java/org/apache/flink/agents/runtime/lifecycle/TaskLifecycleListener.java#L24-L41))
- [ ] **Emit `onRecordStart` when restoring parallel execution.** The parallel restore path requeues unfinished tasks without notifying record start, while completion still emits `onRecordFinished`. Restore should preserve the start/finish pairing, as the serial path does. ([Source](https://github.com/apache/flink-agents/blob/bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138/runtime/src/main/java/org/apache/flink/agents/runtime/operator/ActionExecutionOperator.java#L1627-L1647))
- [ ] **Restore subagent task binding after async resumption.** If action A waits on an async call and action B prepares or finishes, `BaseSubagentSetup.currentTask` becomes B or null. Resuming A restores its runner context but not this binding, so a subsequent subagent submission can use the wrong task's allocator or fail with `No prepared action task to assign sub-agent ids from.` ([Source](https://github.com/apache/flink-agents/blob/bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138/runtime/src/main/java/org/apache/flink/agents/runtime/operator/ActionTaskContextManager.java#L362-L399))
- [ ] **Record input-run failure metrics on the parallel failure path.** An action failure is propagated through `Work.commit()` without calling `markInputRunFailed`. For a fresh input run, this misses `numOfInputRunsFailed`, the active-run gauge decrement, and terminal samples for `inputRunLatencyMs` and `inputRunProcessingLatencyMs`. ([Source](https://github.com/apache/flink-agents/blob/bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138/runtime/src/main/java/org/apache/flink/agents/runtime/operator/ActionExecutionOperator.java#L1847-L1851))
- [ ] **Keep restored synthetic no-op tasks out of action metrics.** When an input matches no action, the parallel engine creates a synthetic no-op task. Restoring a checkpoint containing that queued task increments its `numOfPendingActionTasks`, but dequeue and completion exclude it from action metrics. The gauge therefore remains nonzero after completion. ([Source](https://github.com/apache/flink-agents/blob/bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138/runtime/src/main/java/org/apache/flink/agents/runtime/operator/ActionExecutionOperator.java#L1575-L1607))
## Version
Source checked against `main` at `bd527d7fa8de9083ccdd2a98e1c4dbdef5a98138`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "pltbkd (via GitHub)" <gi...@apache.org>.
pltbkd commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6306630626
I've filed #1299 to run subgraphs on the parallel engine without continuations; it also covers item 3. Reviews welcome if you have bandwidth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "sangkyoonnam (via GitHub)" <gi...@apache.org>.
sangkyoonnam commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6115132529
If you're not already working on them, I'd like to take items 2, 4 and 5 in one PR with a regression test for each.
I checked bd527d7f: the parallel restore path requeues tasks without `notifyRecordStart`, parallel action failures propagate through `Work.commit()` without `markInputRunFailed`, and restored synthetic no-op tasks increment the pending-task gauge without a matching decrement.
I'd keep 1 and 3 out of this PR, since 1 concerns the listener contract and 3 changes the subagent binding. Could you assign this to me if that scope works?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "pltbkd (via GitHub)" <gi...@apache.org>.
pltbkd commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6117692274
Thanks for reporting this, and thanks @CodeTrainerMan and @sangkyoonnam for offering to help.
I’ve worked on both subagents and parallel execution, and I’m currently working on their integration. The plan is to align the parallel engine’s lifecycle semantics with coroutine-based execution, addressing item 3. It also covers early bootstrap dispatch for internal subagents, allowing child actions to run while the caller waits, and provides a shared foundation for the corresponding work in #1220.
Our approaches to item 2 are consistent, so I’ll base my changes on #1224 and reuse its fix rather than duplicate it. Would it be okay for me to take item 3, with #1224 continuing to cover items 1, 2, 4, and 5?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "joeyutong (via GitHub)" <gi...@apache.org>.
joeyutong commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6117138370
Thanks for offering to help, @sangkyoonnam! @CodeTrainerMan has already opened #1224 for these items. Would you be interested in helping review that PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "CodeTrainerMan (via GitHub)" <gi...@apache.org>.
CodeTrainerMan commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6114334256
I'd like to take items 1, 2, 4 and 5 of this issue:
- **1**: fix the thread contract documented on `TaskLifecycleListener` and `ComponentExecutionListener`, so they stop promising mailbox-thread callbacks that the parallel engine does not honor.
- **2**: emit `onRecordStart` when the restore path requeues unfinished tasks, once per key, so the resumed round pairs with the `onRecordFinished` emitted on completion.
- **4**: mark the input run failed before propagating an action failure from the parallel commit path, so `numOfInputRunsFailed` is counted and the active-run gauge and the terminal latency samples are released.
- **5**: skip the synthetic no-op input action when restoring action metrics, matching the dequeue path, so the pending-action-task gauge returns to zero.
Item 3 (rebinding subagent tasks on restore) touches the lock ordering in `ParallelExecutionLock`, so I would leave it to someone with more context on the parallel engine.
I'll open a PR covering those four items. Happy to split it into separate PRs if you prefer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][runtime] Lifecycle and metrics inconsistencies in parallel action execution [flink-agents]
Posted by "CodeTrainerMan (via GitHub)" <gi...@apache.org>.
CodeTrainerMan commented on issue #1218:
URL: https://github.com/apache/flink-agents/issues/1218#issuecomment-6142216967
Thanks @joeyutong — that split works for me. #1224 will keep covering items 1, 2, 4, and 5, and item 3 is yours.
One update on #1224 since this was filed: review also caught a related bug on the same failure path — a preparation-time failure (e.g. a `TaskLifecycleListener` throwing in `onActionPrepared`) surfaced as a `NullPointerException` from `Work.commit()` instead of the original exception, because `actionTask` was dereferenced before it was bound. The fix binds `actionTask` right after the poll and guards the `markInputRunFailed` call, so the input run is now marked failed even on a preparation failure (item 4's intent). It's covered by the new regression test `preparationFailureKeepsCauseAndMarksInputRunFailed`.
Thanks @sangkyoonnam for reviewing, and glad the item-2 approaches line up — looking forward to basing item 3 on #1224.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org