You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by "yrenat (via GitHub)" <gi...@apache.org> on 2026/07/14 02:07:46 UTC
[D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]
GitHub user yrenat created a discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization
## Motivation
Texera allows users to design and execute complex workflows, but during execution it is still difficult to understand where data flow is being constrained. When a workflow becomes slow or appears stuck, users may want to know:
- Which part of the workflow is currently under pressure?
- Is an operator slow because it is processing data, waiting for upstream input, or blocked by downstream backpressure?
- Which workflow edge is accumulating pressure?
- Is the bottleneck caused by data flow between operators rather than by the operator state alone?
Texera already exposes multiple operator-level runtime information, such as execution state and operator statistics. However, flow-control/backpressure information is currently kept mostly inside the execution engine and is not visible in the workflow graph to the users.
This proposal focuses on exposing workflow-level flow-control status so users can better understand runtime bottlenecks without changing the underlying execution behavior.
## Proposed Solution
This proposal focuses on exposing workflow-level flow-control status by reusing Amber’s existing credit-based flow-control state.
Amber already tracks per-channel flow-control information, such as available credit, queued credit, and whether a channel is overloaded. The backend should periodically summarize this channel-level state into workflow-level link pressure information and publish it through the existing workflow websocket event mechanism.
The first version is designed to expose a compact runtime event containing, for each affected logical workflow link:
- Source operator id.
- Target operator id.
- Target input port id.
- Whether the link is currently overloaded.
- Current/max credit usage, or utilization percentage.
- Optional aggregated queued/inflight byte counts.
Because physical execution may use multiple worker-to-worker channels for one logical workflow edge, the backend should aggregate channel-level signals into one logical-link status before sending it to the frontend.
The frontend can then render this as a runtime overlay on the workflow graph. Edges under pressure can be highlighted by color, with compact labels or tooltips showing utilization details.
## Backend Design
The backend will reuse the existing flow-control implementation instead of introducing a separate monitoring system.
The expected backend work includes:
1. Collect flow-control snapshots from existing runtime state.
Amber already maintains per-channel flow-control state internally. The implementation should expose a read-only snapshot of relevant values, such as:
- available credit
- maximum credit
- queued credit
- overloaded/backpressured status
2. Aggregate physical channel state into logical workflow-link state.
Flow control is tracked at the worker-channel level, while the frontend visualizes logical workflow links. Therefore, the backend should group channel-level information by logical workflow connection, such as:
```text
worker channel metrics
-> source logical operator + target logical operator + target input port
-> logical link pressure status
3. Publish a compact websocket event.
A possible event shape is:
```text
FlowControlStatusUpdateEvent {
links: {
sourceOperatorId: string;
targetOperatorId: string;
targetPortId: string;
overloaded: boolean;
queuedCreditBytes?: number;
availableCreditBytes?: number;
maxCreditBytes?: number;
utilization?: number;
};
}
## Frontend Design
The frontend will use the new flow-control status event and render it as a workflow graph overlay.
The first version may have a focus on edge-level visualization:
- Use edge color to indicate pressure severity.
- Show compact edge labels only when useful, such as when pressure is non-zero or when a runtime/debug overlay is enabled.
- Show detailed values, such as utilization percentage or queued bytes, in a tooltip or selected-edge details panel.
- Clear the visualization when execution completes, fails, is killed, or the websocket connection closes.
The visualization should avoid overloading existing operator-state colors. Operator status is already used to show execution state, so flow-control pressure should primarily be shown on workflow edges.
## Related Issues
- #4321: Support flow control/backpressure visualization in backend.
- #4322: Support flow control/backpressure visualization in frontend.
## Discussions are welcomed
1. Should the flow-control status be exposed as a new event in the existing
workflow websocket mechanism?
2. What exact fields should be included in the flow-
control status event?
3. Should the frontend always show pressure labels, or should labels only
appear when the runtime/debug overlay is enabled?
4. What update frequency is appropriate to balance responsiveness and
overhead?
GitHub link: https://github.com/apache/texera/discussions/6401
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization
@yrenat Thanks for sharing these thoughts. Some high-level questions: should we finish the other two issues first? How are new ideas related to those two issues?
GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643299
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]
Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat edited a discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization
## Motivation
Texera allows users to design and execute complex workflows, but during execution it is still difficult to understand where data flow is being constrained. When a workflow becomes slow or appears stuck, users may want to know:
- Which part of the workflow is currently under pressure?
- Is an operator slow because it is processing data, waiting for upstream input, or blocked by downstream backpressure?
- Which workflow edge is accumulating pressure?
- Is the bottleneck caused by data flow between operators rather than by the operator state alone?
Texera already exposes multiple operator-level runtime information, such as execution state and operator statistics. However, flow-control/backpressure information is currently kept mostly inside the execution engine and is not visible in the workflow graph to the users.
This proposal focuses on exposing workflow-level flow-control status so users can better understand runtime bottlenecks without changing the underlying execution behavior.
## Proposed Solution
This proposal focuses on exposing workflow-level flow-control status by reusing Amber’s existing credit-based flow-control state.
Amber already tracks per-channel flow-control information, such as available credit, queued credit, and whether a channel is overloaded. The backend should periodically summarize this channel-level state into workflow-level link pressure information and publish it through the existing workflow websocket event mechanism.
The first version is designed to expose a compact runtime event containing, for each affected logical workflow link:
- Source operator id.
- Target operator id.
- Target input port id.
- Whether the link is currently overloaded.
- Current/max credit usage, or utilization percentage.
- Optional aggregated queued/inflight byte counts.
Because physical execution may use multiple worker-to-worker channels for one logical workflow edge, the backend should aggregate channel-level signals into one logical-link status before sending it to the frontend.
The frontend can then render this as a runtime overlay on the workflow graph. Edges under pressure can be highlighted by color, with compact labels or tooltips showing utilization details.
## Backend Design
The backend will reuse the existing flow-control implementation instead of introducing a separate monitoring system.
The expected backend work includes:
1. Collect flow-control snapshots from existing runtime state.
Amber already maintains per-channel flow-control state internally. The implementation should expose a read-only snapshot of relevant values, such as:
- available credit
- maximum credit
- queued credit
- overloaded/backpressured status
2. Aggregate physical channel state into logical workflow-link state.
Flow control is tracked at the worker-channel level, while the frontend visualizes logical workflow links. Therefore, the backend should group channel-level information by logical workflow connection, such as:
```text
worker channel metrics
-> source logical operator + target logical operator + target input port
-> logical link pressure status
3. Publish a compact websocket event.
A possible event shape is:
```text
FlowControlStatusUpdateEvent {
links: {
sourceOperatorId: string;
targetOperatorId: string;
targetPortId: string;
overloaded: boolean;
queuedCreditBytes?: number;
availableCreditBytes?: number;
maxCreditBytes?: number;
utilization?: number;
};
}
## Frontend Design
The frontend will use the new flow-control status event and render it as a workflow graph overlay.
The first version may have a focus on edge-level visualization:
- Use edge color to indicate pressure severity.
- Show compact edge labels only when useful, such as when pressure is non-zero or when a runtime/debug overlay is enabled.
- Show detailed values, such as utilization percentage or queued bytes, in a tooltip or selected-edge details panel.
- Clear the visualization when execution completes, fails, is killed, or the websocket connection closes.
The visualization should avoid overloading existing operator-state colors. Operator status is already used to show execution state, so flow-control pressure should primarily be shown on workflow edges.
## Related Issues
- #4321: Support flow control/backpressure visualization in backend.
- #4322: Support flow control/backpressure visualization in frontend.
## Discussions are welcomed
1. Should the flow-control status be exposed as a new event in the existing
workflow websocket mechanism?
2. What exact fields should be included in the flow-
control status event?
3. Should the frontend always show pressure labels, or should labels only
appear when the runtime/debug overlay is enabled?
4. What update frequency is appropriate to balance responsiveness and
overhead?
5. Any other comments/suggestions/thoughts
GitHub link: https://github.com/apache/texera/discussions/6401
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]
Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat edited a comment on the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization
Yes of course. Indeed that's what this discussion is for. The two existing issues already define the high-level goal and divide the work into backend and frontend parts, and my discussion is intended to further clarify the implementation-level design, including control body and impacted code blocks. I want to follow the common practice of "issue->discussion->PR" and I also want to clarify and verify my implementation design before I proceed.
GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643374
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]
Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat added a comment to the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization
Yes of course. Indeed that's what this discussion is for. The two existing issues already define the high-level goal and divide the work into backend and frontend parts, and my discussion is intended to further clarify the implementation-level design, including control body and impacted code blocks. I want to follow the common practice of "issue->discussion->PR" and also I want to clarify and verify my design before I proceed. implementation.
GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643374
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org