You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by "yrenat (via GitHub)" <gi...@apache.org> on 2026/07/14 02:07:46 UTC

[D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]

GitHub user yrenat created a discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization

## Motivation

  Texera allows users to design and execute complex workflows, but during execution it is still difficult to understand where data flow is being constrained. When a workflow becomes slow or appears stuck, users may want to know:
  - Which part of the workflow is currently under pressure?
  - Is an operator slow because it is processing data, waiting for upstream input, or blocked by downstream backpressure?
  - Which workflow edge is accumulating pressure?
  - Is the bottleneck caused by data flow between operators rather than by the operator state alone?

  Texera already exposes multiple operator-level runtime information, such as execution state and operator statistics. However, flow-control/backpressure information is currently kept mostly inside the execution engine and is not visible in the workflow graph to the users.

  This proposal focuses on exposing workflow-level flow-control status so users can better understand runtime bottlenecks without changing the underlying execution behavior.

  ## Proposed Solution

  This proposal focuses on exposing workflow-level flow-control status by reusing Amber’s existing credit-based flow-control state.

  Amber already tracks per-channel flow-control information, such as available credit, queued credit, and whether a channel is overloaded. The backend should periodically summarize this channel-level state into workflow-level link pressure information and publish it through the existing workflow websocket event mechanism.

  The first version is designed to expose a compact runtime event containing, for each affected logical workflow link:

  - Source operator id.
  - Target operator id.
  - Target input port id.
  - Whether the link is currently overloaded.
  - Current/max credit usage, or utilization percentage.
  - Optional aggregated queued/inflight byte counts.

  Because physical execution may use multiple worker-to-worker channels for one logical workflow edge, the backend should aggregate channel-level signals into one logical-link status before sending it to the frontend.

  The frontend can then render this as a runtime overlay on the workflow graph. Edges under pressure can be highlighted by color, with compact labels or tooltips showing utilization details. 

  ## Backend Design

  The backend will reuse the existing flow-control implementation instead of introducing a separate monitoring system.

  The expected backend work includes:

  1. Collect flow-control snapshots from existing runtime state.

     Amber already maintains per-channel flow-control state internally. The implementation should expose a read-only snapshot of relevant values, such as:

     - available credit
     - maximum credit
     - queued credit
     - overloaded/backpressured status

  2. Aggregate physical channel state into logical workflow-link state.

     Flow control is tracked at the worker-channel level, while the frontend visualizes logical workflow links. Therefore, the backend should group channel-level information by logical workflow connection, such as:

     ```text
     worker channel metrics
       -> source logical operator + target logical operator + target input port
       -> logical link pressure status

  3. Publish a compact websocket event.

     A possible event shape is:

     ```text
     FlowControlStatusUpdateEvent {
       links: {
         sourceOperatorId: string;
         targetOperatorId: string;
         targetPortId: string;
         overloaded: boolean;
         queuedCreditBytes?: number;
         availableCreditBytes?: number;
         maxCreditBytes?: number;
         utilization?: number;
       };
     }

  ## Frontend Design

  The frontend will use the new flow-control status event and render it as a workflow graph overlay.

  The first version may have a focus on edge-level visualization:

  - Use edge color to indicate pressure severity.
  - Show compact edge labels only when useful, such as when pressure is non-zero or when a runtime/debug overlay is enabled.

  - Show detailed values, such as utilization percentage or queued bytes, in a tooltip or selected-edge details panel.

  - Clear the visualization when execution completes, fails, is killed, or the websocket connection closes.

  The visualization should avoid overloading existing operator-state colors. Operator status is already used to show execution state, so flow-control pressure should primarily be shown on workflow edges.

  ## Related Issues

  - #4321: Support flow control/backpressure visualization in backend.
  - #4322: Support flow control/backpressure visualization in frontend.

  ## Discussions are welcomed

  1. Should the flow-control status be exposed as a new event in the existing
     workflow websocket mechanism?

  2. What exact fields should be included in the flow-
     control status event?

  3. Should the frontend always show pressure labels, or should labels only
     appear when the runtime/debug overlay is enabled?

  4. What update frequency is appropriate to balance responsiveness and
     overhead?


GitHub link: https://github.com/apache/texera/discussions/6401

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization

@yrenat Thanks for sharing these thoughts.  Some high-level questions: should we finish the other two issues first?  How are new ideas related to those two issues?

GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643299

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]

Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat edited a discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization

## Motivation

  Texera allows users to design and execute complex workflows, but during execution it is still difficult to understand where data flow is being constrained. When a workflow becomes slow or appears stuck, users may want to know:
  - Which part of the workflow is currently under pressure?
  - Is an operator slow because it is processing data, waiting for upstream input, or blocked by downstream backpressure?
  - Which workflow edge is accumulating pressure?
  - Is the bottleneck caused by data flow between operators rather than by the operator state alone?

  Texera already exposes multiple operator-level runtime information, such as execution state and operator statistics. However, flow-control/backpressure information is currently kept mostly inside the execution engine and is not visible in the workflow graph to the users.

  This proposal focuses on exposing workflow-level flow-control status so users can better understand runtime bottlenecks without changing the underlying execution behavior.

  ## Proposed Solution

  This proposal focuses on exposing workflow-level flow-control status by reusing Amber’s existing credit-based flow-control state.

  Amber already tracks per-channel flow-control information, such as available credit, queued credit, and whether a channel is overloaded. The backend should periodically summarize this channel-level state into workflow-level link pressure information and publish it through the existing workflow websocket event mechanism.

  The first version is designed to expose a compact runtime event containing, for each affected logical workflow link:

  - Source operator id.
  - Target operator id.
  - Target input port id.
  - Whether the link is currently overloaded.
  - Current/max credit usage, or utilization percentage.
  - Optional aggregated queued/inflight byte counts.

  Because physical execution may use multiple worker-to-worker channels for one logical workflow edge, the backend should aggregate channel-level signals into one logical-link status before sending it to the frontend.

  The frontend can then render this as a runtime overlay on the workflow graph. Edges under pressure can be highlighted by color, with compact labels or tooltips showing utilization details. 

  ## Backend Design

  The backend will reuse the existing flow-control implementation instead of introducing a separate monitoring system.

  The expected backend work includes:

  1. Collect flow-control snapshots from existing runtime state.

     Amber already maintains per-channel flow-control state internally. The implementation should expose a read-only snapshot of relevant values, such as:

     - available credit
     - maximum credit
     - queued credit
     - overloaded/backpressured status

  2. Aggregate physical channel state into logical workflow-link state.

     Flow control is tracked at the worker-channel level, while the frontend visualizes logical workflow links. Therefore, the backend should group channel-level information by logical workflow connection, such as:

     ```text
     worker channel metrics
       -> source logical operator + target logical operator + target input port
       -> logical link pressure status

  3. Publish a compact websocket event.

     A possible event shape is:

     ```text
     FlowControlStatusUpdateEvent {
       links: {
         sourceOperatorId: string;
         targetOperatorId: string;
         targetPortId: string;
         overloaded: boolean;
         queuedCreditBytes?: number;
         availableCreditBytes?: number;
         maxCreditBytes?: number;
         utilization?: number;
       };
     }

  ## Frontend Design

  The frontend will use the new flow-control status event and render it as a workflow graph overlay.

  The first version may have a focus on edge-level visualization:

  - Use edge color to indicate pressure severity.
  - Show compact edge labels only when useful, such as when pressure is non-zero or when a runtime/debug overlay is enabled.

  - Show detailed values, such as utilization percentage or queued bytes, in a tooltip or selected-edge details panel.

  - Clear the visualization when execution completes, fails, is killed, or the websocket connection closes.

  The visualization should avoid overloading existing operator-state colors. Operator status is already used to show execution state, so flow-control pressure should primarily be shown on workflow edges.

  ## Related Issues

  - #4321: Support flow control/backpressure visualization in backend.
  - #4322: Support flow control/backpressure visualization in frontend.

  ## Discussions are welcomed

  1. Should the flow-control status be exposed as a new event in the existing
     workflow websocket mechanism?

  2. What exact fields should be included in the flow-
     control status event?

  3. Should the frontend always show pressure labels, or should labels only
     appear when the runtime/debug overlay is enabled?

  4. What update frequency is appropriate to balance responsiveness and
     overhead?

  5. Any other comments/suggestions/thoughts


GitHub link: https://github.com/apache/texera/discussions/6401

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]

Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat edited a comment on the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization

Yes of course. Indeed that's what this discussion is for. The two existing issues already define the high-level goal and divide the work into backend and frontend parts, and my discussion is intended to further clarify the implementation-level design, including control body and impacted code blocks. I want to follow the common practice of "issue->discussion->PR" and I also want to clarify and verify my implementation design before I proceed.

GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643374

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization [texera]

Posted by "yrenat (via GitHub)" <gi...@apache.org>.
GitHub user yrenat added a comment to the discussion: Improve Workflow Runtime Observability with Execution Status and Flow Control Visualization

Yes of course. Indeed that's what this discussion is for. The two existing issues already define the high-level goal and divide the work into backend and frontend parts, and my discussion is intended to further clarify the implementation-level design, including control body and impacted code blocks. I want to follow the common practice of "issue->discussion->PR" and also I want to clarify and verify my design before I proceed. implementation.

GitHub link: https://github.com/apache/texera/discussions/6401#discussioncomment-17643374

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org