You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "PG1204 (via GitHub)" <gi...@apache.org> on 2026/06/18 18:56:01 UTC
[I] Capture per-operator performance metrics in WorkflowStatusService [texera]
PG1204 opened a new issue, #5773:
URL: https://github.com/apache/texera/issues/5773
### Task Summary
### Task Summary
Type the per-operator statistics fields the backend already sends but the frontend drops, and expose a derived performance model from `WorkflowStatusService`, as the data foundation for the workflow heat-map overlay. No UI change.
### Context
The execution engine emits 11 statistics per operator over the websocket, but the frontend `OperatorStatistics` type in `frontend/src/app/workspace/types/execute-workflow.interface.ts` declares only 6:
```typescript
export interface OperatorStatistics
extends Readonly<{
operatorState: OperatorState;
aggregatedInputRowCount: number;
aggregatedInputSize?: number; // bytes
inputPortMetrics: Record<string, number>;
aggregatedOutputRowCount: number;
aggregatedOutputSize?: number; // bytes
outputPortMetrics: Record<string, number>;
numWorkers?: number;
aggregatedDataProcessingTime?: number; // nanoseconds
aggregatedControlProcessingTime?: number; // nanoseconds
aggregatedIdleTime?: number; // nanoseconds
}> {}
```
The five timing/byte-size fields: aggregatedInputSize, aggregatedOutputSize, aggregatedDataProcessingTime, aggregatedControlProcessingTime, aggregatedIdleTime, all arrive in the JSON payload but are untyped and unused, so they are silently discarded.
WorkflowStatusService is currently a thin pass-through over the OperatorStatisticsUpdateEvent stream and has no derived performance model and no spec file. This sub-task makes it the single source of truth for per-operator performance data, ahead of the overlay built in the following sub-tasks.
Before: websocket -> OperatorStatistics (6 fields; timing & sizes dropped)
After: websocket -> OperatorStatistics (11 fields) -> derived, normalized per-operator performance metrics
### Proposed Change
In frontend/src/app/workspace/types/execute-workflow.interface.ts:
Add the five missing fields to OperatorStatistics, all optional, so the partial objects built in resetStatus / clearStatus still type-check.
New frontend/src/app/workspace/service/workflow-status/performance-metrics.ts
(pure, framework-free):
- A derived OperatorPerformanceMetrics model and a toPerformanceMetrics mapper that defensively defaults every field to 0.
- A HeatmapView enum (Runtime, Throughput, I/O imbalance) and a rawMetricForView selector.
- A normalizeScores helper returning [0, 1] scores, with skew handling and defined empty / single-operator / all-equal behavior.
In frontend/src/app/workspace/service/workflow-status/workflow-status.service.ts:
- Keep the existing pass-through API unchanged.
- Add a derived performance-metrics stream (BehaviorSubject-backed) plus a synchronous snapshot getter.
- Add a public setExternalStatus(...) ingestion method so a later sub-task can feed in restored historical statistics without touching private state.
### Required Test
- New performance-metrics.spec.ts: full mapping, missing-field defaults (no NaN), all-zero, I/O-imbalance ratio with inputRows === 0, unicode operator id, and normalizeScores edge cases (empty, single, all-equal, two-value, heavy-tail, clamping).
- New workflow-status.service.spec.ts: derived stream emits on OperatorStatisticsUpdateEvent, non-matching events are ignored, the snapshot getter is synchronous, and resetStatus / clearStatus / setExternalStatus behave as expected.
### Related
Umbrella issue #5772
RFC discussion #5216
### Task Type
- [x] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [x] Performance
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Capture per-operator performance metrics in WorkflowStatusService [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5773:
URL: https://github.com/apache/texera/issues/5773#issuecomment-4745190167
<!-- texera:template-compliance -->
👋 Thanks for opening this issue, @PG1204!
It looks like the issue description doesn't quite follow our template yet:
- The **Task Summary** section is empty; please fill it in.
Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.
You can find the template prompts by editing the description, or see [CONTRIBUTING.md](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) for the full contribution flow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Capture per-operator performance metrics in WorkflowStatusService [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang closed issue #5773: Capture per-operator performance metrics in WorkflowStatusService
URL: https://github.com/apache/texera/issues/5773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Capture per-operator performance metrics in WorkflowStatusService [texera]
Posted by "PG1204 (via GitHub)" <gi...@apache.org>.
PG1204 commented on issue #5773:
URL: https://github.com/apache/texera/issues/5773#issuecomment-4745197030
/take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org