You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by gi...@apache.org on 2026/08/26 14:23:14 UTC
(seatunnel) branch gh-readonly-queue/dev/pr-11957-cc548bb3f8c3181c820c316d2fc6996e46a08dd9 created (now 7c73866c75)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/dev/pr-11957-cc548bb3f8c3181c820c316d2fc6996e46a08dd9
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
at 7c73866c75 [Feature][Zeta] Add selectable live metrics chart on Job Detail (#11957)
This branch includes the following new commits:
new 7c73866c75 [Feature][Zeta] Add selectable live metrics chart on Job Detail (#11957)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(seatunnel) 01/01: [Feature][Zeta] Add selectable live metrics chart on Job Detail (#11957)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-11957-cc548bb3f8c3181c820c316d2fc6996e46a08dd9
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit 7c73866c752cdbe75c7375ca632c98f713aeccb9
Author: luxiaolong <li...@126.com>
AuthorDate: Wed Aug 26 14:22:48 2026 +0000
[Feature][Zeta] Add selectable live metrics chart on Job Detail (#11957)
Co-authored-by: det101 <lu...@chinatelecom.cn>
Co-authored-by: Cursor <cu...@cursor.com>
---
docs/en/engines/zeta/live-metrics-chart.md | 74 ++++
docs/en/engines/zeta/web-ui.md | 2 +
docs/sidebars.js | 1 +
docs/zh/engines/zeta/live-metrics-chart.md | 74 ++++
docs/zh/engines/zeta/web-ui.md | 2 +
.../seatunnel-engine-ui/package-lock.json | 32 ++
seatunnel-engine/seatunnel-engine-ui/package.json | 3 +-
.../components/directed-acyclic-graph/index.scss | 38 +-
.../components/directed-acyclic-graph/index.tsx | 8 +-
.../src/components/live-metrics-chart/board.tsx | 90 +++++
.../src/components/live-metrics-chart/group.ts | 58 +++
.../src/components/live-metrics-chart/index.tsx | 251 +++++++++++++
.../src/components/live-metrics-chart/types.ts | 42 +++
.../src/locales/en_US/detail.ts | 18 +-
.../src/locales/zh_CN/detail.ts | 18 +-
.../src/service/realtime-metrics/index.ts | 43 ++-
.../src/store/live-metrics-pin/index.ts | 90 +++++
.../src/tests/detail-live-metrics.spec.ts | 141 ++++++++
.../seatunnel-engine-ui/src/tests/detail.spec.ts | 42 ++-
.../src/tests/live-metrics-pin.spec.ts | 68 ++++
.../src/views/jobs/detail-live-metrics.ts | 229 ++++++++++++
.../seatunnel-engine-ui/src/views/jobs/detail.tsx | 393 ++++++++++++---------
22 files changed, 1512 insertions(+), 205 deletions(-)
diff --git a/docs/en/engines/zeta/live-metrics-chart.md b/docs/en/engines/zeta/live-metrics-chart.md
new file mode 100644
index 0000000000..f6ac474f45
--- /dev/null
+++ b/docs/en/engines/zeta/live-metrics-chart.md
@@ -0,0 +1,74 @@
+# Live Metrics Chart
+
+On the Job Detail Overview page, operators can pin numeric realtime metrics and watch them as a live line chart without keeping the vertex or edge drawer open.
+
+This page describes the shipped Job Detail behavior. It reuses the existing realtime metrics endpoints and Overview poll. It does not add a metrics pipeline, a Metrics tab, or long-term history.
+
+## Where it appears
+
+```text
+Overview
+├─ DAG
+├─ Pinned live metrics chart panel
+└─ Existing summary metrics table
+```
+
+- Click a vertex or edge on the DAG to open the detail drawer.
+- The drawer keeps the key-field summary above the divider.
+- Below the divider, the previous series table is a live line chart with Pin / Unpin controls.
+- Pinned series stay on the Overview panel after the drawer closes.
+- Charts are split by unit so mixed metrics stay readable: **ratio** (0–100%), **duration** (ms/record), and **records**. Same-unit series overlay on one chart; Overview places different units in one compact row (up to three columns). Legend labels use short operator ids such as `Source[0]`.
+
+## Pin behavior
+
+Pins are scoped to the current Job Detail visit. They are not written to `localStorage`.
+
+| Event | Behavior |
+|---|---|
+| Pin from the drawer | Add the series to the Overview pinned panel |
+| Close the drawer | Keep pinned series |
+| Switch Overview / Exception / Configuration / Log on the same job page | Keep pinned series |
+| Leave Job Detail | Clear pinned series |
+| Job reaches a terminal state | Clear pinned series |
+| Exceed the pin limit | Reject the new pin and show a short message |
+
+Default pin limit: **6** series.
+
+## Refresh, window, and cost
+
+Pinned series consume the same Overview realtime response already used by the DAG:
+
+- `GET /metrics/realtime/jobs/{jobId}/vertices?windowMs=`
+- `GET /metrics/realtime/jobs/{jobId}/edges?windowMs=`
+
+While Overview is open and the job is running, the page polls both endpoints every 2 seconds. The default window is 3 minutes, capped at 10 minutes. Pinning does not create extra REST traffic: one client still issues the same two requests per poll interval regardless of pin count.
+
+No chart data is written to disk.
+
+## Shared fetch and chart contract
+
+Follow-up observability views such as the runtime graph ([#11351](https://github.com/apache/seatunnel/issues/11351)) and backpressure diagnosis ([#11352](https://github.com/apache/seatunnel/issues/11352)) should reuse these building blocks instead of adding another poll or chart library.
+
+**Reuse**
+
+| Piece | Location | Contract |
+|---|---|---|
+| Job-level fetch | `fetchJobRealtimeMetrics` in `seatunnel-engine-ui` | Loads vertices and edges for one `windowMs`. Callers own the poll loop. The helper does not start a timer. |
+| Live chart | `LiveLineChart` / `LiveMetricsBoard` | Request-agnostic. Pages inject already-fetched series: `{ id, name, unit?, points: [{ ts, value }] }`, plus `windowMs` and optional `emptyText`. The component does not fetch. |
+
+Same-unit series overlay on one chart. Mixed units (ratio, duration, records) are split so scales stay readable.
+
+**Do not reuse** from this feature: the Job Detail pin store, the six-series pin limit, or the Overview pin-panel layout. Those are session UX for this page only.
+
+ECharts is a rendering dependency of `seatunnel-engine-ui` only. Do not add a second chart library.
+
+## Limitations
+
+The first version does not include:
+
+- long-term historical metrics storage or export
+- alerting or threshold notifications
+- custom or derived metric expressions
+- a separate Flink-style Metrics tab
+
+For connector and engine metric semantics, see [Realtime Observability](realtime-observability.md). For the Job Detail screens, see [Web UI](web-ui.md).
diff --git a/docs/en/engines/zeta/web-ui.md b/docs/en/engines/zeta/web-ui.md
index 944d6fa299..6b49006e57 100644
--- a/docs/en/engines/zeta/web-ui.md
+++ b/docs/en/engines/zeta/web-ui.md
@@ -74,6 +74,7 @@ On the Job Detail page, the DAG view can display realtime metrics for the recent
- **Vertex busyness**: busy and idle ratios for Source, Transform, and Sink vertices.
- **Edge downstream wait ratio**: when the job inserts queues at async boundaries or before Sink IO, edges are colored and thickened by downstream wait ratio and queue fill ratio.
- **Interaction**: click a vertex or edge to open the detail drawer and view realtime curves and key fields.
+- **Pinned live chart**: pin one or more numeric metrics from the drawer so live charts remain visible on Overview after the drawer closes. Series are split by unit (ratio, duration, records) so mixed scales stay readable; same-unit metrics overlay for comparison. See [Live Metrics Chart](live-metrics-chart.md) for pin lifecycle, the 6-series limit, and shared polling cost.
This capability requires the job to enable `env.engine.observability` or configure an option that auto-enables it, such as `async_boundaries` or `split_sink_io`. See [Realtime Observability](realtime-observability.md) for configuration and metric semantics.
@@ -106,5 +107,6 @@ The "Master" section displays system monitoring information for master nodes. Us
- [REST API and Web UI](./rest-api-and-web-ui.md)
- [REST API V2](./rest-api-v2.md)
- [Runtime Execution Graph](./runtime-execution-graph.md)
+- [Live Metrics Chart](./live-metrics-chart.md)
- [Job Lifecycle API](./rest-api-job-lifecycle.md)
- [Security](./security.md)
diff --git a/docs/sidebars.js b/docs/sidebars.js
index cd737212fb..a986edc575 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -282,6 +282,7 @@ const sidebars = {
"engines/zeta/log-analysis-with-ai",
"engines/zeta/telemetry",
"engines/zeta/busyness-and-backpressure",
+ "engines/zeta/live-metrics-chart",
"engines/zeta/slot-allocation-strategy",
"engines/zeta/benchmark",
"engines/zeta/tuning-guide"
diff --git a/docs/zh/engines/zeta/live-metrics-chart.md b/docs/zh/engines/zeta/live-metrics-chart.md
new file mode 100644
index 0000000000..73b3867c7a
--- /dev/null
+++ b/docs/zh/engines/zeta/live-metrics-chart.md
@@ -0,0 +1,74 @@
+# 实时指标图(Live Metrics Chart)
+
+在 Job Detail 的 Overview 页,运维可以从节点或边抽屉中 Pin 数值型实时指标,关闭抽屉后仍能在 Overview 上看实时折线,并对比多个 vertex/edge。
+
+本页描述已落地的 Job Detail 行为。它复用现有 realtime metrics 接口和 Overview 轮询,不新增指标管线、Metrics 页签或长期历史存储。
+
+## 出现位置
+
+```text
+Overview
+├─ DAG
+├─ Pin 实时指标图面板
+└─ 现有汇总指标表
+```
+
+- 点击 DAG 上的节点或边打开详情抽屉。
+- 分隔线上方的关键字段摘要保持不变。
+- 分隔线下方由原来的时序表改为实时折线图,并提供 Pin / Unpin。
+- 关闭抽屉后,已 Pin 的 series 仍留在 Overview 面板。
+- 图表按量纲拆开,避免混轴压扁:**占比**(0–100%)、**耗时**(毫秒/条)、**条数**。同量纲叠在一张图上;Overview 上不同量纲并排成一行(最多三列)。图例使用短名,例如 `Source[0]`。
+
+## Pin 行为
+
+Pin 只作用于当前这次 Job Detail 访问,不会写入 `localStorage`。
+
+| 事件 | 行为 |
+|---|---|
+| 从抽屉 Pin | 加入 Overview pin 面板 |
+| 关闭抽屉 | 保留已 Pin series |
+| 在同一作业页切换 Overview / Exception / Configuration / Log | 保留已 Pin series |
+| 离开 Job Detail | 清空 |
+| 作业进入终态 | 清空 |
+| 超过 Pin 上限 | 拒绝新增并给出简短提示 |
+
+默认 Pin 上限:**6** 条 series。
+
+## 刷新、窗口与成本
+
+已 Pin series 只消费 Overview 已经在给 DAG 使用的同一份 realtime 响应:
+
+- `GET /metrics/realtime/jobs/{jobId}/vertices?windowMs=`
+- `GET /metrics/realtime/jobs/{jobId}/edges?windowMs=`
+
+Overview 打开且作业运行时,页面每 2 秒轮询上述两个接口。默认窗口 3 分钟,最大 10 分钟。Pin 不会增加额外 REST 流量:单个客户端无论 Pin 多少条,每个轮询周期仍是这两次请求。
+
+图表数据不落盘。
+
+## 共享拉取与图表合约
+
+后续可观测页面(例如运行时执行图 [#11351](https://github.com/apache/seatunnel/issues/11351)、反压诊断 [#11352](https://github.com/apache/seatunnel/issues/11352))应复用下列积木,不要再开一条轮询或第二套图库。
+
+**可复用**
+
+| 积木 | 位置 | 合约 |
+|---|---|---|
+| Job 级拉取 | `seatunnel-engine-ui` 中的 `fetchJobRealtimeMetrics` | 按一个 `windowMs` 拉取 vertices 与 edges。调用方自己管轮询循环,helper 不启动定时器。 |
+| 实时折线 | `LiveLineChart` / `LiveMetricsBoard` | 请求无关。页面注入已取好的 series:`{ id, name, unit?, points: [{ ts, value }] }`,以及 `windowMs` 和可选 `emptyText`。组件不发 HTTP。 |
+
+同量纲叠线;不同量纲(占比、耗时、条数)拆图,避免混轴。
+
+**不要复用**:Job Detail 的 Pin store、6 条上限、Overview pin 面板布局。这些只服务本页的会话交互。
+
+ECharts 只作为 `seatunnel-engine-ui` 的渲染依赖。不要再引入第二套图库。
+
+## 限制
+
+第一版不包含:
+
+- 长期历史指标存储或导出
+- 告警或阈值通知
+- 自定义或派生指标表达式
+- 独立的 Flink 风格 Metrics 页签
+
+指标语义见 [实时可观测性](realtime-observability.md)。Job Detail 页面说明见 [Web UI](web-ui.md)。
diff --git a/docs/zh/engines/zeta/web-ui.md b/docs/zh/engines/zeta/web-ui.md
index 55598acd0f..0db8407644 100644
--- a/docs/zh/engines/zeta/web-ui.md
+++ b/docs/zh/engines/zeta/web-ui.md
@@ -74,6 +74,7 @@ Web UI 不负责提交作业,也不提供 cancel、stop、savepoint、restore
- **节点忙碌度**:Source/Transform/Sink 的忙闲比例(例如 Source Read/Idle,Transform Busy,Sink Busy)。
- **边的下游等待占比**:当作业在某些位置插入了队列(例如 async boundary 队列、sink 前拆分 IO 队列)时,边会根据下游等待占比与队列填充率进行着色/加粗。
- **交互**:点击节点或边可在右侧抽屉查看该对象的实时曲线与关键字段。
+- **Pin 实时图**:可从抽屉 pin 一条或多条数值指标,关闭抽屉后 Overview 上仍保留实时折线。图表按量纲拆分(占比 / 耗时 / 条数),同量纲才叠线对比。Pin 生命周期、6 条上限与共享轮询成本见:[实时指标图](live-metrics-chart.md)。
> 该能力需要作业侧开启 `env.engine.observability`(或满足默认开启条件),并按需配置 `async_boundaries`、`split_sink_io` 等。
> 详细配置与指标说明请参考:[实时可观测性](realtime-observability.md)。
@@ -107,5 +108,6 @@ Web UI 不负责提交作业,也不提供 cancel、stop、savepoint、restore
- [REST API 与 Web UI](./rest-api-and-web-ui.md)
- [REST API V2](./rest-api-v2.md)
- [运行时执行图](./runtime-execution-graph.md)
+- [实时指标图](./live-metrics-chart.md)
- [作业生命周期 API](./rest-api-job-lifecycle.md)
- [安全](./security.md)
diff --git a/seatunnel-engine/seatunnel-engine-ui/package-lock.json b/seatunnel-engine/seatunnel-engine-ui/package-lock.json
index 3118ae48ec..6af40c419c 100644
--- a/seatunnel-engine/seatunnel-engine-ui/package-lock.json
+++ b/seatunnel-engine/seatunnel-engine-ui/package-lock.json
@@ -16,6 +16,7 @@
"axios": "^1.7.7",
"date-fns": "^3.6.0",
"date-fns-tz": "^3.1.3",
+ "echarts": "^5.6.0",
"naive-ui": "^2.39.0",
"nprogress": "^0.2.0",
"pinia": "^2.1.7",
@@ -4072,6 +4073,22 @@
"safer-buffer": "^2.1.0"
}
},
+ "node_modules/echarts": {
+ "version": "5.6.0",
+ "resolved": "https://registry.npmmirror.com/echarts/-/echarts-5.6.0.tgz",
+ "integrity": "sha512-oTbVTsXfKuEhxftHqL5xprgLoc0k7uScAwtryCgWF6hPYFLRwOUHiFmHGCBKP5NPFNkDVopOieyUqYGH8Fa3kA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "2.3.0",
+ "zrender": "5.6.1"
+ }
+ },
+ "node_modules/echarts/node_modules/tslib": {
+ "version": "2.3.0",
+ "resolved": "https://registry.npmmirror.com/tslib/-/tslib-2.3.0.tgz",
+ "integrity": "sha512-N82ooyxVNm6h1riLCoyS9e3fuJ3AMG2zIZs2Gd1ATcSFjSA23Q0fzjjZeh0jbJvWVDZ0cJT8yaNNaaXHzueNjg==",
+ "license": "0BSD"
+ },
"node_modules/editorconfig": {
"version": "1.0.4",
"resolved": "https://registry.npmmirror.com/editorconfig/-/editorconfig-1.0.4.tgz",
@@ -10308,6 +10325,21 @@
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
+ },
+ "node_modules/zrender": {
+ "version": "5.6.1",
+ "resolved": "https://registry.npmmirror.com/zrender/-/zrender-5.6.1.tgz",
+ "integrity": "sha512-OFXkDJKcrlx5su2XbzJvj/34Q3m6PvyCZkVPHGYpcCJ52ek4U/ymZyfuV1nKE23AyBJ51E/6Yr0mhZ7xGTO4ag==",
+ "license": "BSD-3-Clause",
+ "dependencies": {
+ "tslib": "2.3.0"
+ }
+ },
+ "node_modules/zrender/node_modules/tslib": {
+ "version": "2.3.0",
+ "resolved": "https://registry.npmmirror.com/tslib/-/tslib-2.3.0.tgz",
+ "integrity": "sha512-N82ooyxVNm6h1riLCoyS9e3fuJ3AMG2zIZs2Gd1ATcSFjSA23Q0fzjjZeh0jbJvWVDZ0cJT8yaNNaaXHzueNjg==",
+ "license": "0BSD"
}
}
}
diff --git a/seatunnel-engine/seatunnel-engine-ui/package.json b/seatunnel-engine/seatunnel-engine-ui/package.json
index 6d29bb53ce..49128db201 100644
--- a/seatunnel-engine/seatunnel-engine-ui/package.json
+++ b/seatunnel-engine/seatunnel-engine-ui/package.json
@@ -24,6 +24,7 @@
"axios": "^1.7.7",
"date-fns": "^3.6.0",
"date-fns-tz": "^3.1.3",
+ "echarts": "^5.6.0",
"naive-ui": "^2.39.0",
"nprogress": "^0.2.0",
"pinia": "^2.1.7",
@@ -61,4 +62,4 @@
"vitest": "^1.5.3",
"vue-tsc": "^2.0.21"
}
-}
\ No newline at end of file
+}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.scss b/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.scss
index a5d14099b7..a15870202c 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.scss
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.scss
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
+
.node {
display: flex;
justify-content: space-between;
@@ -26,25 +26,27 @@
border: 2px solid var(--busy-color, #c2c8d5);
border-radius: 4px;
box-shadow: 0 2px 5px 1px rgba(0, 0, 0, 0.06);
- padding: 6px 8px;
+ padding: 4px 8px;
transition: box-shadow 0.2s ease;
.label {
flex: 1;
color: #666;
- font-size: 12px;
+ font-size: 11px;
overflow: hidden;
text-overflow: ellipsis;
- display: -webkit-box;
- -webkit-line-clamp: 2;
- -webkit-box-orient: vertical;
+ white-space: nowrap;
}
}
.dag-wrapper {
position: relative;
width: 100%;
- height: 600px;
- margin-bottom: 12px;
+ height: 128px;
+ margin-bottom: 8px;
+ border: 1px solid #f3f4f6;
+ border-radius: 8px;
+ background: #f8fafc;
+ overflow: hidden;
}
.dag-container {
@@ -55,16 +57,16 @@
.dag-toolbar {
position: absolute;
z-index: 10;
- top: 12px;
- right: 12px;
+ top: 6px;
+ right: 8px;
display: flex;
- gap: 8px;
+ gap: 6px;
}
.dag-btn {
- height: 28px;
- min-width: 28px;
- padding: 0 10px;
+ height: 24px;
+ min-width: 24px;
+ padding: 0 8px;
border-radius: 6px;
border: 1px solid #e5e7eb;
background: rgba(255, 255, 255, 0.92);
@@ -86,11 +88,11 @@
border-radius: 2px;
box-shadow: 0 0 0 4px #d4e8fe;
}
-.x6-edge:hover path:nth-child(2){
+.x6-edge:hover path:nth-child(2) {
stroke-width: 2px;
}
-.x6-edge-selected path:nth-child(2){
+.x6-edge-selected path:nth-child(2) {
stroke-width: 2px !important;
}
@@ -101,9 +103,9 @@
}
@keyframes spin {
from {
- transform: rotate(0deg);
+ transform: rotate(0deg);
}
to {
- transform: rotate(360deg);
+ transform: rotate(360deg);
}
}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.tsx b/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.tsx
index 905ce8c3f9..7ac731653f 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.tsx
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/directed-acyclic-graph/index.tsx
@@ -64,7 +64,7 @@ const nodeWidth = 300
register({
shape: 'dag-node',
width: nodeWidth,
- height: 48,
+ height: 36,
component: AlgoNode,
ports: {
groups: {
@@ -244,8 +244,8 @@ export default defineComponent({
if (!force && userHasZoomed) return
try {
graph.zoomToFit({
- padding: 32,
- maxScale: 1.2,
+ padding: { top: 16, right: 80, bottom: 16, left: 16 },
+ maxScale: 1.6,
minScale: 0.2
})
graph.centerContent()
@@ -412,7 +412,7 @@ export default defineComponent({
const matrix = [] as Vertex[][]
const items: Cell.Metadata[] = []
- const offsetY = 140
+ const offsetY = 72
const offsetX = nodeWidth + 200
const processed = [] as Vertex[]
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/board.tsx b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/board.tsx
new file mode 100644
index 0000000000..af4d3ed7fd
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/board.tsx
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { computed, defineComponent, type PropType } from 'vue'
+import LiveLineChart from './index'
+import { groupSeriesByUnit } from './group'
+import type { LiveMetricSeries, MetricUnit } from './types'
+
+export default defineComponent({
+ name: 'LiveMetricsBoard',
+ props: {
+ series: {
+ type: Array as PropType<LiveMetricSeries[]>,
+ default: () => []
+ },
+ windowMs: {
+ type: Number,
+ default: 3 * 60 * 1000
+ },
+ emptyText: {
+ type: String,
+ default: 'No metrics'
+ },
+ height: {
+ type: Number,
+ default: 220
+ },
+ unitTitles: {
+ type: Object as PropType<Partial<Record<MetricUnit, string>>>,
+ default: () => ({})
+ },
+ /** Overview uses a single compact row; the drawer stays stacked. */
+ layout: {
+ type: String as PropType<'stack' | 'row'>,
+ default: 'stack'
+ }
+ },
+ setup(props) {
+ const groups = computed(() => groupSeriesByUnit(props.series || []))
+
+ return () => {
+ if (!groups.value.length) {
+ return <div class="text-sm text-gray-400 py-2 text-center leading-6">{props.emptyText}</div>
+ }
+ const n = groups.value.length
+ const gridClass =
+ props.layout === 'row'
+ ? n >= 3
+ ? 'grid grid-cols-3 gap-2'
+ : n === 2
+ ? 'grid grid-cols-2 gap-2'
+ : 'grid grid-cols-1 gap-2'
+ : 'flex flex-col gap-3'
+ return (
+ <div class={gridClass}>
+ {groups.value.map((group) => (
+ <div key={group.unit} class="min-w-0">
+ <div class="text-xs text-gray-500 mb-1 leading-4">
+ {props.unitTitles[group.unit] || group.unit}
+ </div>
+ <div class="bg-white rounded border border-gray-100 overflow-hidden">
+ <LiveLineChart
+ series={group.series}
+ windowMs={props.windowMs}
+ emptyText={props.emptyText}
+ height={props.height}
+ unit={group.unit}
+ />
+ </div>
+ </div>
+ ))}
+ </div>
+ )
+ }
+ }
+})
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/group.ts b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/group.ts
new file mode 100644
index 0000000000..fe0f704e25
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/group.ts
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import type { LiveMetricSeries, MetricUnit } from './types'
+
+export const METRIC_UNIT_ORDER: MetricUnit[] = ['ratio', 'duration', 'count']
+
+export const inferMetricUnit = (field: string): MetricUnit => {
+ if (field.endsWith('NsPerRecord') || field.toLowerCase().includes('msperrecord')) {
+ return 'duration'
+ }
+ if (field.endsWith('Ratio') || field === 'bpRatio') {
+ return 'ratio'
+ }
+ return 'count'
+}
+
+export const groupSeriesByUnit = (
+ series: LiveMetricSeries[]
+): Array<{ unit: MetricUnit; series: LiveMetricSeries[] }> => {
+ const buckets: Record<MetricUnit, LiveMetricSeries[]> = {
+ ratio: [],
+ duration: [],
+ count: []
+ }
+ series.forEach((item) => {
+ buckets[item.unit || 'count'].push(item)
+ })
+ return METRIC_UNIT_ORDER.filter((unit) => buckets[unit].length).map((unit) => ({
+ unit,
+ series: buckets[unit]
+ }))
+}
+
+export const formatMetricValue = (unit: MetricUnit | undefined, value: number): string => {
+ if (!Number.isFinite(value)) return '-'
+ if (unit === 'ratio') {
+ return `${(value * 100).toFixed(1)}%`
+ }
+ if (unit === 'duration') {
+ return `${value.toFixed(2)} ms`
+ }
+ return Number.isInteger(value) ? String(value) : value.toFixed(1)
+}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/index.tsx b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/index.tsx
new file mode 100644
index 0000000000..65a9db069f
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/index.tsx
@@ -0,0 +1,251 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { defineComponent, onBeforeUnmount, onMounted, ref, watch, type PropType } from 'vue'
+import * as echarts from 'echarts/core'
+import { LineChart } from 'echarts/charts'
+import {
+ GridComponent,
+ LegendComponent,
+ TitleComponent,
+ TooltipComponent
+} from 'echarts/components'
+import { CanvasRenderer } from 'echarts/renderers'
+import type { LiveMetricSeries, MetricUnit } from './types'
+import { formatMetricValue } from './group'
+
+export type { LiveLineChartProps, LiveMetricPoint, LiveMetricSeries, MetricUnit } from './types'
+
+echarts.use([
+ LineChart,
+ GridComponent,
+ LegendComponent,
+ TitleComponent,
+ TooltipComponent,
+ CanvasRenderer
+])
+
+const COLORS = ['#4678B9', '#18A058', '#F0A020', '#D03050', '#2080F0', '#8A2BE2']
+
+export default defineComponent({
+ name: 'LiveLineChart',
+ props: {
+ series: {
+ type: Array as PropType<LiveMetricSeries[]>,
+ default: () => []
+ },
+ windowMs: {
+ type: Number,
+ default: 3 * 60 * 1000
+ },
+ emptyText: {
+ type: String,
+ default: 'No metrics'
+ },
+ height: {
+ type: Number,
+ default: 260
+ },
+ unit: {
+ type: String as PropType<MetricUnit>,
+ default: undefined
+ }
+ },
+ setup(props) {
+ const el = ref<HTMLDivElement>()
+ let chart: echarts.ECharts | undefined
+
+ const render = () => {
+ if (!el.value) return
+ if (!chart) {
+ chart = echarts.init(el.value)
+ }
+ const series = props.series || []
+ if (!series.length) {
+ chart.clear()
+ chart.setOption({
+ title: {
+ text: props.emptyText,
+ left: 'center',
+ top: 'middle',
+ textStyle: { color: '#999', fontSize: 14, fontWeight: 'normal' }
+ }
+ })
+ return
+ }
+
+ const now = Date.now()
+ const from = now - props.windowMs
+ const unit = props.unit || series[0]?.unit
+ const single = series.length === 1
+ const compact = props.height < 160
+
+ const yAxis =
+ unit === 'ratio'
+ ? {
+ type: 'value' as const,
+ min: 0,
+ max: 1,
+ scale: false,
+ name: compact ? '' : '%',
+ nameTextStyle: { color: '#999', padding: [0, 0, 0, 8], fontSize: 10 },
+ axisLabel: {
+ fontSize: compact ? 9 : 10,
+ margin: compact ? 4 : 8,
+ formatter: (v: number) => `${Math.round(v * 100)}`
+ },
+ splitLine: { lineStyle: { type: 'dashed', color: '#eee' } }
+ }
+ : {
+ type: 'value' as const,
+ min: compact ? 0 : undefined,
+ scale: !compact,
+ name: compact ? '' : unit === 'duration' ? 'ms' : '',
+ nameTextStyle: { color: '#999', fontSize: 10 },
+ axisLabel: { fontSize: compact ? 9 : 10, margin: compact ? 4 : 8 },
+ splitLine: { lineStyle: { type: 'dashed', color: '#eee' } }
+ }
+
+ chart.setOption(
+ {
+ title: single
+ ? {
+ show: true,
+ text: series[0].name,
+ left: 0,
+ top: 0,
+ textStyle: { color: '#666', fontSize: compact ? 11 : 12, fontWeight: 'normal' }
+ }
+ : { show: false },
+ color: COLORS,
+ tooltip: {
+ trigger: 'axis',
+ formatter: (params: unknown) => {
+ const items = Array.isArray(params) ? params : [params]
+ const first = items[0] as { axisValue?: number }
+ const time = first?.axisValue ? new Date(first.axisValue).toLocaleTimeString() : ''
+ const lines = items.map((raw) => {
+ const item = raw as {
+ marker?: string
+ seriesName?: string
+ value?: [number, number]
+ }
+ const match = series.find((s) => s.name === item.seriesName)
+ const label = match?.fullName || item.seriesName || ''
+ const value = Array.isArray(item.value) ? item.value[1] : Number.NaN
+ return `${item.marker || ''}${label}: ${formatMetricValue(unit, value)}`
+ })
+ return [time, ...lines].join('<br/>')
+ }
+ },
+ legend: single
+ ? { show: false }
+ : {
+ type: 'scroll',
+ top: 0,
+ itemWidth: compact ? 10 : 14,
+ itemHeight: compact ? 8 : 10,
+ textStyle: { fontSize: compact ? 10 : 11 },
+ data: series.map((s) => s.name)
+ },
+ grid: {
+ left: compact ? 28 : 48,
+ right: compact ? 4 : 24,
+ top: compact ? (single ? 16 : 20) : single ? 28 : 36,
+ bottom: compact ? 0 : 32,
+ containLabel: !compact
+ },
+ xAxis: {
+ type: 'time',
+ min: from,
+ max: now,
+ boundaryGap: false,
+ axisTick: { show: !compact },
+ axisLine: { show: true, lineStyle: { color: compact ? '#e5e7eb' : '#d1d5db' } },
+ axisLabel: compact
+ ? { show: false, margin: 0 }
+ : {
+ formatter: (value: number) =>
+ new Date(value).toLocaleTimeString(undefined, {
+ hour: '2-digit',
+ minute: '2-digit',
+ second: '2-digit'
+ })
+ },
+ splitLine: { show: false }
+ },
+ yAxis,
+ series: series.map((s, i) => {
+ const color = COLORS[i % COLORS.length]
+ return {
+ name: s.name,
+ type: 'line',
+ showSymbol: false,
+ smooth: true,
+ lineStyle: { width: compact ? 1.5 : 2, color },
+ itemStyle: { color },
+ areaStyle: compact
+ ? {
+ color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [
+ { offset: 0, color: `${color}73` },
+ { offset: 1, color: `${color}1A` }
+ ])
+ }
+ : undefined,
+ data: [...(s.points || [])]
+ .filter((p) => p.ts >= from)
+ .sort((a, b) => a.ts - b.ts)
+ .map((p) => [p.ts, p.value])
+ }
+ })
+ },
+ true
+ )
+ }
+
+ const onResize = () => chart?.resize()
+ let resizeObserver: ResizeObserver | undefined
+
+ onMounted(() => {
+ render()
+ window.addEventListener('resize', onResize)
+ if (el.value && typeof ResizeObserver !== 'undefined') {
+ resizeObserver = new ResizeObserver(() => onResize())
+ resizeObserver.observe(el.value)
+ }
+ })
+ onBeforeUnmount(() => {
+ window.removeEventListener('resize', onResize)
+ resizeObserver?.disconnect()
+ chart?.dispose()
+ chart = undefined
+ })
+ watch(
+ () => [props.series, props.windowMs, props.emptyText, props.unit, props.height],
+ () => render(),
+ { deep: true }
+ )
+
+ return () => (
+ <div
+ ref={el}
+ class="w-full"
+ style={{ height: `${props.height}px`, minHeight: `${props.height}px` }}
+ />
+ )
+ }
+})
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/types.ts b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/types.ts
new file mode 100644
index 0000000000..836824795f
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/components/live-metrics-chart/types.ts
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+export type MetricUnit = 'ratio' | 'duration' | 'count'
+
+export interface LiveMetricPoint {
+ ts: number
+ value: number
+}
+
+export interface LiveMetricSeries {
+ /** Stable id, e.g. vertex:12:sourceReadRatio */
+ id: string
+ /** Short legend label, e.g. Source[0] · Source Read Ratio */
+ name: string
+ /** Full label for tooltip when the legend is abbreviated */
+ fullName?: string
+ unit: MetricUnit
+ points: LiveMetricPoint[]
+}
+
+export interface LiveLineChartProps {
+ series: LiveMetricSeries[]
+ windowMs: number
+ emptyText?: string
+ height?: number
+ unit?: MetricUnit
+}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/locales/en_US/detail.ts b/seatunnel-engine/seatunnel-engine-ui/src/locales/en_US/detail.ts
index 5bf47c677e..af6b27456c 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/locales/en_US/detail.ts
+++ b/seatunnel-engine/seatunnel-engine-ui/src/locales/en_US/detail.ts
@@ -23,7 +23,7 @@ export default {
overview: 'Overview',
exception: 'Exception',
configuration: 'Configuration',
- log: 'Log',
+ log: 'Log'
},
table: {
name: 'Name',
@@ -34,7 +34,7 @@ export default {
receivedQps: 'Received QPS',
writeQps: 'Write QPS',
receivedBytesPerSecond: 'Received Bytes PerSecond',
- writeBytesPerSecond: 'Write Bytes PerSecond',
+ writeBytesPerSecond: 'Write Bytes PerSecond'
},
observability: {
time: 'Time',
@@ -51,6 +51,18 @@ export default {
writeMsPerRecord: 'Write (ms/record)',
// Edge
bpRatio: 'Downstream Wait Ratio',
- queueFillRatio: 'Queue Fill Ratio',
+ queueFillRatio: 'Queue Fill Ratio'
},
+ liveMetrics: {
+ pinnedTitle: 'Pinned live metrics',
+ pinnedHint: 'Session · max {limit} · shared Overview poll',
+ emptyPinned: 'Pin a metric from the vertex/edge drawer to keep watching here.',
+ pin: 'Pin',
+ unpin: 'Unpin',
+ pinLimit: 'Pin limit reached ({limit}). Unpin one first.',
+ chartEmpty: 'No series in the current window',
+ unitRatio: 'Ratio',
+ unitDuration: 'Duration (ms/record)',
+ unitCount: 'Records'
+ }
}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/locales/zh_CN/detail.ts b/seatunnel-engine/seatunnel-engine-ui/src/locales/zh_CN/detail.ts
index a905021899..a0ae8626a2 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/locales/zh_CN/detail.ts
+++ b/seatunnel-engine/seatunnel-engine-ui/src/locales/zh_CN/detail.ts
@@ -23,7 +23,7 @@ export default {
overview: '概览',
exception: '异常',
configuration: '配置',
- log: '日志',
+ log: '日志'
},
table: {
name: '名称',
@@ -34,7 +34,7 @@ export default {
receivedQps: '读取 QPS',
writeQps: '写入 QPS',
receivedBytesPerSecond: '读取字节/秒',
- writeBytesPerSecond: '写入字节/秒',
+ writeBytesPerSecond: '写入字节/秒'
},
observability: {
time: '时间',
@@ -51,6 +51,18 @@ export default {
writeMsPerRecord: '写入耗时(毫秒/条)',
// Edge
bpRatio: '下游等待占比',
- queueFillRatio: '队列填充率',
+ queueFillRatio: '队列填充率'
},
+ liveMetrics: {
+ pinnedTitle: '已固定实时指标',
+ pinnedHint: '会话级 · 最多 {limit} 条 · 复用 Overview 轮询',
+ emptyPinned: '在节点/边抽屉中 Pin 指标后,可在此持续观察。',
+ pin: '固定',
+ unpin: '取消固定',
+ pinLimit: '已达 Pin 上限({limit}),请先取消一条。',
+ chartEmpty: '当前窗口暂无时序数据',
+ unitRatio: '占比',
+ unitDuration: '耗时(毫秒/条)',
+ unitCount: '条数'
+ }
}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/service/realtime-metrics/index.ts b/seatunnel-engine/seatunnel-engine-ui/src/service/realtime-metrics/index.ts
index da4a86e746..21a8207790 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/service/realtime-metrics/index.ts
+++ b/seatunnel-engine/seatunnel-engine-ui/src/service/realtime-metrics/index.ts
@@ -71,6 +71,46 @@ export interface RealtimeVerticesResponse {
}>
}
+/** Overview poll interval. Pinning must not add extra REST traffic. */
+export const REALTIME_POLL_INTERVAL_MS = 2000
+/** Default query window (3 minutes). */
+export const REALTIME_WINDOW_MS_DEFAULT = 3 * 60 * 1000
+/** Hard cap for the query window (10 minutes). */
+export const REALTIME_WINDOW_MS_MAX = 10 * 60 * 1000
+
+export const effectiveRealtimeWindowMs = (windowMs = REALTIME_WINDOW_MS_DEFAULT) =>
+ Math.min(windowMs, REALTIME_WINDOW_MS_MAX)
+
+/**
+ * Shared job-level fetch used by Job Detail Overview and follow-up
+ * observability views. Callers own the poll loop; this helper does not start one.
+ */
+export const fetchJobRealtimeMetrics = async (
+ jobId: string,
+ windowMs = REALTIME_WINDOW_MS_DEFAULT
+) => {
+ const effectiveWindowMs = effectiveRealtimeWindowMs(windowMs)
+ const [edges, vertices] = await Promise.all([
+ getRealtimeJobEdges(jobId, effectiveWindowMs),
+ getRealtimeJobVertices(jobId, effectiveWindowMs)
+ ])
+ return { edges, vertices, windowMs: effectiveWindowMs }
+}
+
+export const describeRealtimeFetchError = (error: unknown): string => {
+ const status = (error as { response?: { status?: number } })?.response?.status
+ if (status === 503) {
+ return 'Realtime metrics disabled on master'
+ }
+ if (status === 404) {
+ return 'Realtime metrics job not found'
+ }
+ if (status === 401 || status === 403) {
+ return 'Realtime metrics unauthorized'
+ }
+ return 'Failed to fetch realtime metrics'
+}
+
export const getRealtimeJobEdges = (jobId: string, windowMs: number) =>
get<RealtimeEdgesResponse>(`/metrics/realtime/jobs/${jobId}/edges`, { windowMs })
@@ -79,5 +119,6 @@ export const getRealtimeJobVertices = (jobId: string, windowMs: number) =>
export const RealtimeMetricsService = {
getRealtimeJobEdges,
- getRealtimeJobVertices
+ getRealtimeJobVertices,
+ fetchJobRealtimeMetrics
}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/store/live-metrics-pin/index.ts b/seatunnel-engine/seatunnel-engine-ui/src/store/live-metrics-pin/index.ts
new file mode 100644
index 0000000000..139b353b46
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/store/live-metrics-pin/index.ts
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { defineStore } from 'pinia'
+
+export const LIVE_METRICS_PIN_LIMIT = 6
+
+export interface PinnedMetricRef {
+ id: string
+ name: string
+ /** vertex | edge */
+ kind: 'vertex' | 'edge'
+ /** vertexId or targetVertexId for edge */
+ targetId: number
+ /** field key on the realtime point */
+ field: string
+}
+
+interface LiveMetricsPinState {
+ jobId: string | null
+ pins: PinnedMetricRef[]
+}
+
+export const useLiveMetricsPinStore = defineStore({
+ id: 'live-metrics-pin',
+ state: (): LiveMetricsPinState => ({
+ jobId: null,
+ pins: []
+ }),
+ getters: {
+ pinIds(state): Set<string> {
+ return new Set(state.pins.map((p) => p.id))
+ },
+ isPinned:
+ (state) =>
+ (id: string): boolean =>
+ state.pins.some((p) => p.id === id),
+ remaining(state): number {
+ return Math.max(0, LIVE_METRICS_PIN_LIMIT - state.pins.length)
+ }
+ },
+ actions: {
+ /**
+ * Bind pins to the current Job Detail visit. Switching job clears pins.
+ */
+ ensureJob(jobId: string) {
+ if (this.jobId !== jobId) {
+ this.jobId = jobId
+ this.pins = []
+ }
+ },
+ clear() {
+ this.pins = []
+ this.jobId = null
+ },
+ /**
+ * @returns true if pinned; false if already present or limit reached
+ */
+ pin(ref: PinnedMetricRef): 'ok' | 'exists' | 'limit' {
+ if (this.pins.some((p) => p.id === ref.id)) return 'exists'
+ if (this.pins.length >= LIVE_METRICS_PIN_LIMIT) return 'limit'
+ this.pins.push(ref)
+ return 'ok'
+ },
+ unpin(id: string) {
+ this.pins = this.pins.filter((p) => p.id !== id)
+ },
+ toggle(ref: PinnedMetricRef): 'ok' | 'exists' | 'limit' | 'removed' {
+ if (this.pins.some((p) => p.id === ref.id)) {
+ this.unpin(ref.id)
+ return 'removed'
+ }
+ return this.pin(ref)
+ }
+ }
+})
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/tests/detail-live-metrics.spec.ts b/seatunnel-engine/seatunnel-engine-ui/src/tests/detail-live-metrics.spec.ts
new file mode 100644
index 0000000000..cd6668f959
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/tests/detail-live-metrics.spec.ts
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { describe, expect, test } from 'vitest'
+import {
+ buildSeriesFromVertexPoints,
+ decodeTargetVertexId,
+ resolvePinnedSeries,
+ shortOperatorLabel,
+ vertexPinFields,
+ vertexSeriesId
+} from '@/views/jobs/detail-live-metrics'
+import {
+ formatMetricValue,
+ groupSeriesByUnit,
+ inferMetricUnit
+} from '@/components/live-metrics-chart/group'
+import {
+ describeRealtimeFetchError,
+ effectiveRealtimeWindowMs,
+ REALTIME_WINDOW_MS_DEFAULT,
+ REALTIME_WINDOW_MS_MAX
+} from '@/service/realtime-metrics'
+import type { RealtimeVertexPoint, RealtimeVerticesResponse } from '@/service/realtime-metrics'
+
+describe('detail live metrics helpers', () => {
+ test('decodes negative queue ids to target vertex ids', () => {
+ expect(decodeTargetVertexId(7)).toBe(7)
+ expect(decodeTargetVertexId(-2)).toBe(1)
+ expect(decodeTargetVertexId(-10)).toBe(5)
+ })
+
+ test('builds a vertex series sorted by ts and mapped to ms', () => {
+ const field = vertexPinFields('transform').find(
+ (item) => item.field === 'transformProcessNsPerRecord'
+ )
+ expect(field).toBeDefined()
+ expect(field?.unit).toBe('duration')
+ const series = buildSeriesFromVertexPoints(
+ 12,
+ 'Transform[0]',
+ field!,
+ [
+ { ts: 2000, transformProcessNsPerRecord: 4_000_000 } as RealtimeVertexPoint,
+ { ts: 1000, transformProcessNsPerRecord: 2_000_000 } as RealtimeVertexPoint
+ ],
+ 10
+ )
+ expect(series.id).toBe(vertexSeriesId(12, 'transformProcessNsPerRecord'))
+ expect(series.points).toEqual([
+ { ts: 1000, value: 2 },
+ { ts: 2000, value: 4 }
+ ])
+ })
+
+ test('resolves pinned vertex series from the shared realtime response', () => {
+ const vertices: RealtimeVerticesResponse = {
+ bucketMs: 5000,
+ fromMs: 0,
+ toMs: 5000,
+ vertices: [
+ {
+ vertexId: 1,
+ points: [{ ts: 1000, sourceReadRatio: 0.4 } as RealtimeVertexPoint]
+ }
+ ]
+ }
+ const series = resolvePinnedSeries(
+ [
+ {
+ id: 'vertex:1:sourceReadRatio',
+ name: 'Source[0] · Source Read Ratio',
+ kind: 'vertex',
+ targetId: 1,
+ field: 'sourceReadRatio'
+ }
+ ],
+ vertices,
+ undefined,
+ { 1: 'Source[0]' },
+ 10
+ )
+ expect(series).toHaveLength(1)
+ expect(series[0].name).toBe('Source[0] · Source Read Ratio')
+ expect(series[0].unit).toBe('ratio')
+ expect(series[0].points).toEqual([{ ts: 1000, value: 0.4 }])
+ })
+})
+
+describe('metric display grouping', () => {
+ test('shortens vertex names to operator identifiers', () => {
+ expect(shortOperatorLabel('pipeline-1 [Source[0]-FakeSource]')).toBe('Source[0]')
+ expect(shortOperatorLabel('plain-name')).toBe('plain-name')
+ })
+
+ test('splits mixed units so ratios are not flattened by counts', () => {
+ expect(inferMetricUnit('sourceReadRatio')).toBe('ratio')
+ expect(inferMetricUnit('sinkWriteNsPerRecord')).toBe('duration')
+ expect(inferMetricUnit('sinkRecordsIn')).toBe('count')
+ const groups = groupSeriesByUnit([
+ { id: 'a', name: 'read', unit: 'ratio', points: [{ ts: 1, value: 0.4 }] },
+ { id: 'b', name: 'in', unit: 'count', points: [{ ts: 1, value: 80 }] }
+ ])
+ expect(groups.map((g) => g.unit)).toEqual(['ratio', 'count'])
+ expect(formatMetricValue('ratio', 0.452)).toBe('45.2%')
+ expect(formatMetricValue('duration', 1.888)).toBe('1.89 ms')
+ })
+})
+
+describe('shared realtime fetch helpers', () => {
+ test('caps the query window at 10 minutes', () => {
+ expect(effectiveRealtimeWindowMs()).toBe(REALTIME_WINDOW_MS_DEFAULT)
+ expect(effectiveRealtimeWindowMs(20 * 60 * 1000)).toBe(REALTIME_WINDOW_MS_MAX)
+ })
+
+ test('maps fetch errors without leaking response bodies', () => {
+ expect(describeRealtimeFetchError({ response: { status: 503 } })).toBe(
+ 'Realtime metrics disabled on master'
+ )
+ expect(describeRealtimeFetchError({ response: { status: 404 } })).toBe(
+ 'Realtime metrics job not found'
+ )
+ expect(describeRealtimeFetchError(new Error('network'))).toBe(
+ 'Failed to fetch realtime metrics'
+ )
+ })
+})
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/tests/detail.spec.ts b/seatunnel-engine/seatunnel-engine-ui/src/tests/detail.spec.ts
index 2b5be15960..6dfbabdfdc 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/tests/detail.spec.ts
+++ b/seatunnel-engine/seatunnel-engine-ui/src/tests/detail.spec.ts
@@ -17,8 +17,9 @@
import { describe, test, expect, vi, beforeEach } from 'vitest'
import { flushPromises, mount } from '@vue/test-utils'
-import { createApp } from 'vue'
+import { createApp, h } from 'vue'
import { createPinia, setActivePinia } from 'pinia'
+import { NMessageProvider } from 'naive-ui'
import i18n from '@/locales'
import detail from '@/views/jobs/detail'
import { getJobInfo } from '@/service/job'
@@ -38,6 +39,28 @@ vi.mock('@/components/directed-acyclic-graph', () => ({
default: { template: '<div></div>' }
}))
+vi.mock('@/components/live-metrics-chart', () => ({
+ default: { template: '<div></div>' }
+}))
+
+vi.mock('@/components/live-metrics-chart/board', () => ({
+ default: { template: '<div></div>' }
+}))
+
+vi.mock('@/service/realtime-metrics', async () => {
+ const actual = await vi.importActual<typeof import('@/service/realtime-metrics')>(
+ '@/service/realtime-metrics'
+ )
+ return {
+ ...actual,
+ fetchJobRealtimeMetrics: vi.fn().mockResolvedValue({
+ edges: { bucketMs: 5000, fromMs: 0, toMs: 0, edges: [] },
+ vertices: { bucketMs: 5000, fromMs: 0, toMs: 0, vertices: [] },
+ windowMs: actual.REALTIME_WINDOW_MS_DEFAULT
+ })
+ }
+})
+
describe('detail', () => {
const app = createApp({})
beforeEach(() => {
@@ -95,11 +118,20 @@ describe('detail', () => {
vi.mocked(getJobInfo).mockResolvedValue(mockJob)
- const wrapper = mount(detail, {
- global: {
- plugins: [i18n]
+ const pinia = createPinia()
+ setActivePinia(pinia)
+ const wrapper = mount(
+ {
+ setup() {
+ return () => h(NMessageProvider, null, { default: () => h(detail) })
+ }
+ },
+ {
+ global: {
+ plugins: [i18n, pinia]
+ }
}
- })
+ )
await flushPromises()
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/tests/live-metrics-pin.spec.ts b/seatunnel-engine/seatunnel-engine-ui/src/tests/live-metrics-pin.spec.ts
new file mode 100644
index 0000000000..c958f2d4f3
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/tests/live-metrics-pin.spec.ts
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { beforeEach, describe, expect, it } from 'vitest'
+import { createPinia, setActivePinia } from 'pinia'
+import { LIVE_METRICS_PIN_LIMIT, useLiveMetricsPinStore } from '@/store/live-metrics-pin'
+
+const samplePin = (index: number) => ({
+ id: `vertex:${index}:sourceReadRatio`,
+ name: `Source ${index} · Source Read Ratio`,
+ kind: 'vertex' as const,
+ targetId: index,
+ field: 'sourceReadRatio'
+})
+
+describe('live metrics pin store', () => {
+ beforeEach(() => {
+ setActivePinia(createPinia())
+ })
+
+ it('pins a series and unpins it', () => {
+ const store = useLiveMetricsPinStore()
+ store.ensureJob('job-1')
+ expect(store.pin(samplePin(1))).toBe('ok')
+ expect(store.pins).toHaveLength(1)
+ expect(store.isPinned('vertex:1:sourceReadRatio')).toBe(true)
+ expect(store.pin(samplePin(1))).toBe('exists')
+ expect(store.toggle(samplePin(1))).toBe('removed')
+ expect(store.pins).toHaveLength(0)
+ })
+
+ it('rejects pins beyond the documented limit', () => {
+ const store = useLiveMetricsPinStore()
+ store.ensureJob('job-1')
+ for (let i = 1; i <= LIVE_METRICS_PIN_LIMIT; i++) {
+ expect(store.pin(samplePin(i))).toBe('ok')
+ }
+ expect(store.pin(samplePin(LIVE_METRICS_PIN_LIMIT + 1))).toBe('limit')
+ expect(store.pins).toHaveLength(LIVE_METRICS_PIN_LIMIT)
+ expect(store.remaining).toBe(0)
+ })
+
+ it('clears pins when switching jobs or leaving the page', () => {
+ const store = useLiveMetricsPinStore()
+ store.ensureJob('job-1')
+ store.pin(samplePin(1))
+ store.ensureJob('job-2')
+ expect(store.pins).toHaveLength(0)
+ store.pin(samplePin(2))
+ store.clear()
+ expect(store.pins).toHaveLength(0)
+ expect(store.jobId).toBeNull()
+ })
+})
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail-live-metrics.ts b/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail-live-metrics.ts
new file mode 100644
index 0000000000..6225083a53
--- /dev/null
+++ b/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail-live-metrics.ts
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import type { LiveMetricSeries, MetricUnit } from '@/components/live-metrics-chart/types'
+import { inferMetricUnit } from '@/components/live-metrics-chart/group'
+import type { PinnedMetricRef } from '@/store/live-metrics-pin'
+import type {
+ RealtimeEdgePoint,
+ RealtimeEdgesResponse,
+ RealtimeVertexPoint,
+ RealtimeVerticesResponse
+} from '@/service/realtime-metrics'
+import { extractVertexIdentifier } from './detail-metrics'
+
+export type Translate = (key: string) => string
+
+export type PinableField = {
+ field: string
+ label: string
+ unit: MetricUnit
+ /** Transform point value for charting (e.g. ns -> ms) */
+ mapValue?: (v: number) => number
+}
+
+const identityTranslate: Translate = (key) => key
+const ratio = (v: number) => v
+const nsToMs = (v: number) => v / 1_000_000
+
+export const shortOperatorLabel = (name?: string): string => {
+ if (!name) return ''
+ return extractVertexIdentifier(name) || name
+}
+
+export const vertexPinFields = (
+ type?: string,
+ t: Translate = identityTranslate
+): PinableField[] => {
+ if (type === 'source') {
+ return [
+ {
+ field: 'sourceReadRatio',
+ label: t('detail.observability.sourceReadRatio'),
+ unit: 'ratio',
+ mapValue: ratio
+ },
+ {
+ field: 'sourceIdleRatio',
+ label: t('detail.observability.sourceIdleRatio'),
+ unit: 'ratio',
+ mapValue: ratio
+ }
+ ]
+ }
+ if (type === 'transform') {
+ return [
+ {
+ field: 'transformBusyRatio',
+ label: t('detail.observability.transformBusyRatio'),
+ unit: 'ratio',
+ mapValue: ratio
+ },
+ {
+ field: 'transformProcessNsPerRecord',
+ label: t('detail.observability.processMsPerRecord'),
+ unit: 'duration',
+ mapValue: nsToMs
+ },
+ { field: 'transformRecordsIn', label: t('detail.observability.recordsIn'), unit: 'count' },
+ { field: 'transformRecordsOut', label: t('detail.observability.recordsOut'), unit: 'count' }
+ ]
+ }
+ if (type === 'sink') {
+ return [
+ {
+ field: 'sinkBusyRatio',
+ label: t('detail.observability.sinkBusyRatio'),
+ unit: 'ratio',
+ mapValue: ratio
+ },
+ {
+ field: 'sinkWriteNsPerRecord',
+ label: t('detail.observability.writeMsPerRecord'),
+ unit: 'duration',
+ mapValue: nsToMs
+ },
+ { field: 'sinkRecordsIn', label: t('detail.observability.recordsIn'), unit: 'count' }
+ ]
+ }
+ return []
+}
+
+export const edgePinFields = (t: Translate = identityTranslate): PinableField[] => [
+ { field: 'bpRatio', label: t('detail.observability.bpRatio'), unit: 'ratio', mapValue: ratio },
+ {
+ field: 'queueFillRatio',
+ label: t('detail.observability.queueFillRatio'),
+ unit: 'ratio',
+ mapValue: ratio
+ }
+]
+
+export const vertexSeriesId = (vertexId: number, field: string) => `vertex:${vertexId}:${field}`
+export const edgeSeriesId = (targetVertexId: number, field: string) =>
+ `edge:${targetVertexId}:${field}`
+
+const readPointField = (point: Record<string, unknown>, field: string): number | undefined => {
+ const v = point[field]
+ return typeof v === 'number' && Number.isFinite(v) ? v : undefined
+}
+
+export const buildSeriesFromVertexPoints = (
+ vertexId: number,
+ vertexName: string,
+ field: PinableField,
+ points: RealtimeVertexPoint[],
+ limit: number
+): LiveMetricSeries => {
+ const mapValue = field.mapValue || ((v: number) => v)
+ const shortName = `${shortOperatorLabel(vertexName)} · ${field.label}`
+ return {
+ id: vertexSeriesId(vertexId, field.field),
+ name: shortName,
+ fullName: `${vertexName} · ${field.label}`,
+ unit: field.unit,
+ points: [...points]
+ .slice(-limit)
+ .sort((a, b) => a.ts - b.ts)
+ .map((p) => {
+ const raw = readPointField(p as unknown as Record<string, unknown>, field.field)
+ return raw === undefined ? null : { ts: p.ts, value: mapValue(raw) }
+ })
+ .filter((p): p is { ts: number; value: number } => p !== null)
+ }
+}
+
+export const buildSeriesFromEdgePoints = (
+ targetVertexId: number,
+ edgeLabel: string,
+ field: PinableField,
+ points: RealtimeEdgePoint[],
+ limit: number
+): LiveMetricSeries => {
+ const mapValue = field.mapValue || ((v: number) => v)
+ const shortName = `${edgeLabel} · ${field.label}`
+ return {
+ id: edgeSeriesId(targetVertexId, field.field),
+ name: shortName,
+ fullName: shortName,
+ unit: field.unit,
+ points: [...points]
+ .slice(-limit)
+ .sort((a, b) => a.ts - b.ts)
+ .map((p) => {
+ const raw = readPointField(p as unknown as Record<string, unknown>, field.field)
+ return raw === undefined ? null : { ts: p.ts, value: mapValue(raw) }
+ })
+ .filter((p): p is { ts: number; value: number } => p !== null)
+ }
+}
+
+export const decodeTargetVertexId = (queueId: number) => {
+ if (queueId >= 0) return queueId
+ const abs = Math.abs(queueId)
+ if (!abs) return undefined
+ if (abs % 2 === 0) return abs / 2
+ return (abs - 1) / 2
+}
+
+export const resolvePinnedSeries = (
+ pins: PinnedMetricRef[],
+ vertices: RealtimeVerticesResponse | undefined,
+ edges: RealtimeEdgesResponse | undefined,
+ vertexNameById: Record<number, string>,
+ limit: number
+): LiveMetricSeries[] => {
+ return pins
+ .map((pin) => {
+ if (pin.kind === 'vertex') {
+ const points = vertices?.vertices?.find((v) => v.vertexId === pin.targetId)?.points || []
+ const field: PinableField = {
+ field: pin.field,
+ label: pin.name.split(' · ').slice(1).join(' · ') || pin.field,
+ unit: inferMetricUnit(pin.field),
+ mapValue: pin.field.endsWith('NsPerRecord') ? nsToMs : undefined
+ }
+ return buildSeriesFromVertexPoints(
+ pin.targetId,
+ vertexNameById[pin.targetId] || String(pin.targetId),
+ field,
+ points,
+ limit
+ )
+ }
+ const points =
+ edges?.edges?.find(
+ (e) => (e.targetVertexId ?? decodeTargetVertexId(e.queueId)) === pin.targetId
+ )?.points || []
+ const field: PinableField = {
+ field: pin.field,
+ label: pin.name.split(' · ').slice(1).join(' · ') || pin.field,
+ unit: inferMetricUnit(pin.field)
+ }
+ return buildSeriesFromEdgePoints(
+ pin.targetId,
+ pin.name.split(' · ')[0] || 'edge',
+ field,
+ points,
+ limit
+ )
+ })
+ .map((s, idx) => ({
+ ...s,
+ name: pins[idx]?.name || s.name
+ }))
+}
diff --git a/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail.tsx b/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail.tsx
index cf15932833..ac8eaf0b31 100644
--- a/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail.tsx
+++ b/seatunnel-engine/seatunnel-engine-ui/src/views/jobs/detail.tsx
@@ -23,9 +23,12 @@ import {
NDataTable,
type DataTableColumns,
NDrawer,
- NDrawerContent
+ NDrawerContent,
+ NButton,
+ NSpace,
+ useMessage
} from 'naive-ui'
-import {computed, defineComponent, onUnmounted, reactive, ref, watch} from 'vue'
+import { computed, defineComponent, onUnmounted, reactive, ref, watch } from 'vue'
import { getJobInfo } from '@/service/job'
import { useRoute } from 'vue-router'
import type { Job, Vertex } from '@/service/job/types'
@@ -33,27 +36,53 @@ import { useI18n } from 'vue-i18n'
import { getRemainTime } from '@/utils/time'
import { parse } from 'date-fns'
import DAG, { type DagEdgeInfo } from '@/components/directed-acyclic-graph'
+import LiveMetricsBoard from '@/components/live-metrics-chart/board'
import { getColorFromStatus } from '@/utils/getTypeFromStatus'
import './detail.scss'
import Configuration from '@/components/configuration'
import JobLog from '@/components/job-log'
-import { formatPercentFromRatio } from '@/utils/format'
import {
- getRealtimeJobEdges,
- getRealtimeJobVertices,
+ REALTIME_POLL_INTERVAL_MS,
+ REALTIME_WINDOW_MS_DEFAULT,
+ describeRealtimeFetchError,
+ effectiveRealtimeWindowMs,
+ fetchJobRealtimeMetrics,
type RealtimeEdgesResponse,
type RealtimeEdgePoint,
type RealtimeVerticesResponse,
type RealtimeVertexPoint
} from '@/service/realtime-metrics'
-import { readVertexMetricValue, collectVertexMetrics, extractVertexIdentifier } from './detail-metrics'
+import {
+ readVertexMetricValue,
+ collectVertexMetrics,
+ extractVertexIdentifier
+} from './detail-metrics'
+import {
+ LIVE_METRICS_PIN_LIMIT,
+ useLiveMetricsPinStore,
+ type PinnedMetricRef
+} from '@/store/live-metrics-pin'
+import {
+ buildSeriesFromEdgePoints,
+ buildSeriesFromVertexPoints,
+ decodeTargetVertexId,
+ edgePinFields,
+ edgeSeriesId,
+ resolvePinnedSeries,
+ shortOperatorLabel,
+ vertexPinFields,
+ vertexSeriesId
+} from './detail-live-metrics'
export default defineComponent({
setup() {
const { t } = useI18n()
const route = useRoute()
+ const message = useMessage()
+ const pinStore = useLiveMetricsPinStore()
const jobId = route.params.jobId as string
+ pinStore.ensureJob(jobId)
const job = reactive({} as Job)
const duration = ref('')
let timer: NodeJS.Timeout
@@ -66,6 +95,7 @@ export default defineComponent({
duration.value = getRemainTime(Math.abs(Date.now() - d.getTime()))
if (isTerminalState(job.jobStatus)) {
clearTimeout(fetchTimer)
+ pinStore.clear()
return
}
fetchTimer = setTimeout(fetch, 5000)
@@ -89,10 +119,11 @@ export default defineComponent({
clearInterval(timer)
clearTimeout(fetchTimer)
clearInterval(realtimeTimer)
+ pinStore.clear()
})
const isTerminalState = (status: string) => {
- return ['FINISHED', 'FAILED', 'CANCELED','SAVEPOINT_DONE'].includes(status)
+ return ['FINISHED', 'FAILED', 'CANCELED', 'SAVEPOINT_DONE'].includes(status)
}
const isRunningState = (status: string) => {
@@ -238,8 +269,7 @@ export default defineComponent({
const realtimeEdges = ref<RealtimeEdgesResponse>()
const realtimeVertices = ref<RealtimeVerticesResponse>()
const realtimeTick = ref(0)
- const realtimeWindowMs = 3 * 60 * 1000
- const realtimeWindowMsMax = 10 * 60 * 1000
+ const realtimeWindowMs = REALTIME_WINDOW_MS_DEFAULT
const realtimeError = ref<string | null>(null)
const realtimeConsecutiveErrors = ref(0)
let realtimeTimer: NodeJS.Timeout
@@ -247,10 +277,7 @@ export default defineComponent({
if (!isRunningState(job.jobStatus)) return
if (select.value !== 'Overview') return
try {
- const [edges, vertices] = await Promise.all([
- getRealtimeJobEdges(jobId, Math.min(realtimeWindowMs, realtimeWindowMsMax)),
- getRealtimeJobVertices(jobId, Math.min(realtimeWindowMs, realtimeWindowMsMax))
- ])
+ const { edges, vertices } = await fetchJobRealtimeMetrics(jobId, realtimeWindowMs)
realtimeEdges.value = edges
realtimeVertices.value = vertices
realtimeTick.value++
@@ -258,16 +285,7 @@ export default defineComponent({
realtimeConsecutiveErrors.value = 0
} catch (e) {
realtimeConsecutiveErrors.value++
- const status = (e as any)?.response?.status
- if (status === 503) {
- realtimeError.value = 'Realtime metrics disabled on master'
- } else if (status === 404) {
- realtimeError.value = 'Realtime metrics job not found'
- } else if (status === 401 || status === 403) {
- realtimeError.value = 'Realtime metrics unauthorized'
- } else {
- realtimeError.value = 'Failed to fetch realtime metrics'
- }
+ realtimeError.value = describeRealtimeFetchError(e)
if (realtimeConsecutiveErrors.value === 1) {
console.warn('Fetch realtime metrics failed:', e)
}
@@ -276,7 +294,7 @@ export default defineComponent({
const startRealtimePolling = () => {
clearInterval(realtimeTimer)
fetchRealtime()
- realtimeTimer = setInterval(fetchRealtime, 2000)
+ realtimeTimer = setInterval(fetchRealtime, REALTIME_POLL_INTERVAL_MS)
}
const stopRealtimePolling = () => {
clearInterval(realtimeTimer)
@@ -295,13 +313,6 @@ export default defineComponent({
const realtimeEdgeStats = computed<Record<number, RealtimeEdgePoint>>(() => {
const stats: Record<number, RealtimeEdgePoint> = {}
const edges = realtimeEdges.value?.edges || []
- const decodeTargetVertexId = (queueId: number) => {
- if (queueId >= 0) return queueId
- const abs = Math.abs(queueId)
- if (!abs) return undefined
- if (abs % 2 === 0) return abs / 2
- return (abs - 1) / 2
- }
edges.forEach((e) => {
const last = e.points?.[e.points.length - 1]
if (!last) return
@@ -324,42 +335,110 @@ export default defineComponent({
const realtimeSeriesLimit = computed(() => {
const bucketMs = realtimeEdges.value?.bucketMs || realtimeVertices.value?.bucketMs || 5000
- const effectiveWindowMs = Math.min(realtimeWindowMs, realtimeWindowMsMax)
+ const effectiveWindowMs = effectiveRealtimeWindowMs(realtimeWindowMs)
const safeBucketMs = Math.max(1, bucketMs)
return Math.max(1, Math.ceil(effectiveWindowMs / safeBucketMs) + 1)
})
- const focusedEdgeSeries = computed(() => {
- const targetId = focusedEdge.value?.targetVertexId
- if (!targetId) return []
- const decodeTargetVertexId = (queueId: number) => {
- if (queueId >= 0) return queueId
- const abs = Math.abs(queueId)
- if (!abs) return undefined
- if (abs % 2 === 0) return abs / 2
- return (abs - 1) / 2
- }
+ const vertexNameById = computed(() => {
+ const map: Record<number, string> = {}
+ job.jobDag?.vertexInfoMap?.forEach((v) => {
+ map[v.vertexId] = v.vertexName
+ })
+ return map
+ })
+
+ const pinnedSeries = computed(() => {
+ // depend on realtimeTick so chart refreshes with poll
+ void realtimeTick.value
+ return resolvePinnedSeries(
+ pinStore.pins,
+ realtimeVertices.value,
+ realtimeEdges.value,
+ vertexNameById.value,
+ realtimeSeriesLimit.value
+ )
+ })
+
+ const drawerVertexChartSeries = computed(() => {
+ const vertex = job.jobDag?.vertexInfoMap?.find((v) => v.vertexId === focusedId.value)
+ if (!vertex) return []
const points =
- realtimeEdges.value?.edges?.find(
- (e) => (e.targetVertexId ?? decodeTargetVertexId(e.queueId)) === targetId
- )?.points || []
- return points
- .slice(-realtimeSeriesLimit.value)
- .slice()
- .sort((a, b) => b.ts - a.ts)
+ realtimeVertices.value?.vertices?.find((v) => v.vertexId === vertex.vertexId)?.points || []
+ return vertexPinFields(vertex.type, t).map((field) => {
+ return buildSeriesFromVertexPoints(
+ vertex.vertexId,
+ vertex.vertexName,
+ field,
+ points,
+ realtimeSeriesLimit.value
+ )
+ })
})
- const focusedVertexSeries = computed(() => {
- const vertexId = focusedId.value
- if (!vertexId) return []
+ const drawerEdgeChartSeries = computed(() => {
+ const edge = focusedEdge.value
+ if (!edge) return []
+ const input = job.jobDag?.vertexInfoMap?.find((v) => v.vertexId === edge.inputVertexId)
+ const target = job.jobDag?.vertexInfoMap?.find((v) => v.vertexId === edge.targetVertexId)
+ const label = `${shortOperatorLabel(input?.vertexName || String(edge.inputVertexId))} → ${shortOperatorLabel(target?.vertexName || String(edge.targetVertexId))}`
const points =
- realtimeVertices.value?.vertices?.find((v) => v.vertexId === vertexId)?.points || []
- return points
- .slice(-realtimeSeriesLimit.value)
- .slice()
- .sort((a, b) => b.ts - a.ts)
+ realtimeEdges.value?.edges?.find(
+ (e) => (e.targetVertexId ?? decodeTargetVertexId(e.queueId)) === edge.targetVertexId
+ )?.points || []
+ return edgePinFields(t).map((field) => {
+ return buildSeriesFromEdgePoints(
+ edge.targetVertexId,
+ label,
+ field,
+ points,
+ realtimeSeriesLimit.value
+ )
+ })
})
+ const onTogglePin = (ref: PinnedMetricRef) => {
+ const result = pinStore.toggle(ref)
+ if (result === 'limit') {
+ message.warning(t('detail.liveMetrics.pinLimit', { limit: LIVE_METRICS_PIN_LIMIT }))
+ }
+ }
+
+ const renderPinControls = (
+ kind: 'vertex' | 'edge',
+ targetId: number,
+ baseName: string,
+ fields: ReturnType<typeof vertexPinFields>
+ ) => (
+ <NSpace class="mb-3" size="small" wrap>
+ {fields.map((field) => {
+ const id =
+ kind === 'vertex'
+ ? vertexSeriesId(targetId, field.field)
+ : edgeSeriesId(targetId, field.field)
+ const pinned = pinStore.isPinned(id)
+ return (
+ <NButton
+ size="tiny"
+ type={pinned ? 'primary' : 'default'}
+ secondary={!pinned}
+ onClick={() =>
+ onTogglePin({
+ id,
+ name: `${shortOperatorLabel(baseName)} · ${field.label}`,
+ kind,
+ targetId,
+ field: field.field
+ })
+ }
+ >
+ {pinned ? t('detail.liveMetrics.unpin') : t('detail.liveMetrics.pin')} · {field.label}
+ </NButton>
+ )
+ })}
+ </NSpace>
+ )
+
const focusedVertex = computed(() => {
const vertex = job.jobDag?.vertexInfoMap?.find((v) => v.vertexId === focusedId.value)
const metrics = {} as any
@@ -390,13 +469,16 @@ export default defineComponent({
const vertexId = extractVertexIdentifier(vertex.vertexName)
if (vertexId) {
if (job.metrics?.FlushSignalTotalPerVertex?.[vertexId]) {
- metrics[`FlushSignalTotal.${vertexId}`] = job.metrics.FlushSignalTotalPerVertex[vertexId]
+ metrics[`FlushSignalTotal.${vertexId}`] =
+ job.metrics.FlushSignalTotalPerVertex[vertexId]
}
if (job.metrics?.FlushSignalQueueSuccessTotalPerVertex?.[vertexId]) {
- metrics[`FlushSignalQueueSuccess.${vertexId}`] = job.metrics.FlushSignalQueueSuccessTotalPerVertex[vertexId]
+ metrics[`FlushSignalQueueSuccess.${vertexId}`] =
+ job.metrics.FlushSignalQueueSuccessTotalPerVertex[vertexId]
}
if (job.metrics?.FlushSignalQueueFailureTotalPerVertex?.[vertexId]) {
- metrics[`FlushSignalQueueFailure.${vertexId}`] = job.metrics.FlushSignalQueueFailureTotalPerVertex[vertexId]
+ metrics[`FlushSignalQueueFailure.${vertexId}`] =
+ job.metrics.FlushSignalQueueFailureTotalPerVertex[vertexId]
}
}
}
@@ -415,10 +497,12 @@ export default defineComponent({
const vertexId = extractVertexIdentifier(vertex.vertexName)
if (vertexId) {
if (job.metrics?.FlushSignalSinkSuccessTotalPerVertex?.[vertexId]) {
- metrics[`FlushSignalSinkSuccess.${vertexId}`] = job.metrics.FlushSignalSinkSuccessTotalPerVertex[vertexId]
+ metrics[`FlushSignalSinkSuccess.${vertexId}`] =
+ job.metrics.FlushSignalSinkSuccessTotalPerVertex[vertexId]
}
if (job.metrics?.FlushSignalSinkFailureTotalPerVertex?.[vertexId]) {
- metrics[`FlushSignalSinkFailure.${vertexId}`] = job.metrics.FlushSignalSinkFailureTotalPerVertex[vertexId]
+ metrics[`FlushSignalSinkFailure.${vertexId}`] =
+ job.metrics.FlushSignalSinkFailureTotalPerVertex[vertexId]
}
}
}
@@ -472,106 +556,21 @@ export default defineComponent({
return { onClick: () => onFocus(row) }
}
- const edgePointColumns: DataTableColumns<RealtimeEdgePoint> = [
- {
- title: t('detail.observability.time'),
- key: 'ts',
- render: (row) => new Date(row.ts).toLocaleTimeString()
- },
- {
- title: t('detail.observability.bpRatio'),
- key: 'bpRatio',
- render: (row) => formatPercentFromRatio(row.bpRatio)
- },
- {
- title: t('detail.observability.queueFillRatio'),
- key: 'queueFillRatio',
- render: (row) => formatPercentFromRatio(row.queueFillRatio)
- }
- ]
-
- const vertexPointColumns = computed<DataTableColumns<RealtimeVertexPoint>>(() => {
- const base: DataTableColumns<RealtimeVertexPoint> = [
- {
- title: t('detail.observability.time'),
- key: 'ts',
- render: (row) => new Date(row.ts).toLocaleTimeString()
- }
- ]
- const v = focusedVertex.value as any
- const type = v?.type
- if (type === 'source') {
- return base.concat([
- {
- title: t('detail.observability.sourceReadRatio'),
- key: 'sourceReadRatio',
- render: (row) => formatPercentFromRatio(row.sourceReadRatio)
- },
- {
- title: t('detail.observability.sourceIdleRatio'),
- key: 'sourceIdleRatio',
- render: (row) => formatPercentFromRatio(row.sourceIdleRatio)
- }
- ])
- }
- if (type === 'transform') {
- return base.concat([
- {
- title: t('detail.observability.transformBusyRatio'),
- key: 'transformBusyRatio',
- render: (row) => formatPercentFromRatio(row.transformBusyRatio)
- },
- {
- title: t('detail.observability.processMsPerRecord'),
- key: 'transformProcessNsPerRecord',
- render: (row) => (row.transformProcessNsPerRecord / 1_000_000).toFixed(3)
- },
- {
- title: t('detail.observability.recordsIn'),
- key: 'transformRecordsIn'
- },
- {
- title: t('detail.observability.recordsOut'),
- key: 'transformRecordsOut'
- }
- ])
- }
- if (type === 'sink') {
- return base.concat([
- {
- title: t('detail.observability.sinkBusyRatio'),
- key: 'sinkBusyRatio',
- render: (row) => formatPercentFromRatio(row.sinkBusyRatio)
- },
- {
- title: t('detail.observability.writeMsPerRecord'),
- key: 'sinkWriteNsPerRecord',
- render: (row) => (row.sinkWriteNsPerRecord / 1_000_000).toFixed(3)
- },
- {
- title: t('detail.observability.recordsIn'),
- key: 'sinkRecordsIn'
- }
- ])
- }
- // Fallback: show a minimal common view.
- return base
- })
return () => (
<div class="w-full bg-white px-12 pt-6 pb-12 border border-gray-100 rounded-xl">
- <div class="font-bold text-xl">
- {job.jobName}
- <NTag bordered={false} color={getColorFromStatus(job.jobStatus)} class="ml-3">
- {job.jobStatus}
- </NTag>
- {realtimeError.value ? (
- <span title={realtimeError.value}>
- <NTag bordered={false} type="warning" class="ml-3">
- Realtime metrics unavailable
- </NTag>
- </span>
- ) : null}
- </div>
+ <div class="font-bold text-xl">
+ {job.jobName}
+ <NTag bordered={false} color={getColorFromStatus(job.jobStatus)} class="ml-3">
+ {job.jobStatus}
+ </NTag>
+ {realtimeError.value ? (
+ <span title={realtimeError.value}>
+ <NTag bordered={false} type="warning" class="ml-3">
+ Realtime metrics unavailable
+ </NTag>
+ </span>
+ ) : null}
+ </div>
<div class="mt-3 flex items-center gap-3">
<span>{t('detail.id')}:</span>
<span class="font-bold">{job.jobId}</span>
@@ -594,6 +593,38 @@ export default defineComponent({
realtimeVertexStats={realtimeVertexStats.value}
realtimeTick={realtimeTick.value}
/>
+ <div class="mt-2 mb-2 border border-gray-100 rounded-lg px-3 pt-2 pb-2 bg-gray-50">
+ <div class="flex items-baseline justify-between mb-2">
+ <div class="font-semibold text-base">{t('detail.liveMetrics.pinnedTitle')}</div>
+ <div class="text-xs text-gray-500">
+ {t('detail.liveMetrics.pinnedHint', { limit: LIVE_METRICS_PIN_LIMIT })}
+ {pinStore.pins.length
+ ? ` · ${pinStore.pins.length}/${LIVE_METRICS_PIN_LIMIT}`
+ : ''}
+ </div>
+ </div>
+ {pinStore.pins.length ? (
+ <NSpace class="mb-2" size="small" wrap>
+ {pinStore.pins.map((p) => (
+ <NTag key={p.id} closable type="info" onClose={() => pinStore.unpin(p.id)}>
+ {p.name}
+ </NTag>
+ ))}
+ </NSpace>
+ ) : null}
+ <LiveMetricsBoard
+ series={pinnedSeries.value}
+ windowMs={effectiveRealtimeWindowMs(realtimeWindowMs)}
+ emptyText={t('detail.liveMetrics.emptyPinned')}
+ height={140}
+ layout="row"
+ unitTitles={{
+ ratio: t('detail.liveMetrics.unitRatio'),
+ duration: t('detail.liveMetrics.unitDuration'),
+ count: t('detail.liveMetrics.unitCount')
+ }}
+ />
+ </div>
<NDataTable
columns={columns}
data={tableData.value}
@@ -630,22 +661,44 @@ export default defineComponent({
<NDrawerContent title={focusedEdgeInfo.value?.['edge.id']} closable>
<Configuration data={focusedEdgeInfo.value}></Configuration>
<NDivider />
- <NDataTable
- columns={edgePointColumns}
- data={focusedEdgeSeries.value}
- pagination={false}
- bordered
+ {renderPinControls(
+ 'edge',
+ focusedEdge.value.targetVertexId,
+ `${shortOperatorLabel(String(focusedEdgeInfo.value?.['edge.from']))} → ${shortOperatorLabel(String(focusedEdgeInfo.value?.['edge.to']))}`,
+ edgePinFields(t)
+ )}
+ <LiveMetricsBoard
+ series={drawerEdgeChartSeries.value}
+ windowMs={effectiveRealtimeWindowMs(realtimeWindowMs)}
+ emptyText={t('detail.liveMetrics.chartEmpty')}
+ height={180}
+ unitTitles={{
+ ratio: t('detail.liveMetrics.unitRatio'),
+ duration: t('detail.liveMetrics.unitDuration'),
+ count: t('detail.liveMetrics.unitCount')
+ }}
/>
</NDrawerContent>
) : (
<NDrawerContent title={focusedVertex.value?.vertexName} closable>
<Configuration data={focusedVertex.value}></Configuration>
<NDivider />
- <NDataTable
- columns={vertexPointColumns.value}
- data={focusedVertexSeries.value}
- pagination={false}
- bordered
+ {renderPinControls(
+ 'vertex',
+ focusedId.value,
+ focusedVertex.value?.vertexName || String(focusedId.value),
+ vertexPinFields((focusedVertex.value as any)?.type, t)
+ )}
+ <LiveMetricsBoard
+ series={drawerVertexChartSeries.value}
+ windowMs={effectiveRealtimeWindowMs(realtimeWindowMs)}
+ emptyText={t('detail.liveMetrics.chartEmpty')}
+ height={180}
+ unitTitles={{
+ ratio: t('detail.liveMetrics.unitRatio'),
+ duration: t('detail.liveMetrics.unitDuration'),
+ count: t('detail.liveMetrics.unitCount')
+ }}
/>
</NDrawerContent>
)}